using Pumas
using Pumas.Experimental: VEM
using PumasUtilities
using PharmaDatasets
using DataFramesMeta
using CairoMakie
using AlgebraOfGraphics
using SummaryTables
using Random: default_rng, seed!
seed!(123)Random.TaskLocalRNG()
VEM is an experimental estimation method introduced in Pumas 2.8. Its API may change in future releases without deprecation.
In this tutorial, we introduce Variational Expectation Maximization (VEM), a new estimation method available in Pumas.Experimental. We will cover the conceptual background, fit a warfarin PK model with VEM, compare the results to FOCE, and provide a comprehensive reference of VEM options.
using Pumas
using Pumas.Experimental: VEM
using PumasUtilities
using PharmaDatasets
using DataFramesMeta
using CairoMakie
using AlgebraOfGraphics
using SummaryTables
using Random: default_rng, seed!
seed!(123)Random.TaskLocalRNG()
In nonlinear mixed-effects (NLME) modeling, we want to estimate population parameters while accounting for between-subject variability through random effects. The marginal likelihood requires integrating over all possible random effect values for every subject, and these integrals are generally intractable.
Different estimation methods handle this challenge in different ways:
VEM works by maximizing the Evidence Lower Bound (ELBO), which is a lower bound on the log marginal likelihood. The ELBO equals the log-likelihood minus a KL divergence term that measures how far the approximate posterior is from the true posterior. A higher ELBO typically means a better fit.
Intuitively, VEM simultaneously finds:
This joint optimization is performed in a single level, unlike FOCE/LaplaceI which alternates between inner (random effects) and outer (population parameters) optimization loops.
One key advantage of VEM is that it can handle any response data distribution, including discrete, censored, truncated, time-to-event and even custom distributions. FOCE by comparison is limited to a restricted set of distributions, and while LaplaceI can handle any response distribution, it can be very slow for complex models.
VEM has two optimization variants controlled by the presample keyword:
Presampling (presample=true, default) |
Resampling (presample=false) |
|
|---|---|---|
| Optimizer | L-BFGS (deterministic) | Adam (stochastic) |
| Phases | 1 (default) | 10 (default) |
| Samples | Pre-drawn at start of phase, reused across iterations | Re-drawn at every iteration |
In the resampling variant, the number of samples increases (by default, unless overridden) and the learning rate of Adam decreases from one phase to the next.
VEM approximates each subject’s random effect posterior with a multivariate normal distribution. The diagonal keyword controls the structure of this approximation:
diagonal=true (default): assumes independent random effects in the posterior. Faster, fewer parameters to optimize.diagonal=false: captures correlations between random effects in the posterior. Slower but may be more accurate when random effects are correlated.We will fit an oral PKPD model to the warfarin dataset, first using VEM and then comparing with FOCE.
warfarin_model = @model begin
@param begin
# PK
pop_CL ∈ RealDomain(lower = 0.0)
pop_V ∈ RealDomain(lower = 0.0)
pop_tabs ∈ RealDomain(lower = 0.0)
pop_lag ∈ RealDomain(lower = 0.0, upper = 1.0)
pk_Ω ∈ PSDDomain(3)
σ_prop ∈ RealDomain(lower = 0.0)
σ_add ∈ RealDomain(lower = 0.0)
# PD
pop_e0 ∈ RealDomain(lower = 0.0)
pop_emax ∈ RealDomain()
pop_c50 ∈ RealDomain(lower = 0.0)
pop_tover ∈ RealDomain(lower = 0.0)
pd_Ω ∈ PSDDomain(3)
σ_fx ∈ RealDomain(lower = 0.0)
end
@random begin
pk_η ~ MvNormal(pk_Ω)
pd_η ~ MvNormal(pd_Ω)
end
@covariates begin
FSZV
FSZCL
end
@pre begin
# PK
CL = FSZCL * pop_CL * exp(pk_η[1])
Vc = FSZV * pop_V * exp(pk_η[2])
tabs = pop_tabs * exp(pk_η[3])
Ka = log(2) / tabs
# PD
e0 = pop_e0 * exp(pd_η[1])
emax = pop_emax
c50 = pop_c50 * exp(pd_η[2])
tover = pop_tover * exp(pd_η[3])
kout = log(2) / tover
rin = e0 * kout
end
@dosecontrol begin
lags = (Depot = pop_lag,)
end
@init begin
Turnover = e0
end
@vars begin
cp := Central / Vc
ratein := Ka * Depot
pd := 1 + emax * cp / (c50 + cp)
end
@dynamics begin
Depot' = -ratein
Central' = ratein - CL * cp
Turnover' = rin * pd - kout * Turnover
end
@derived begin
conc ~ @. CombinedNormal(cp, σ_add, σ_prop)
pca ~ @. Normal(Turnover, σ_fx)
end
endPumasModel
Parameters: pop_CL, pop_V, pop_tabs, pop_lag, pk_Ω, σ_prop, σ_add, pop_e0, pop_emax, pop_c50, pop_tover, pd_Ω, σ_fx
Random effects: pk_η, pd_η
Covariates: FSZV, FSZCL
Dynamical system variables: Depot, Central, Turnover
Dynamical system type: Nonlinear ODE
Derived: conc, pca
Observed: conc, pca
The @model definition is identical regardless of whether you fit with VEM or FOCE/LaplaceI. VEM works with standard @model definitions – no special model macro is required.
emax
Note that no random effect is used on emax. We tried adding a random effect on emax and estimating the model parameters. However, that resulted in the row and column of pd_Ω corresponding to that random effect to be all close to 0 which caused numerical issues for the asymptotic esimtator of the confidence interval. So no random effect is used on emax here for simplicity.
The model definition above gives each parameter’s domain but no values. Those are kept separately and passed to fit as the third argument.
warfarin_init = (;
# PK
pop_CL = 0.15,
pop_V = 10.0,
pop_tabs = 0.5,
pop_lag = 0.5,
pk_Ω = Diagonal([0.1, 0.1, 0.1]),
σ_prop = 0.1,
σ_add = 0.5,
# PD
pop_e0 = 100.0,
pop_emax = -1.0,
pop_c50 = 1.0,
pop_tover = 14.0,
pd_Ω = Diagonal([0.01, 0.01, 0.01]),
σ_fx = 0.01,
)(pop_CL = 0.15, pop_V = 10.0, pop_tabs = 0.5, pop_lag = 0.5, pk_Ω = Diagonal([0.1, 0.1, 0.1]), σ_prop = 0.1, σ_add = 0.5, pop_e0 = 100.0, pop_emax = -1.0, pop_c50 = 1.0, pop_tover = 14.0, pd_Ω = Diagonal([0.01, 0.01, 0.01]), σ_fx = 0.01)
These are starting values, not estimates. They are chosen to be plausible for warfarin and of the right order of magnitude, without being read off a previous analysis of these data. Published analyses of the warfarin data of (O’Reilly et al. 1963; O’Reilly and Aggeler 1968) report a clearance near 0.134 L/h/70kg and a central volume near 8.11 L/70kg (Holford 2010), which is the neighbourhood the fits below should reach.
warfarin_data = @chain dataset("pumas/warfarin_pumas") begin
@rtransform begin
:FSZV = :wtbl / 70
:FSZCL = (:wtbl / 70)^0.75
end
# Concentrations of exactly zero are below the limit of quantification
@rtransform @passmissing :conc = iszero(:conc) ? missing : :conc
end
first(warfarin_data, 5)| Row | id | time | evid | amt | cmt | conc | pca | wtbl | age | sex | FSZV | FSZCL |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Int64 | Float64 | Int64 | Float64? | Int64? | Float64? | Float64? | Float64 | Int64 | String1 | Float64 | Float64 | |
| 1 | 1 | 0.0 | 1 | 100.0 | 1 | missing | missing | 66.7 | 50 | M | 0.952857 | 0.96443 |
| 2 | 1 | 0.5 | 0 | missing | missing | missing | missing | 66.7 | 50 | M | 0.952857 | 0.96443 |
| 3 | 1 | 1.0 | 0 | missing | missing | 1.9 | missing | 66.7 | 50 | M | 0.952857 | 0.96443 |
| 4 | 1 | 2.0 | 0 | missing | missing | 3.3 | missing | 66.7 | 50 | M | 0.952857 | 0.96443 |
| 5 | 1 | 3.0 | 0 | missing | missing | 6.6 | missing | 66.7 | 50 | M | 0.952857 | 0.96443 |
pop = read_pumas(
warfarin_data;
id = :id,
time = :time,
amt = :amt,
cmt = :cmt,
evid = :evid,
covariates = [:FSZV, :FSZCL],
observations = [:conc, :pca],
)Population
Subjects: 32
Covariates: FSZV, FSZCL
Observations: conc, pca
Fitting with VEM follows the same fit interface as all other estimation methods. Simply pass VEM() as the estimation algorithm:
@time fpm_vem =
fit(warfarin_model, pop, warfarin_init, VEM(); optim_options = (; show_every = 20))[ Info: Checking the initial parameter values. [ Info: The initial negative log-likelihood and its gradient are finite. Check passed. Iter Function value Gradient norm 0 1.370837e+08 2.741671e+08 * time: 1.7881393432617188e-5 20 1.564026e+03 6.122502e+02 * time: 12.843366861343384 40 1.089103e+03 1.694481e+02 * time: 25.48479199409485 60 9.628931e+02 8.742054e+01 * time: 37.98119783401489 80 9.229682e+02 3.339154e+01 * time: 51.08288097381592 100 8.854265e+02 2.973702e+01 * time: 65.68621802330017 120 8.543415e+02 3.583795e+01 * time: 80.04285788536072 140 7.954205e+02 1.365657e+01 * time: 96.64965200424194 160 7.506251e+02 1.367600e+01 * time: 112.97022795677185 180 7.305785e+02 7.946752e+00 * time: 128.41657781600952 200 7.227177e+02 9.669863e+00 * time: 144.1189248561859 220 7.192623e+02 1.050782e+01 * time: 160.97357082366943 240 7.179981e+02 3.724778e+00 * time: 177.12614798545837 260 7.175690e+02 2.301297e+00 * time: 192.53336191177368 202.409365 seconds (216.35 M allocations: 49.708 GiB, 8.24% gc time, 1.65% compilation time)
FittedPumasModel
Dynamical system type: Nonlinear ODE
Solver(s): (OrdinaryDiffEqVerner.Vern7,OrdinaryDiffEqRosenbrock.Rodas5P)
Number of subjects: 32
Observation records: Active Missing
conc: 247 51
pca: 232 66
Total: 479 117
Number of parameters: Constant Optimized
0 23
Estimation method: VEM (EXPERIMENTAL)
-----------------------
Estimate
-----------------------
pop_CL 0.13453
pop_V 7.9978
pop_tabs 0.39644
pop_lag 0.88715
pk_Ω₁,₁ 0.066617
pk_Ω₂,₁ 0.0087813
pk_Ω₃,₁ 0.03865
pk_Ω₂,₂ 0.020336
pk_Ω₃,₂ -0.026128
pk_Ω₃,₃ 0.98922
σ_prop 0.086936
σ_add 0.40221
pop_e0 96.334
pop_emax -1.07
pop_c50 1.5399
pop_tover 14.123
pd_Ω₁,₁ 0.0030506
pd_Ω₂,₁ -0.0035378
pd_Ω₃,₁ 0.0012884
pd_Ω₂,₂ 0.15121
pd_Ω₃,₂ -0.033438
pd_Ω₃,₃ 0.013114
σ_fx 3.8386
-----------------------
During fitting, VEM displays a convergence trace showing the negative ELBO (the objective being minimized). The objective value is the negative of the lower bound on the log marginal likelihood, so lower values indicate a better fit.
Population parameter estimates can be extracted with coef:
simple_table(coeftable(fpm_vem))| parameter | description | constant | estimate |
| pop_CL | pop_CL | false | 0.135 |
| pop_V | pop_V | false | 8 |
| pop_tabs | pop_tabs | false | 0.396 |
| pop_lag | pop_lag | false | 0.887 |
| pk_Ω₁,₁ | pk_Ω₁,₁ | false | 0.0666 |
| pk_Ω₂,₁ | pk_Ω₂,₁ | false | 0.00878 |
| pk_Ω₃,₁ | pk_Ω₃,₁ | false | 0.0387 |
| pk_Ω₂,₂ | pk_Ω₂,₂ | false | 0.0203 |
| pk_Ω₃,₂ | pk_Ω₃,₂ | false | -0.0261 |
| pk_Ω₃,₃ | pk_Ω₃,₃ | false | 0.989 |
| σ_prop | σ_prop | false | 0.0869 |
| σ_add | σ_add | false | 0.402 |
| pop_e0 | pop_e0 | false | 96.3 |
| pop_emax | pop_emax | false | -1.07 |
| pop_c50 | pop_c50 | false | 1.54 |
| pop_tover | pop_tover | false | 14.1 |
| pd_Ω₁,₁ | pd_Ω₁,₁ | false | 0.00305 |
| pd_Ω₂,₁ | pd_Ω₂,₁ | false | -0.00354 |
| pd_Ω₃,₁ | pd_Ω₃,₁ | false | 0.00129 |
| pd_Ω₂,₂ | pd_Ω₂,₂ | false | 0.151 |
| pd_Ω₃,₂ | pd_Ω₃,₂ | false | -0.0334 |
| pd_Ω₃,₃ | pd_Ω₃,₃ | false | 0.0131 |
| σ_fx | σ_fx | false | 3.84 |
The posterior approximation modes of the random effects for each subject are available via empirical_bayes:
ebes = empirical_bayes(fpm_vem)
first(ebes, 3)3-element Vector{@NamedTuple{pk_η::Vector{Float64}, pd_η::Vector{Float64}}}:
(pk_η = [0.7633979507167724, 0.06262002422414326, 1.5556607656380148], pd_η = [-0.043885040897370386, 0.024512735415743975, 0.047430651899558286])
(pk_η = [0.028963819307562175, 0.05556058347354534, 0.49741869058327526], pd_η = [0.041023290535329354, 0.3523518822722693, -0.09258365993614884])
(pk_η = [-0.2969995728624792, -0.03351332562511915, -0.38595164249093505], pd_η = [0.012830929683162167, -0.6692621174224556, 0.08296529831994724])
Note that these generally may not correspond to modes of the true posterior.
VEM does not compute the log-likelihood during fitting. To evaluate the log-likelihood after fitting, you must specify an approximation method:
ll_foce = loglikelihood(fpm_vem, FOCE())-1062.7904574118725
When using FOCE() (or LaplaceI()) for log-likelihood evaluation, the empirical Bayes estimates (EBEs) are computed for each subject starting from the mode of the prior (i.e., zero random effects in the Gaussian case) rather than the VEM posterior approximation’s mode.
For well-behaved models, this is unlikely to cause a significant difference in the log-likelihood approximation. However, this behavior will likely change in a future release to use the VEM posterior approximation’s mode as the starting point for the Laplace approximation.
Model diagnostics are available through inspect. For models with Gaussian error models (like our proportional normal error), you must explicitly specify how to compute weighted residuals using wres_approx:
res = inspect(fpm_vem; wres_approx = FOCE())[ Info: Calculating predictions. [ Info: Calculating weighted residuals. [ Info: Calculating empirical bayes. [ Info: Evaluating dose control parameters. [ Info: Evaluating individual parameters. [ Info: Done.
FittedPumasModelInspection
Likelihood approximation used for weighted residuals: FOCE
When computing the weights of the weighted residuals, the linearization of the mean and variance are performed at the mode of the approximate posterior for each subject, learned by VEM, rather than the mode of the true posterior.
The predictions used for the residuals are also computed using the mode of the VEM posterior approximation, which may differ from the EBEs or the modes of the true posterior.
Goodness-of-fit plots work as other estimation methods in Pumas:
goodness_of_fit(res, observations = [:conc])goodness_of_fit(res, observations = [:pca])wres_approx Required for Gaussian Errors
When using VEM with Gaussian-like error models (Normal, ProportionalNormal, etc.), you must pass wres_approx = FOCE() or wres_approx = LaplaceI() to inspect. Omitting it will raise an error. For other error models (Beta, Gamma, Poisson, Bernoulli, TimeToEvent), this is not needed.
Predictions and visual predictive checks work as other estimation methods in Pumas:
preds = predict(fpm_vem)
first(preds, 3)Prediction
Subjects: 3
Predictions: conc, pca
Covariates: FSZV, FSZCL
The individual predictions are computed using the mode of the VEM posterior approximation for each subject, which may differ from the EBEs or the modes of the true posterior.
conc_vpc_res = vpc(fpm_vem, observations = [:conc]);vpc_plot(conc_vpc_res)pca_vpc_res = vpc(fpm_vem, observations = [:pca]);vpc_plot(pca_vpc_res)VEM does not currently support infer for computing standard errors and confidence intervals:
infer(fpm_vem)ArgumentError: Asymptotic inference is not supported for the estimation method VEM (EXPERIMENTAL). Stacktrace: [1] infer(fpm::Pumas.FittedPumasModel{Pumas.PumasModel{(pop_CL = 1, pop_V = 1, pop_tabs = 1, pop_lag = 1, pk_Ω = 6, σ_prop = 1, σ_add = 1, pop_e0 = 1, pop_emax = 1, pop_c50 = 1, pop_tover = 1, pd_Ω = 6, σ_fx = 1), 6, (:Depot, :Central, :Turnover), (:CL, :Vc, :tabs, :Ka, :e0, :emax, :c50, :tover, :kout, :rin), Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.ParamSet{@NamedTuple{pk_η::Pumas.RandomEffect{false, Main.Notebook.var"#13#14"}, pd_η::Pumas.RandomEffect{false, Main.Notebook.var"#15#16"}}}, Pumas.PreObj{Pumas.TimeDispatcher{Main.Notebook.var"#17#18", Main.Notebook.var"#19#20"}}, Pumas.DCPObj{Main.Notebook.var"#25#26"}, Main.Notebook.var"#27#28", SciMLBase.ODEProblem{Nothing, Tuple{Nothing, Nothing}, false, Nothing, SciMLBase.ODEFunction{false, SciMLBase.FullSpecialize, ModelingToolkitBase.GeneratedFunctionWrapper{(2, 3, false), Main.Notebook.var"#29#30", Main.Notebook.var"#31#32"}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, ModelingToolkitBase.ObservedFunctionCache{ModelingToolkitBase.System, Nothing}, Nothing, ModelingToolkitBase.System, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, Pumas.DerivedObj{(:conc, :pca), (:Ka, :emax, :Vc, :c50), (:Central, :Depot, :Turnover), (), Main.Notebook.var"#33#34"}, Main.Notebook.var"#35#36", ModelingToolkitBase.System, Pumas.PumasModelOptions{Pumas.AllSteadyStateStates}}, Vector{Pumas.Subject{@NamedTuple{conc::Vector{Union{Missing, Float64}}, pca::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{FSZV::Float64, FSZCL::Float64}}, Pumas.DosageRegimen{Float64, Union{Missing, Int64}, Float64, Float64, Float64, Float64}, Vector{Float64}}}, Pumas.Experimental.VEMResult{Pumas.Experimental.ThreadedVEMObj{true, false, true, true, Pumas.PumasModel{(pop_CL = 1, pop_V = 1, pop_tabs = 1, pop_lag = 1, pk_Ω = 6, σ_prop = 1, σ_add = 1, pop_e0 = 1, pop_emax = 1, pop_c50 = 1, pop_tover = 1, pd_Ω = 6, σ_fx = 1), 6, (:Depot, :Central, :Turnover), (:CL, :Vc, :tabs, :Ka, :e0, :emax, :c50, :tover, :kout, :rin), Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.ParamSet{@NamedTuple{pk_η::Pumas.RandomEffect{false, Main.Notebook.var"#13#14"}, pd_η::Pumas.RandomEffect{false, Main.Notebook.var"#15#16"}}}, Pumas.PreObj{Pumas.TimeDispatcher{Main.Notebook.var"#17#18", Main.Notebook.var"#19#20"}}, Pumas.DCPObj{Main.Notebook.var"#25#26"}, Main.Notebook.var"#27#28", SciMLBase.ODEProblem{Nothing, Tuple{Nothing, Nothing}, false, Nothing, SciMLBase.ODEFunction{false, SciMLBase.FullSpecialize, ModelingToolkitBase.GeneratedFunctionWrapper{(2, 3, false), Main.Notebook.var"#29#30", Main.Notebook.var"#31#32"}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, ModelingToolkitBase.ObservedFunctionCache{ModelingToolkitBase.System, Nothing}, Nothing, ModelingToolkitBase.System, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, Pumas.DerivedObj{(:conc, :pca), (:Ka, :emax, :Vc, :c50), (:Central, :Depot, :Turnover), (), Main.Notebook.var"#33#34"}, Main.Notebook.var"#35#36", ModelingToolkitBase.System, Pumas.PumasModelOptions{Pumas.AllSteadyStateStates}}, Vector{Pumas.Subject{@NamedTuple{conc::Vector{Union{Missing, Float64}}, pca::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{FSZV::Float64, FSZCL::Float64}}, Pumas.DosageRegimen{Float64, Union{Missing, Int64}, Float64, Float64, Float64, Float64}, Vector{Float64}}}, @NamedTuple{pop_CL::Float64, pop_V::Float64, pop_tabs::Float64, pop_lag::Float64, pk_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_prop::Float64, σ_add::Float64, pop_e0::Float64, pop_emax::Float64, pop_c50::Float64, pop_tover::Float64, pd_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_fx::Float64}, Random.TaskLocalRNG, Nothing, Nothing, @NamedTuple{ss_abstol::Float64, ss_reltol::Float64, ss_maxiters::Int64, abstol::Float64, reltol::Float64, alg::OrdinaryDiffEqCore.CompositeAlgorithm{0, Tuple{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}}, OrdinaryDiffEqCore.AutoSwitch{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}, Rational{Int64}, Int64}}, continuity::Symbol}, Vector{Array{Float64, 3}}, TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::TransformVariables.Constant{Nothing}, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::TransformVariables.Constant{Nothing}, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Vector{Vector{Float64}}, Vector{Vector{Float64}}, DifferentiationInterfaceForwardDiffExt.ForwardDiffGradientPrep{Tuple{Pumas._L_param{false, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}}, ADTypes.AutoForwardDiff{nothing, Nothing}, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, Tuple{}}, ForwardDiff.GradientConfig{ForwardDiff.Tag{Pumas._L_param{false, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}}, Float64}, Float64, 12, Vector{ForwardDiff.Dual{ForwardDiff.Tag{Pumas._L_param{false, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}}, Float64}, Float64, 12}}}, Tuple{}}, Vector{DifferentiationInterfaceReverseDiffExt.ReverseDiffGradientPrep{Tuple{typeof(Pumas.Experimental.G1_prior), ADTypes.AutoReverseDiff{false}, Vector{Float64}, Tuple{DifferentiationInterface.Constant{Pumas.Experimental._L_rfx2{true, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::TransformVariables.Constant{Nothing}, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::TransformVariables.Constant{Nothing}, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.PumasModel{(pop_CL = 1, pop_V = 1, pop_tabs = 1, pop_lag = 1, pk_Ω = 6, σ_prop = 1, σ_add = 1, pop_e0 = 1, pop_emax = 1, pop_c50 = 1, pop_tover = 1, pd_Ω = 6, σ_fx = 1), 6, (:Depot, :Central, :Turnover), (:CL, :Vc, :tabs, :Ka, :e0, :emax, :c50, :tover, :kout, :rin), Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.ParamSet{@NamedTuple{pk_η::Pumas.RandomEffect{false, Main.Notebook.var"#13#14"}, pd_η::Pumas.RandomEffect{false, Main.Notebook.var"#15#16"}}}, Pumas.PreObj{Pumas.TimeDispatcher{Main.Notebook.var"#17#18", Main.Notebook.var"#19#20"}}, Pumas.DCPObj{Main.Notebook.var"#25#26"}, Main.Notebook.var"#27#28", SciMLBase.ODEProblem{Nothing, Tuple{Nothing, Nothing}, false, Nothing, SciMLBase.ODEFunction{false, SciMLBase.FullSpecialize, ModelingToolkitBase.GeneratedFunctionWrapper{(2, 3, false), Main.Notebook.var"#29#30", Main.Notebook.var"#31#32"}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, ModelingToolkitBase.ObservedFunctionCache{ModelingToolkitBase.System, Nothing}, Nothing, ModelingToolkitBase.System, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, Pumas.DerivedObj{(:conc, :pca), (:Ka, :emax, :Vc, :c50), (:Central, :Depot, :Turnover), (), Main.Notebook.var"#33#34"}, Main.Notebook.var"#35#36", ModelingToolkitBase.System, Pumas.PumasModelOptions{Pumas.AllSteadyStateStates}}, Pumas.Subject{@NamedTuple{conc::Vector{Union{Missing, Float64}}, pca::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{FSZV::Float64, FSZCL::Float64}}, Pumas.DosageRegimen{Float64, Union{Missing, Int64}, Float64, Float64, Float64, Float64}, Vector{Float64}}, @NamedTuple{ss_abstol::Float64, ss_reltol::Float64, ss_maxiters::Int64, abstol::Float64, reltol::Float64, alg::OrdinaryDiffEqCore.CompositeAlgorithm{0, Tuple{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}}, OrdinaryDiffEqCore.AutoSwitch{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}, Rational{Int64}, Int64}}, continuity::Symbol}, Matrix{Float64}, @NamedTuple{pop_CL::Float64, pop_V::Float64, pop_tabs::Float64, pop_lag::Float64, pk_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_prop::Float64, σ_add::Float64, pop_e0::Float64, pop_emax::Float64, pop_c50::Float64, pop_tover::Float64, pd_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_fx::Float64}}}}}, ReverseDiff.GradientConfig{ReverseDiff.TrackedArray{Float64, Float64, 1, Vector{Float64}, Vector{Float64}}}, Nothing}}, Vector{DifferentiationInterfaceForwardDiffExt.ForwardDiffGradientPrep{Tuple{typeof(Pumas.Experimental.G1_likelihood), ADTypes.AutoForwardDiff{nothing, Nothing}, Vector{Float64}, Tuple{DifferentiationInterface.Constant{Pumas.Experimental._L_rfx2{true, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::TransformVariables.Constant{Nothing}, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::TransformVariables.Constant{Nothing}, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.PumasModel{(pop_CL = 1, pop_V = 1, pop_tabs = 1, pop_lag = 1, pk_Ω = 6, σ_prop = 1, σ_add = 1, pop_e0 = 1, pop_emax = 1, pop_c50 = 1, pop_tover = 1, pd_Ω = 6, σ_fx = 1), 6, (:Depot, :Central, :Turnover), (:CL, :Vc, :tabs, :Ka, :e0, :emax, :c50, :tover, :kout, :rin), Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.ParamSet{@NamedTuple{pk_η::Pumas.RandomEffect{false, Main.Notebook.var"#13#14"}, pd_η::Pumas.RandomEffect{false, Main.Notebook.var"#15#16"}}}, Pumas.PreObj{Pumas.TimeDispatcher{Main.Notebook.var"#17#18", Main.Notebook.var"#19#20"}}, Pumas.DCPObj{Main.Notebook.var"#25#26"}, Main.Notebook.var"#27#28", SciMLBase.ODEProblem{Nothing, Tuple{Nothing, Nothing}, false, Nothing, SciMLBase.ODEFunction{false, SciMLBase.FullSpecialize, ModelingToolkitBase.GeneratedFunctionWrapper{(2, 3, false), Main.Notebook.var"#29#30", Main.Notebook.var"#31#32"}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, ModelingToolkitBase.ObservedFunctionCache{ModelingToolkitBase.System, Nothing}, Nothing, ModelingToolkitBase.System, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, Pumas.DerivedObj{(:conc, :pca), (:Ka, :emax, :Vc, :c50), (:Central, :Depot, :Turnover), (), Main.Notebook.var"#33#34"}, Main.Notebook.var"#35#36", ModelingToolkitBase.System, Pumas.PumasModelOptions{Pumas.AllSteadyStateStates}}, Pumas.Subject{@NamedTuple{conc::Vector{Union{Missing, Float64}}, pca::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{FSZV::Float64, FSZCL::Float64}}, Pumas.DosageRegimen{Float64, Union{Missing, Int64}, Float64, Float64, Float64, Float64}, Vector{Float64}}, @NamedTuple{ss_abstol::Float64, ss_reltol::Float64, ss_maxiters::Int64, abstol::Float64, reltol::Float64, alg::OrdinaryDiffEqCore.CompositeAlgorithm{0, Tuple{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}}, OrdinaryDiffEqCore.AutoSwitch{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}, Rational{Int64}, Int64}}, continuity::Symbol}, Matrix{Float64}, @NamedTuple{pop_CL::Float64, pop_V::Float64, pop_tabs::Float64, pop_lag::Float64, pk_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_prop::Float64, σ_add::Float64, pop_e0::Float64, pop_emax::Float64, pop_c50::Float64, pop_tover::Float64, pd_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_fx::Float64}}}}}, ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(Pumas.Experimental.G1_likelihood), Float64}, Float64, 9, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(Pumas.Experimental.G1_likelihood), Float64}, Float64, 9}}}, Tuple{Nothing}}}, Vector{DifferentiationInterfaceReverseDiffExt.ReverseDiffTwoArgJacobianPrep{Tuple{typeof(Pumas.Experimental.get_randeffs!), SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, ADTypes.AutoReverseDiff{false}, Vector{Float64}, Tuple{DifferentiationInterface.Constant{Pumas.Experimental._L_rfx2{true, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::TransformVariables.Constant{Nothing}, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::TransformVariables.Constant{Nothing}, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.PumasModel{(pop_CL = 1, pop_V = 1, pop_tabs = 1, pop_lag = 1, pk_Ω = 6, σ_prop = 1, σ_add = 1, pop_e0 = 1, pop_emax = 1, pop_c50 = 1, pop_tover = 1, pd_Ω = 6, σ_fx = 1), 6, (:Depot, :Central, :Turnover), (:CL, :Vc, :tabs, :Ka, :e0, :emax, :c50, :tover, :kout, :rin), Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.ParamSet{@NamedTuple{pk_η::Pumas.RandomEffect{false, Main.Notebook.var"#13#14"}, pd_η::Pumas.RandomEffect{false, Main.Notebook.var"#15#16"}}}, Pumas.PreObj{Pumas.TimeDispatcher{Main.Notebook.var"#17#18", Main.Notebook.var"#19#20"}}, Pumas.DCPObj{Main.Notebook.var"#25#26"}, Main.Notebook.var"#27#28", SciMLBase.ODEProblem{Nothing, Tuple{Nothing, Nothing}, false, Nothing, SciMLBase.ODEFunction{false, SciMLBase.FullSpecialize, ModelingToolkitBase.GeneratedFunctionWrapper{(2, 3, false), Main.Notebook.var"#29#30", Main.Notebook.var"#31#32"}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, ModelingToolkitBase.ObservedFunctionCache{ModelingToolkitBase.System, Nothing}, Nothing, ModelingToolkitBase.System, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, Pumas.DerivedObj{(:conc, :pca), (:Ka, :emax, :Vc, :c50), (:Central, :Depot, :Turnover), (), Main.Notebook.var"#33#34"}, Main.Notebook.var"#35#36", ModelingToolkitBase.System, Pumas.PumasModelOptions{Pumas.AllSteadyStateStates}}, Pumas.Subject{@NamedTuple{conc::Vector{Union{Missing, Float64}}, pca::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{FSZV::Float64, FSZCL::Float64}}, Pumas.DosageRegimen{Float64, Union{Missing, Int64}, Float64, Float64, Float64, Float64}, Vector{Float64}}, @NamedTuple{ss_abstol::Float64, ss_reltol::Float64, ss_maxiters::Int64, abstol::Float64, reltol::Float64, alg::OrdinaryDiffEqCore.CompositeAlgorithm{0, Tuple{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}}, OrdinaryDiffEqCore.AutoSwitch{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}, Rational{Int64}, Int64}}, continuity::Symbol}, Matrix{Float64}, @NamedTuple{pop_CL::Float64, pop_V::Float64, pop_tabs::Float64, pop_lag::Float64, pk_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_prop::Float64, σ_add::Float64, pop_e0::Float64, pop_emax::Float64, pop_c50::Float64, pop_tover::Float64, pd_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_fx::Float64}}}}}, ReverseDiff.JacobianConfig{ReverseDiff.TrackedArray{Float64, Float64, 1, Vector{Float64}, Vector{Float64}}, Vector{ReverseDiff.TrackedReal{Float64, Float64, Nothing}}}, Nothing}}, ADTypes.AutoForwardDiff{nothing, Nothing}, ADTypes.AutoReverseDiff{false}, ADTypes.AutoForwardDiff{nothing, Nothing}, Vector{Matrix{Float64}}}, Optim.MultivariateOptimizationResults{Optim.LBFGS{Nothing, LineSearches.InitialStatic{Float64}, LineSearches.BackTracking{Float64, Int64}, Returns{Nothing}}, Vector{Float64}, Float64, Float64, Vector{Optim.OptimizationState{Float64, Optim.LBFGS{Nothing, LineSearches.InitialStatic{Float64}, LineSearches.BackTracking{Float64, Int64}, Returns{Nothing}}}}, @NamedTuple{x_converged::Bool, f_converged::Bool, g_converged::Bool, f_limit_reached::Bool, g_limit_reached::Bool, h_limit_reached::Bool, time_limit::Bool, callback::Bool, f_increased::Bool, ls_failed::Bool, iterations::Bool, small_trustregion_radius::Bool}}, Vector{Float64}}, Pumas.Experimental.VEM{ADTypes.AutoForwardDiff{nothing, Nothing}}, Vector{SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}}, @NamedTuple{diffeq_options::@NamedTuple{ss_abstol::Float64, ss_reltol::Float64, ss_maxiters::Int64, abstol::Float64, reltol::Float64, alg::OrdinaryDiffEqCore.CompositeAlgorithm{0, Tuple{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}}, OrdinaryDiffEqCore.AutoSwitch{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}, Rational{Int64}, Int64}}, continuity::Symbol}, ensemblealg::SciMLBase.EnsembleThreads, constantcoef::Tuple{}}, Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.OptimState{NLSolversBase.OnceDifferentiable{Float64, Vector{Float64}, Float64, Vector{Float64}}, Optim.LBFGSState{Vector{Float64}, Vector{Vector{Float64}}, Vector{Vector{Float64}}, Float64, Vector{Float64}}, Optim.Options{Float64, Nothing}}}; level::Float64, rethrow_error::Bool, ensemblealg::SciMLBase.EnsembleThreads, sandwich_estimator::Bool) @ Pumas.Experimental ~/run/_work/PumasTutorials.jl/PumasTutorials.jl/custom_julia_depot/packages/Pumas/btqOJ/src/estimation/vem.jl:2588 [2] infer(fpm::Pumas.FittedPumasModel{Pumas.PumasModel{(pop_CL = 1, pop_V = 1, pop_tabs = 1, pop_lag = 1, pk_Ω = 6, σ_prop = 1, σ_add = 1, pop_e0 = 1, pop_emax = 1, pop_c50 = 1, pop_tover = 1, pd_Ω = 6, σ_fx = 1), 6, (:Depot, :Central, :Turnover), (:CL, :Vc, :tabs, :Ka, :e0, :emax, :c50, :tover, :kout, :rin), Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.ParamSet{@NamedTuple{pk_η::Pumas.RandomEffect{false, Main.Notebook.var"#13#14"}, pd_η::Pumas.RandomEffect{false, Main.Notebook.var"#15#16"}}}, Pumas.PreObj{Pumas.TimeDispatcher{Main.Notebook.var"#17#18", Main.Notebook.var"#19#20"}}, Pumas.DCPObj{Main.Notebook.var"#25#26"}, Main.Notebook.var"#27#28", SciMLBase.ODEProblem{Nothing, Tuple{Nothing, Nothing}, false, Nothing, SciMLBase.ODEFunction{false, SciMLBase.FullSpecialize, ModelingToolkitBase.GeneratedFunctionWrapper{(2, 3, false), Main.Notebook.var"#29#30", Main.Notebook.var"#31#32"}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, ModelingToolkitBase.ObservedFunctionCache{ModelingToolkitBase.System, Nothing}, Nothing, ModelingToolkitBase.System, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, Pumas.DerivedObj{(:conc, :pca), (:Ka, :emax, :Vc, :c50), (:Central, :Depot, :Turnover), (), Main.Notebook.var"#33#34"}, Main.Notebook.var"#35#36", ModelingToolkitBase.System, Pumas.PumasModelOptions{Pumas.AllSteadyStateStates}}, Vector{Pumas.Subject{@NamedTuple{conc::Vector{Union{Missing, Float64}}, pca::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{FSZV::Float64, FSZCL::Float64}}, Pumas.DosageRegimen{Float64, Union{Missing, Int64}, Float64, Float64, Float64, Float64}, Vector{Float64}}}, Pumas.Experimental.VEMResult{Pumas.Experimental.ThreadedVEMObj{true, false, true, true, Pumas.PumasModel{(pop_CL = 1, pop_V = 1, pop_tabs = 1, pop_lag = 1, pk_Ω = 6, σ_prop = 1, σ_add = 1, pop_e0 = 1, pop_emax = 1, pop_c50 = 1, pop_tover = 1, pd_Ω = 6, σ_fx = 1), 6, (:Depot, :Central, :Turnover), (:CL, :Vc, :tabs, :Ka, :e0, :emax, :c50, :tover, :kout, :rin), Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.ParamSet{@NamedTuple{pk_η::Pumas.RandomEffect{false, Main.Notebook.var"#13#14"}, pd_η::Pumas.RandomEffect{false, Main.Notebook.var"#15#16"}}}, Pumas.PreObj{Pumas.TimeDispatcher{Main.Notebook.var"#17#18", Main.Notebook.var"#19#20"}}, Pumas.DCPObj{Main.Notebook.var"#25#26"}, Main.Notebook.var"#27#28", SciMLBase.ODEProblem{Nothing, Tuple{Nothing, Nothing}, false, Nothing, SciMLBase.ODEFunction{false, SciMLBase.FullSpecialize, ModelingToolkitBase.GeneratedFunctionWrapper{(2, 3, false), Main.Notebook.var"#29#30", Main.Notebook.var"#31#32"}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, ModelingToolkitBase.ObservedFunctionCache{ModelingToolkitBase.System, Nothing}, Nothing, ModelingToolkitBase.System, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, Pumas.DerivedObj{(:conc, :pca), (:Ka, :emax, :Vc, :c50), (:Central, :Depot, :Turnover), (), Main.Notebook.var"#33#34"}, Main.Notebook.var"#35#36", ModelingToolkitBase.System, Pumas.PumasModelOptions{Pumas.AllSteadyStateStates}}, Vector{Pumas.Subject{@NamedTuple{conc::Vector{Union{Missing, Float64}}, pca::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{FSZV::Float64, FSZCL::Float64}}, Pumas.DosageRegimen{Float64, Union{Missing, Int64}, Float64, Float64, Float64, Float64}, Vector{Float64}}}, @NamedTuple{pop_CL::Float64, pop_V::Float64, pop_tabs::Float64, pop_lag::Float64, pk_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_prop::Float64, σ_add::Float64, pop_e0::Float64, pop_emax::Float64, pop_c50::Float64, pop_tover::Float64, pd_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_fx::Float64}, Random.TaskLocalRNG, Nothing, Nothing, @NamedTuple{ss_abstol::Float64, ss_reltol::Float64, ss_maxiters::Int64, abstol::Float64, reltol::Float64, alg::OrdinaryDiffEqCore.CompositeAlgorithm{0, Tuple{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}}, OrdinaryDiffEqCore.AutoSwitch{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}, Rational{Int64}, Int64}}, continuity::Symbol}, Vector{Array{Float64, 3}}, TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::TransformVariables.Constant{Nothing}, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::TransformVariables.Constant{Nothing}, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Vector{Vector{Float64}}, Vector{Vector{Float64}}, DifferentiationInterfaceForwardDiffExt.ForwardDiffGradientPrep{Tuple{Pumas._L_param{false, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}}, ADTypes.AutoForwardDiff{nothing, Nothing}, SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, Tuple{}}, ForwardDiff.GradientConfig{ForwardDiff.Tag{Pumas._L_param{false, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}}, Float64}, Float64, 12, Vector{ForwardDiff.Dual{ForwardDiff.Tag{Pumas._L_param{false, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}}, Float64}, Float64, 12}}}, Tuple{}}, Vector{DifferentiationInterfaceReverseDiffExt.ReverseDiffGradientPrep{Tuple{typeof(Pumas.Experimental.G1_prior), ADTypes.AutoReverseDiff{false}, Vector{Float64}, Tuple{DifferentiationInterface.Constant{Pumas.Experimental._L_rfx2{true, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::TransformVariables.Constant{Nothing}, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::TransformVariables.Constant{Nothing}, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.PumasModel{(pop_CL = 1, pop_V = 1, pop_tabs = 1, pop_lag = 1, pk_Ω = 6, σ_prop = 1, σ_add = 1, pop_e0 = 1, pop_emax = 1, pop_c50 = 1, pop_tover = 1, pd_Ω = 6, σ_fx = 1), 6, (:Depot, :Central, :Turnover), (:CL, :Vc, :tabs, :Ka, :e0, :emax, :c50, :tover, :kout, :rin), Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.ParamSet{@NamedTuple{pk_η::Pumas.RandomEffect{false, Main.Notebook.var"#13#14"}, pd_η::Pumas.RandomEffect{false, Main.Notebook.var"#15#16"}}}, Pumas.PreObj{Pumas.TimeDispatcher{Main.Notebook.var"#17#18", Main.Notebook.var"#19#20"}}, Pumas.DCPObj{Main.Notebook.var"#25#26"}, Main.Notebook.var"#27#28", SciMLBase.ODEProblem{Nothing, Tuple{Nothing, Nothing}, false, Nothing, SciMLBase.ODEFunction{false, SciMLBase.FullSpecialize, ModelingToolkitBase.GeneratedFunctionWrapper{(2, 3, false), Main.Notebook.var"#29#30", Main.Notebook.var"#31#32"}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, ModelingToolkitBase.ObservedFunctionCache{ModelingToolkitBase.System, Nothing}, Nothing, ModelingToolkitBase.System, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, Pumas.DerivedObj{(:conc, :pca), (:Ka, :emax, :Vc, :c50), (:Central, :Depot, :Turnover), (), Main.Notebook.var"#33#34"}, Main.Notebook.var"#35#36", ModelingToolkitBase.System, Pumas.PumasModelOptions{Pumas.AllSteadyStateStates}}, Pumas.Subject{@NamedTuple{conc::Vector{Union{Missing, Float64}}, pca::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{FSZV::Float64, FSZCL::Float64}}, Pumas.DosageRegimen{Float64, Union{Missing, Int64}, Float64, Float64, Float64, Float64}, Vector{Float64}}, @NamedTuple{ss_abstol::Float64, ss_reltol::Float64, ss_maxiters::Int64, abstol::Float64, reltol::Float64, alg::OrdinaryDiffEqCore.CompositeAlgorithm{0, Tuple{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}}, OrdinaryDiffEqCore.AutoSwitch{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}, Rational{Int64}, Int64}}, continuity::Symbol}, Matrix{Float64}, @NamedTuple{pop_CL::Float64, pop_V::Float64, pop_tabs::Float64, pop_lag::Float64, pk_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_prop::Float64, σ_add::Float64, pop_e0::Float64, pop_emax::Float64, pop_c50::Float64, pop_tover::Float64, pd_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_fx::Float64}}}}}, ReverseDiff.GradientConfig{ReverseDiff.TrackedArray{Float64, Float64, 1, Vector{Float64}, Vector{Float64}}}, Nothing}}, Vector{DifferentiationInterfaceForwardDiffExt.ForwardDiffGradientPrep{Tuple{typeof(Pumas.Experimental.G1_likelihood), ADTypes.AutoForwardDiff{nothing, Nothing}, Vector{Float64}, Tuple{DifferentiationInterface.Constant{Pumas.Experimental._L_rfx2{true, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::TransformVariables.Constant{Nothing}, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::TransformVariables.Constant{Nothing}, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.PumasModel{(pop_CL = 1, pop_V = 1, pop_tabs = 1, pop_lag = 1, pk_Ω = 6, σ_prop = 1, σ_add = 1, pop_e0 = 1, pop_emax = 1, pop_c50 = 1, pop_tover = 1, pd_Ω = 6, σ_fx = 1), 6, (:Depot, :Central, :Turnover), (:CL, :Vc, :tabs, :Ka, :e0, :emax, :c50, :tover, :kout, :rin), Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.ParamSet{@NamedTuple{pk_η::Pumas.RandomEffect{false, Main.Notebook.var"#13#14"}, pd_η::Pumas.RandomEffect{false, Main.Notebook.var"#15#16"}}}, Pumas.PreObj{Pumas.TimeDispatcher{Main.Notebook.var"#17#18", Main.Notebook.var"#19#20"}}, Pumas.DCPObj{Main.Notebook.var"#25#26"}, Main.Notebook.var"#27#28", SciMLBase.ODEProblem{Nothing, Tuple{Nothing, Nothing}, false, Nothing, SciMLBase.ODEFunction{false, SciMLBase.FullSpecialize, ModelingToolkitBase.GeneratedFunctionWrapper{(2, 3, false), Main.Notebook.var"#29#30", Main.Notebook.var"#31#32"}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, ModelingToolkitBase.ObservedFunctionCache{ModelingToolkitBase.System, Nothing}, Nothing, ModelingToolkitBase.System, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, Pumas.DerivedObj{(:conc, :pca), (:Ka, :emax, :Vc, :c50), (:Central, :Depot, :Turnover), (), Main.Notebook.var"#33#34"}, Main.Notebook.var"#35#36", ModelingToolkitBase.System, Pumas.PumasModelOptions{Pumas.AllSteadyStateStates}}, Pumas.Subject{@NamedTuple{conc::Vector{Union{Missing, Float64}}, pca::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{FSZV::Float64, FSZCL::Float64}}, Pumas.DosageRegimen{Float64, Union{Missing, Int64}, Float64, Float64, Float64, Float64}, Vector{Float64}}, @NamedTuple{ss_abstol::Float64, ss_reltol::Float64, ss_maxiters::Int64, abstol::Float64, reltol::Float64, alg::OrdinaryDiffEqCore.CompositeAlgorithm{0, Tuple{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}}, OrdinaryDiffEqCore.AutoSwitch{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}, Rational{Int64}, Int64}}, continuity::Symbol}, Matrix{Float64}, @NamedTuple{pop_CL::Float64, pop_V::Float64, pop_tabs::Float64, pop_lag::Float64, pk_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_prop::Float64, σ_add::Float64, pop_e0::Float64, pop_emax::Float64, pop_c50::Float64, pop_tover::Float64, pd_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_fx::Float64}}}}}, ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(Pumas.Experimental.G1_likelihood), Float64}, Float64, 9, Vector{ForwardDiff.Dual{ForwardDiff.Tag{typeof(Pumas.Experimental.G1_likelihood), Float64}, Float64, 9}}}, Tuple{Nothing}}}, Vector{DifferentiationInterfaceReverseDiffExt.ReverseDiffTwoArgJacobianPrep{Tuple{typeof(Pumas.Experimental.get_randeffs!), SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}, ADTypes.AutoReverseDiff{false}, Vector{Float64}, Tuple{DifferentiationInterface.Constant{Pumas.Experimental._L_rfx2{true, (:pop_CL, :pop_V, :pop_tabs, :pop_lag, :pk_Ω, :σ_prop, :σ_add, :pop_e0, :pop_emax, :pop_c50, :pop_tover, :pd_Ω, :σ_fx), TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::Pumas.PSDTransform, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::Pumas.PSDTransform, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, TransformVariables.TransformTuple{@NamedTuple{pop_CL::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_V::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tabs::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_lag::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVScale{Float64}, TransformVariables.TVLogistic}}, pk_Ω::TransformVariables.Constant{Nothing}, σ_prop::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, σ_add::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_e0::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_emax::TransformVariables.Identity, pop_c50::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pop_tover::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}, pd_Ω::TransformVariables.Constant{Nothing}, σ_fx::TransformVariables.CompositeScalarTransform{Tuple{TransformVariables.TVShift{Float64}, TransformVariables.TVExp}}}}, Pumas.PumasModel{(pop_CL = 1, pop_V = 1, pop_tabs = 1, pop_lag = 1, pk_Ω = 6, σ_prop = 1, σ_add = 1, pop_e0 = 1, pop_emax = 1, pop_c50 = 1, pop_tover = 1, pd_Ω = 6, σ_fx = 1), 6, (:Depot, :Central, :Turnover), (:CL, :Vc, :tabs, :Ka, :e0, :emax, :c50, :tover, :kout, :rin), Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.ParamSet{@NamedTuple{pk_η::Pumas.RandomEffect{false, Main.Notebook.var"#13#14"}, pd_η::Pumas.RandomEffect{false, Main.Notebook.var"#15#16"}}}, Pumas.PreObj{Pumas.TimeDispatcher{Main.Notebook.var"#17#18", Main.Notebook.var"#19#20"}}, Pumas.DCPObj{Main.Notebook.var"#25#26"}, Main.Notebook.var"#27#28", SciMLBase.ODEProblem{Nothing, Tuple{Nothing, Nothing}, false, Nothing, SciMLBase.ODEFunction{false, SciMLBase.FullSpecialize, ModelingToolkitBase.GeneratedFunctionWrapper{(2, 3, false), Main.Notebook.var"#29#30", Main.Notebook.var"#31#32"}, LinearAlgebra.UniformScaling{Bool}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, ModelingToolkitBase.ObservedFunctionCache{ModelingToolkitBase.System, Nothing}, Nothing, ModelingToolkitBase.System, Nothing, Nothing}, @Kwargs{}, SciMLBase.StandardODEProblem}, Pumas.DerivedObj{(:conc, :pca), (:Ka, :emax, :Vc, :c50), (:Central, :Depot, :Turnover), (), Main.Notebook.var"#33#34"}, Main.Notebook.var"#35#36", ModelingToolkitBase.System, Pumas.PumasModelOptions{Pumas.AllSteadyStateStates}}, Pumas.Subject{@NamedTuple{conc::Vector{Union{Missing, Float64}}, pca::Vector{Union{Missing, Float64}}}, Pumas.ConstantCovar{@NamedTuple{FSZV::Float64, FSZCL::Float64}}, Pumas.DosageRegimen{Float64, Union{Missing, Int64}, Float64, Float64, Float64, Float64}, Vector{Float64}}, @NamedTuple{ss_abstol::Float64, ss_reltol::Float64, ss_maxiters::Int64, abstol::Float64, reltol::Float64, alg::OrdinaryDiffEqCore.CompositeAlgorithm{0, Tuple{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}}, OrdinaryDiffEqCore.AutoSwitch{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}, Rational{Int64}, Int64}}, continuity::Symbol}, Matrix{Float64}, @NamedTuple{pop_CL::Float64, pop_V::Float64, pop_tabs::Float64, pop_lag::Float64, pk_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_prop::Float64, σ_add::Float64, pop_e0::Float64, pop_emax::Float64, pop_c50::Float64, pop_tover::Float64, pd_Ω::PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}, σ_fx::Float64}}}}}, ReverseDiff.JacobianConfig{ReverseDiff.TrackedArray{Float64, Float64, 1, Vector{Float64}, Vector{Float64}}, Vector{ReverseDiff.TrackedReal{Float64, Float64, Nothing}}}, Nothing}}, ADTypes.AutoForwardDiff{nothing, Nothing}, ADTypes.AutoReverseDiff{false}, ADTypes.AutoForwardDiff{nothing, Nothing}, Vector{Matrix{Float64}}}, Optim.MultivariateOptimizationResults{Optim.LBFGS{Nothing, LineSearches.InitialStatic{Float64}, LineSearches.BackTracking{Float64, Int64}, Returns{Nothing}}, Vector{Float64}, Float64, Float64, Vector{Optim.OptimizationState{Float64, Optim.LBFGS{Nothing, LineSearches.InitialStatic{Float64}, LineSearches.BackTracking{Float64, Int64}, Returns{Nothing}}}}, @NamedTuple{x_converged::Bool, f_converged::Bool, g_converged::Bool, f_limit_reached::Bool, g_limit_reached::Bool, h_limit_reached::Bool, time_limit::Bool, callback::Bool, f_increased::Bool, ls_failed::Bool, iterations::Bool, small_trustregion_radius::Bool}}, Vector{Float64}}, Pumas.Experimental.VEM{ADTypes.AutoForwardDiff{nothing, Nothing}}, Vector{SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}}, @NamedTuple{diffeq_options::@NamedTuple{ss_abstol::Float64, ss_reltol::Float64, ss_maxiters::Int64, abstol::Float64, reltol::Float64, alg::OrdinaryDiffEqCore.CompositeAlgorithm{0, Tuple{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}}, OrdinaryDiffEqCore.AutoSwitch{OrdinaryDiffEqVerner.Vern7{typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!), Static.False, Val{true}}, OrdinaryDiffEqRosenbrock.Rodas5P{1, ADTypes.AutoForwardDiff{1, Nothing}, LinearSolve.LUFactorization{LinearAlgebra.RowMaximum}, typeof(OrdinaryDiffEqCore.DEFAULT_PRECS), Val{:forward}(), true, nothing, typeof(OrdinaryDiffEqCore.trivial_limiter!), typeof(OrdinaryDiffEqCore.trivial_limiter!)}, Rational{Int64}, Int64}}, continuity::Symbol}, ensemblealg::SciMLBase.EnsembleThreads, constantcoef::Tuple{}}, Pumas.ParamSet{@NamedTuple{pop_CL::Pumas.RealDomain{Float64, Nothing, Float64}, pop_V::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tabs::Pumas.RealDomain{Float64, Nothing, Float64}, pop_lag::Pumas.RealDomain{Float64, Float64, Float64}, pk_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_prop::Pumas.RealDomain{Float64, Nothing, Float64}, σ_add::Pumas.RealDomain{Float64, Nothing, Float64}, pop_e0::Pumas.RealDomain{Float64, Nothing, Float64}, pop_emax::Pumas.RealDomain{Nothing, Nothing, Float64}, pop_c50::Pumas.RealDomain{Float64, Nothing, Float64}, pop_tover::Pumas.RealDomain{Float64, Nothing, Float64}, pd_Ω::Pumas.PSDDomain{PDMats.PDMat{Float64, LinearAlgebra.Symmetric{Float64, Matrix{Float64}}, LinearAlgebra.Cholesky{Float64, Matrix{Float64}}}}, σ_fx::Pumas.RealDomain{Float64, Nothing, Float64}}}, Pumas.OptimState{NLSolversBase.OnceDifferentiable{Float64, Vector{Float64}, Float64, Vector{Float64}}, Optim.LBFGSState{Vector{Float64}, Vector{Vector{Float64}}, Vector{Vector{Float64}}, Float64, Vector{Float64}}, Optim.Options{Float64, Nothing}}}) @ Pumas.Experimental ~/run/_work/PumasTutorials.jl/PumasTutorials.jl/custom_julia_depot/packages/Pumas/btqOJ/src/estimation/vem.jl:2581 [3] top-level scope @ ~/run/_work/PumasTutorials.jl/PumasTutorials.jl/tutorials/estimation/01-vem-introduction.qmd:436
The recommended workaround is to use the VEM estimates as a starting point for a FOCE/LaplaceI fit, then run inference on the FOCE/LaplaceI result. This typically converges very quickly since VEM provides excellent initial values:
@time fpm_foce_from_vem = fit(
warfarin_model,
pop,
coef(fpm_vem),
FOCE();
init_randeffs = empirical_bayes(fpm_vem),
optim_options = (; show_every = 20),
)Iter Function value Gradient norm
0 1.062790e+03 1.792981e+01
* time: 9.107589721679688e-5
20 1.060478e+03 5.650826e-02
* time: 35.07386016845703
41.588715 seconds (18.25 M allocations: 6.916 GiB, 4.93% gc time, 2.01% compilation time)
FittedPumasModel
Dynamical system type: Nonlinear ODE
Solver(s): (OrdinaryDiffEqVerner.Vern7,OrdinaryDiffEqRosenbrock.Rodas5P)
Number of subjects: 32
Observation records: Active Missing
conc: 247 51
pca: 232 66
Total: 479 117
Number of parameters: Constant Optimized
0 23
Likelihood approximation: FOCE
Likelihood optimizer: BFGS
Termination Reason: GradientNorm
Log-likelihood value: -1060.4784
-----------------------
Estimate
-----------------------
pop_CL 0.13502
pop_V 8.0302
pop_tabs 0.53613
pop_lag 0.87911
pk_Ω₁,₁ 0.068383
pk_Ω₂,₁ 0.0078911
pk_Ω₃,₁ 0.032845
pk_Ω₂,₂ 0.021011
pk_Ω₃,₂ -0.0037261
pk_Ω₃,₃ 0.84876
σ_prop 0.086852
σ_add 0.43139
pop_e0 96.362
pop_emax -1.0874
pop_c50 1.629
pop_tover 14.421
pd_Ω₁,₁ 0.0028558
pd_Ω₂,₁ 0.0017457
pd_Ω₃,₁ 0.0005316
pd_Ω₂,₂ 0.14308
pd_Ω₃,₂ -0.03598
pd_Ω₃,₃ 0.017667
σ_fx 3.5643
-----------------------
loglikelihood(fpm_foce_from_vem)-1060.4784281340214
infer(fpm_foce_from_vem)[ Info: Calculating: variance-covariance matrix. [ Info: Done.
Asymptotic inference results using sandwich estimator
Dynamical system type: Nonlinear ODE
Solver(s): (OrdinaryDiffEqVerner.Vern7,OrdinaryDiffEqRosenbrock.Rodas5P)
Number of subjects: 32
Observation records: Active Missing
conc: 247 51
pca: 232 66
Total: 479 117
Number of parameters: Constant Optimized
0 23
Likelihood approximation: FOCE
Likelihood optimizer: BFGS
Termination Reason: GradientNorm
Log-likelihood value: -1060.4784
----------------------------------------------------------------
Estimate SE 95.0% C.I.
----------------------------------------------------------------
pop_CL 0.13502 0.0066431 [ 0.122 ; 0.14804 ]
pop_V 8.0302 0.21531 [ 7.6082 ; 8.4522 ]
pop_tabs 0.53613 0.17456 [ 0.194 ; 0.87826 ]
pop_lag 0.87911 0.053534 [ 0.77419 ; 0.98404 ]
pk_Ω₁,₁ 0.068383 0.024782 [ 0.01981 ; 0.11696 ]
pk_Ω₂,₁ 0.0078911 0.0072278 [ -0.0062751 ; 0.022057 ]
pk_Ω₃,₁ 0.032845 0.055674 [ -0.076274 ; 0.14196 ]
pk_Ω₂,₂ 0.021011 0.0063455 [ 0.008574 ; 0.033448 ]
pk_Ω₃,₂ -0.0037261 0.059504 [ -0.12035 ; 0.1129 ]
pk_Ω₃,₃ 0.84876 0.3884 [ 0.087499 ; 1.61 ]
σ_prop 0.086852 0.013681 [ 0.060037 ; 0.11367 ]
σ_add 0.43139 0.072076 [ 0.29013 ; 0.57266 ]
pop_e0 96.362 1.1767 [ 94.055 ; 98.668 ]
pop_emax -1.0874 0.045932 [ -1.1775 ; -0.9974 ]
pop_c50 1.629 0.32997 [ 0.98228 ; 2.2757 ]
pop_tover 14.421 0.76654 [ 12.918 ; 15.923 ]
pd_Ω₁,₁ 0.0028558 0.0010026 [ 0.00089064; 0.0048209]
pd_Ω₂,₁ 0.0017457 0.0037382 [ -0.005581 ; 0.0090724]
pd_Ω₃,₁ 0.0005316 0.0022352 [ -0.0038494 ; 0.0049126]
pd_Ω₂,₂ 0.14308 0.043295 [ 0.05822 ; 0.22793 ]
pd_Ω₃,₂ -0.03598 0.014257 [ -0.063923 ; -0.0080377]
pd_Ω₃,₃ 0.017667 0.0092047 [ -0.00037388; 0.035708 ]
σ_fx 3.5643 0.30609 [ 2.9644 ; 4.1643 ]
----------------------------------------------------------------
Initializing a FOCE fit with VEM estimates and running for 0 or more iterations is also a way to get a fitted Pumas model with standard EBEs (true posterior mode, instead of the VEM posterior approximation’s mode) that can be used for downstream functions, such as predict, inspect and loglikelihood.
Using VEM to initialize FOCE is a powerful workflow. VEM provides good population parameter estimates and per-subject random effect posteriors, which together allow FOCE to converge quickly and reliably.
Let us also fit the model directly with FOCE from the starting values above for comparison:
@time fpm_foce_direct =
fit(warfarin_model, pop, warfarin_init, FOCE(); optim_options = (; show_every = 20))Iter Function value Gradient norm
0 5.000726e+06 9.991374e+06
* time: 0.00011396408081054688
20 1.712871e+03 2.628079e+02
* time: 29.26493000984192
40 1.226181e+03 1.772134e+02
* time: 54.97340703010559
60 1.189379e+03 8.230921e+01
* time: 80.97688698768616
80 1.162811e+03 8.317623e+01
* time: 106.51585507392883
100 1.144415e+03 3.683253e+01
* time: 133.15930199623108
120 1.134098e+03 2.054584e+01
* time: 157.86574697494507
140 1.125687e+03 5.096888e+01
* time: 183.51366090774536
160 1.118357e+03 3.044155e+01
* time: 208.34813690185547
180 1.111687e+03 3.780514e+01
* time: 232.02583193778992
200 1.109201e+03 1.197017e+01
* time: 257.68543696403503
220 1.108590e+03 1.388418e+00
* time: 286.6743609905243
240 1.108491e+03 1.280312e+00
* time: 311.20018100738525
260 1.108438e+03 7.018229e-01
* time: 334.74742698669434
280 1.108414e+03 2.499001e-01
* time: 357.8505289554596
300 1.108407e+03 5.740849e-02
* time: 379.0789279937744
320 1.108407e+03 2.046286e-02
* time: 400.0554609298706
340 1.108407e+03 7.750814e-03
* time: 420.53369402885437
441.745753 seconds (132.26 M allocations: 65.558 GiB, 5.60% gc time, 0.04% compilation time)
FittedPumasModel
Dynamical system type: Nonlinear ODE
Solver(s): (OrdinaryDiffEqVerner.Vern7,OrdinaryDiffEqRosenbrock.Rodas5P)
Number of subjects: 32
Observation records: Active Missing
conc: 247 51
pca: 232 66
Total: 479 117
Number of parameters: Constant Optimized
0 23
Likelihood approximation: FOCE
Likelihood optimizer: BFGS
Termination Reason: GradientNorm
Log-likelihood value: -1108.407
------------------------
Estimate
------------------------
pop_CL 0.13506
pop_V 7.1456
pop_tabs 0.61907
pop_lag 0.87233
pk_Ω₁,₁ 0.067995
pk_Ω₂,₁ 0.028048
pk_Ω₃,₁ 0.010207
pk_Ω₂,₂ 0.46439
pk_Ω₃,₂ -0.52773
pk_Ω₃,₃ 1.4127
σ_prop 0.088233
σ_add 0.44546
pop_e0 96.345
pop_emax -1.0865
pop_c50 1.6247
pop_tover 14.406
pd_Ω₁,₁ 0.0028373
pd_Ω₂,₁ 0.0016626
pd_Ω₃,₁ 0.00055343
pd_Ω₂,₂ 0.13759
pd_Ω₃,₂ -0.033347
pd_Ω₃,₃ 0.016697
σ_fx 3.5228
------------------------
loglikelihood(fpm_foce_direct)-1108.4070366184428
compare_estimates(; VEM = fpm_vem, VEM_FOCE = fpm_foce_from_vem, FOCE = fpm_foce_direct)| Row | parameter | VEM | VEM_FOCE | FOCE |
|---|---|---|---|---|
| String | Float64? | Float64? | Float64? | |
| 1 | pop_CL | 0.13453 | 0.135019 | 0.135055 |
| 2 | pop_V | 7.99776 | 8.03022 | 7.14558 |
| 3 | pop_tabs | 0.396436 | 0.536134 | 0.619069 |
| 4 | pop_lag | 0.88715 | 0.879113 | 0.872328 |
| 5 | pk_Ω₁,₁ | 0.0666168 | 0.0683831 | 0.0679948 |
| 6 | pk_Ω₂,₁ | 0.0087813 | 0.00789109 | 0.0280481 |
| 7 | pk_Ω₃,₁ | 0.0386503 | 0.032845 | 0.0102071 |
| 8 | pk_Ω₂,₂ | 0.0203362 | 0.0210109 | 0.464393 |
| 9 | pk_Ω₃,₂ | -0.0261285 | -0.00372611 | -0.527727 |
| 10 | pk_Ω₃,₃ | 0.98922 | 0.848756 | 1.41271 |
| 11 | σ_prop | 0.0869357 | 0.0868518 | 0.0882328 |
| 12 | σ_add | 0.40221 | 0.431393 | 0.445463 |
| 13 | pop_e0 | 96.334 | 96.3617 | 96.3449 |
| 14 | pop_emax | -1.07 | -1.08743 | -1.08655 |
| 15 | pop_c50 | 1.53989 | 1.62901 | 1.62474 |
| 16 | pop_tover | 14.1233 | 14.4208 | 14.406 |
| 17 | pd_Ω₁,₁ | 0.00305062 | 0.00285578 | 0.0028373 |
| 18 | pd_Ω₂,₁ | -0.00353783 | 0.0017457 | 0.00166261 |
| 19 | pd_Ω₃,₁ | 0.00128838 | 0.000531597 | 0.000553428 |
| 20 | pd_Ω₂,₂ | 0.151206 | 0.143078 | 0.137592 |
| 21 | pd_Ω₃,₂ | -0.0334376 | -0.0359802 | -0.0333467 |
| 22 | pd_Ω₃,₃ | 0.0131138 | 0.017667 | 0.0166972 |
| 23 | σ_fx | 3.83855 | 3.56433 | 3.52277 |
DataFrame(
Method = [
"Deterministic VEM",
"FOCE initialized from deterministic VEM",
"FOCE direct",
],
LogLikelihood = [
loglikelihood(fpm_vem, FOCE()),
loglikelihood(fpm_foce_from_vem),
loglikelihood(fpm_foce_direct),
],
) |> simple_table| Method | LogLikelihood |
| Deterministic VEM | -1063 |
| FOCE initialized from deterministic VEM | -1060 |
| FOCE direct | -1108 |
This section provides a comprehensive reference for all VEM keyword arguments. The options that control the algorithm itself are keyword arguments to VEM(). The optimizer (optim_alg, optim_options), the random number generator (rng), the parallelism (ensemblealg), the ODE solver options (diffeq_options), and the parameters held constant (constantcoef) are keyword arguments to fit, as they are for the other estimation methods. Each section below states which of the two it documents.
presample)The presample keyword controls which VEM variant is used. Many other defaults change depending on this choice:
| Option | Presampling (presample=true, default) |
Resampling (presample=false) |
|
|---|---|---|---|
| Optimizer | optim_alg (to fit) |
L-BFGS (deterministic) | Adam (stochastic) |
| Phases | nphases |
1 | 10 |
| Initial samples | init_nsamples |
15 | 3 |
| Sample increment | nsamples_increment |
half the initial samples (7 by default) | the initial samples (3 by default) |
| Max samples | max_nsamples |
Unlimited | 25 |
Presampling draws samples once per phase and reuses them across iterations, which allows a deterministic optimizer (L-BFGS) to be used. Resampling draws fresh samples at every iteration, which requires a stochastic optimizer (Adam) but can be more appropriate for DeepPumas models with neural networks. However, generally the stochastic optimizer needs many more iterations to converge than the deterministic one, so presampling is a good default for most models.
Note that all of the above options can be overridden regardless of the presample setting.
# Resampling VEM
@time fpm_vem_resample = fit(
warfarin_model,
pop,
warfarin_init,
VEM(; presample = false, nphases = 5);
optim_options = (; iterations = 1000, show_every = 20),
)[ Info: phase iter obj ∇obj s∇obj [ Info: 1 / 5 1 / 1000 1.4e+08 3.6e+08 1.0e+00 [ Info: 1 / 5 20 / 1000 2.7e+06 1.4e+07 4.0e-02 [ Info: 1 / 5 40 / 1000 8.9e+05 3.7e+06 1.0e-02 [ Info: 1 / 5 50 / 1000 6.9e+05 5.7e+06 1.6e-02 [ Info: Terminating phase due to sgtol. [ Info: 2 / 5 1 / 1000 6.5e+05 4.2e+06 1.0e+00 [ Info: 2 / 5 20 / 1000 2.6e+05 1.6e+06 3.9e-01 [ Info: 2 / 5 40 / 1000 1.4e+05 4.0e+05 9.7e-02 [ Info: 2 / 5 60 / 1000 8.8e+04 2.0e+05 4.8e-02 [ Info: 2 / 5 68 / 1000 7.5e+04 1.5e+05 3.6e-02 [ Info: Terminating phase due to sgtol. [ Info: 3 / 5 1 / 1000 7.6e+04 1.6e+05 1.0e+00 [ Info: 3 / 5 20 / 1000 6.4e+04 1.7e+05 1.1e+00 [ Info: 3 / 5 40 / 1000 5.4e+04 1.4e+05 8.8e-01 [ Info: 3 / 5 60 / 1000 4.6e+04 9.5e+04 6.1e-01 [ Info: 3 / 5 80 / 1000 4.1e+04 8.2e+04 5.3e-01 [ Info: 3 / 5 100 / 1000 3.6e+04 7.9e+04 5.1e-01 [ Info: 3 / 5 120 / 1000 3.2e+04 6.8e+04 4.3e-01 [ Info: 3 / 5 140 / 1000 2.9e+04 6.0e+04 3.8e-01 [ Info: 3 / 5 160 / 1000 2.7e+04 5.7e+04 3.7e-01 [ Info: 3 / 5 180 / 1000 2.4e+04 4.9e+04 3.1e-01 [ Info: 3 / 5 200 / 1000 2.2e+04 5.8e+04 3.7e-01 [ Info: 3 / 5 220 / 1000 2.0e+04 4.3e+04 2.7e-01 [ Info: 3 / 5 240 / 1000 1.9e+04 4.3e+04 2.8e-01 [ Info: 3 / 5 260 / 1000 1.8e+04 3.6e+04 2.3e-01 [ Info: 3 / 5 280 / 1000 1.7e+04 4.1e+04 2.6e-01 [ Info: 3 / 5 300 / 1000 1.6e+04 3.2e+04 2.0e-01 [ Info: 3 / 5 320 / 1000 1.5e+04 2.9e+04 1.9e-01 [ Info: 3 / 5 340 / 1000 1.4e+04 2.8e+04 1.8e-01 [ Info: 3 / 5 360 / 1000 1.3e+04 3.8e+04 2.4e-01 [ Info: 3 / 5 380 / 1000 1.2e+04 4.1e+04 2.6e-01 [ Info: 3 / 5 400 / 1000 1.2e+04 2.4e+04 1.6e-01 [ Info: 3 / 5 420 / 1000 1.1e+04 2.3e+04 1.5e-01 [ Info: 3 / 5 440 / 1000 1.1e+04 2.0e+04 1.3e-01 [ Info: 3 / 5 460 / 1000 1.0e+04 1.9e+04 1.2e-01 [ Info: 3 / 5 480 / 1000 9.9e+03 1.9e+04 1.2e-01 [ Info: 3 / 5 500 / 1000 9.3e+03 1.7e+04 1.1e-01 [ Info: 3 / 5 520 / 1000 9.0e+03 1.7e+04 1.1e-01 [ Info: 3 / 5 540 / 1000 8.6e+03 1.6e+04 1.0e-01 [ Info: 3 / 5 560 / 1000 8.2e+03 1.6e+04 1.1e-01 [ Info: 3 / 5 580 / 1000 7.9e+03 1.9e+04 1.2e-01 [ Info: 3 / 5 600 / 1000 7.6e+03 1.4e+04 9.1e-02 [ Info: 3 / 5 620 / 1000 7.4e+03 1.3e+04 8.5e-02 [ Info: 3 / 5 640 / 1000 7.1e+03 1.4e+04 8.7e-02 [ Info: 3 / 5 660 / 1000 7.0e+03 1.4e+04 9.1e-02 [ Info: 3 / 5 680 / 1000 6.7e+03 1.2e+04 7.5e-02 [ Info: 3 / 5 700 / 1000 6.5e+03 1.2e+04 7.8e-02 [ Info: 3 / 5 720 / 1000 6.3e+03 1.3e+04 8.3e-02 [ Info: 3 / 5 740 / 1000 6.1e+03 1.1e+04 6.9e-02 [ Info: 3 / 5 760 / 1000 5.9e+03 1.2e+04 7.8e-02 [ Info: 3 / 5 780 / 1000 5.8e+03 9.8e+03 6.3e-02 [ Info: 3 / 5 800 / 1000 5.6e+03 1.1e+04 6.9e-02 [ Info: 3 / 5 820 / 1000 5.4e+03 9.7e+03 6.2e-02 [ Info: 3 / 5 840 / 1000 5.3e+03 8.8e+03 5.6e-02 [ Info: 3 / 5 860 / 1000 5.2e+03 1.1e+04 7.2e-02 [ Info: 3 / 5 880 / 1000 5.0e+03 8.6e+03 5.5e-02 [ Info: 3 / 5 900 / 1000 4.9e+03 8.2e+03 5.2e-02 [ Info: 3 / 5 920 / 1000 4.8e+03 1.1e+04 7.0e-02 [ Info: 3 / 5 940 / 1000 4.7e+03 7.4e+03 4.8e-02 [ Info: 3 / 5 960 / 1000 4.6e+03 7.5e+03 4.8e-02 [ Info: 3 / 5 980 / 1000 4.4e+03 8.2e+03 5.3e-02 [ Info: 3 / 5 1000 / 1000 4.4e+03 7.5e+03 4.8e-02 [ Info: 4 / 5 1 / 1000 4.4e+03 7.9e+03 1.0e+00 [ Info: 4 / 5 20 / 1000 4.3e+03 7.7e+03 9.7e-01 [ Info: 4 / 5 40 / 1000 4.2e+03 7.1e+03 9.0e-01 [ Info: 4 / 5 60 / 1000 4.1e+03 6.8e+03 8.7e-01 [ Info: 4 / 5 80 / 1000 4.0e+03 6.2e+03 7.9e-01 [ Info: 4 / 5 100 / 1000 4.0e+03 6.2e+03 7.9e-01 [ Info: 4 / 5 120 / 1000 3.9e+03 5.8e+03 7.4e-01 [ Info: 4 / 5 140 / 1000 3.8e+03 5.8e+03 7.4e-01 [ Info: 4 / 5 160 / 1000 3.8e+03 7.3e+03 9.3e-01 [ Info: 4 / 5 180 / 1000 3.7e+03 5.5e+03 7.0e-01 [ Info: 4 / 5 200 / 1000 3.6e+03 5.5e+03 7.0e-01 [ Info: 4 / 5 220 / 1000 3.6e+03 5.4e+03 6.9e-01 [ Info: 4 / 5 240 / 1000 3.5e+03 5.2e+03 6.6e-01 [ Info: 4 / 5 260 / 1000 3.5e+03 6.0e+03 7.6e-01 [ Info: 4 / 5 280 / 1000 3.4e+03 5.0e+03 6.4e-01 [ Info: 4 / 5 300 / 1000 3.4e+03 6.3e+03 8.0e-01 [ Info: 4 / 5 320 / 1000 3.3e+03 5.5e+03 7.0e-01 [ Info: 4 / 5 340 / 1000 3.3e+03 5.5e+03 6.9e-01 [ Info: 4 / 5 360 / 1000 3.2e+03 4.9e+03 6.2e-01 [ Info: 4 / 5 380 / 1000 3.2e+03 4.8e+03 6.1e-01 [ Info: 4 / 5 400 / 1000 3.1e+03 5.2e+03 6.6e-01 [ Info: 4 / 5 420 / 1000 3.1e+03 4.5e+03 5.7e-01 [ Info: 4 / 5 440 / 1000 3.1e+03 4.1e+03 5.3e-01 [ Info: 4 / 5 460 / 1000 3.0e+03 4.1e+03 5.2e-01 [ Info: 4 / 5 480 / 1000 3.0e+03 4.1e+03 5.2e-01 [ Info: 4 / 5 500 / 1000 2.9e+03 4.0e+03 5.0e-01 [ Info: 4 / 5 520 / 1000 2.9e+03 3.8e+03 4.8e-01 [ Info: 4 / 5 540 / 1000 2.9e+03 3.7e+03 4.7e-01 [ Info: 4 / 5 560 / 1000 2.8e+03 3.9e+03 4.9e-01 [ Info: 4 / 5 580 / 1000 2.8e+03 3.7e+03 4.7e-01 [ Info: 4 / 5 600 / 1000 2.8e+03 4.2e+03 5.3e-01 [ Info: 4 / 5 620 / 1000 2.7e+03 3.5e+03 4.5e-01 [ Info: 4 / 5 640 / 1000 2.7e+03 3.3e+03 4.3e-01 [ Info: 4 / 5 660 / 1000 2.7e+03 3.4e+03 4.3e-01 [ Info: 4 / 5 680 / 1000 2.6e+03 3.4e+03 4.3e-01 [ Info: 4 / 5 700 / 1000 2.6e+03 3.4e+03 4.3e-01 [ Info: 4 / 5 720 / 1000 2.6e+03 3.3e+03 4.2e-01 [ Info: 4 / 5 740 / 1000 2.6e+03 3.1e+03 3.9e-01 [ Info: 4 / 5 760 / 1000 2.5e+03 3.7e+03 4.7e-01 [ Info: 4 / 5 780 / 1000 2.5e+03 3.2e+03 4.0e-01 [ Info: 4 / 5 800 / 1000 2.5e+03 3.2e+03 4.0e-01 [ Info: 4 / 5 820 / 1000 2.5e+03 3.3e+03 4.2e-01 [ Info: 4 / 5 840 / 1000 2.4e+03 4.1e+03 5.2e-01 [ Info: 4 / 5 860 / 1000 2.4e+03 2.9e+03 3.6e-01 [ Info: 4 / 5 880 / 1000 2.4e+03 2.9e+03 3.7e-01 [ Info: 4 / 5 900 / 1000 2.4e+03 2.7e+03 3.4e-01 [ Info: 4 / 5 920 / 1000 2.3e+03 2.8e+03 3.5e-01 [ Info: 4 / 5 940 / 1000 2.3e+03 3.0e+03 3.7e-01 [ Info: 4 / 5 960 / 1000 2.3e+03 2.5e+03 3.2e-01 [ Info: 4 / 5 980 / 1000 2.3e+03 2.5e+03 3.2e-01 [ Info: 4 / 5 1000 / 1000 2.3e+03 2.6e+03 3.3e-01 [ Info: 5 / 5 1 / 1000 2.3e+03 2.7e+03 1.0e+00 [ Info: 5 / 5 20 / 1000 2.2e+03 2.5e+03 9.5e-01 [ Info: 5 / 5 40 / 1000 2.2e+03 2.4e+03 9.2e-01 [ Info: 5 / 5 60 / 1000 2.2e+03 2.5e+03 9.2e-01 [ Info: 5 / 5 80 / 1000 2.2e+03 2.7e+03 1.0e+00 [ Info: 5 / 5 100 / 1000 2.2e+03 2.6e+03 9.7e-01 [ Info: 5 / 5 120 / 1000 2.2e+03 2.4e+03 9.1e-01 [ Info: 5 / 5 140 / 1000 2.2e+03 2.4e+03 9.0e-01 [ Info: 5 / 5 160 / 1000 2.2e+03 2.4e+03 9.0e-01 [ Info: 5 / 5 180 / 1000 2.2e+03 2.5e+03 9.5e-01 [ Info: 5 / 5 200 / 1000 2.2e+03 2.3e+03 8.8e-01 [ Info: 5 / 5 220 / 1000 2.2e+03 2.3e+03 8.9e-01 [ Info: 5 / 5 240 / 1000 2.2e+03 2.4e+03 9.0e-01 [ Info: 5 / 5 260 / 1000 2.2e+03 2.4e+03 9.0e-01 [ Info: 5 / 5 280 / 1000 2.2e+03 2.3e+03 8.6e-01 [ Info: 5 / 5 300 / 1000 2.2e+03 2.4e+03 8.9e-01 [ Info: 5 / 5 320 / 1000 2.2e+03 2.3e+03 8.7e-01 [ Info: 5 / 5 340 / 1000 2.2e+03 2.3e+03 8.5e-01 [ Info: 5 / 5 360 / 1000 2.2e+03 2.3e+03 8.5e-01 [ Info: 5 / 5 380 / 1000 2.1e+03 2.2e+03 8.4e-01 [ Info: 5 / 5 400 / 1000 2.1e+03 2.2e+03 8.5e-01 [ Info: 5 / 5 420 / 1000 2.1e+03 2.3e+03 8.5e-01 [ Info: 5 / 5 440 / 1000 2.1e+03 2.5e+03 9.3e-01 [ Info: 5 / 5 460 / 1000 2.1e+03 2.2e+03 8.4e-01 [ Info: 5 / 5 480 / 1000 2.1e+03 2.5e+03 9.4e-01 [ Info: 5 / 5 500 / 1000 2.1e+03 2.2e+03 8.4e-01 [ Info: 5 / 5 520 / 1000 2.1e+03 2.2e+03 8.4e-01 [ Info: 5 / 5 540 / 1000 2.1e+03 2.4e+03 9.2e-01 [ Info: 5 / 5 560 / 1000 2.1e+03 2.5e+03 9.5e-01 [ Info: 5 / 5 580 / 1000 2.1e+03 2.1e+03 8.1e-01 [ Info: 5 / 5 600 / 1000 2.1e+03 2.3e+03 8.5e-01 [ Info: 5 / 5 620 / 1000 2.1e+03 2.1e+03 8.0e-01 [ Info: 5 / 5 640 / 1000 2.1e+03 2.2e+03 8.4e-01 [ Info: 5 / 5 660 / 1000 2.1e+03 2.2e+03 8.4e-01 [ Info: 5 / 5 680 / 1000 2.1e+03 2.1e+03 8.0e-01 [ Info: 5 / 5 700 / 1000 2.1e+03 2.1e+03 7.9e-01 [ Info: 5 / 5 720 / 1000 2.1e+03 2.3e+03 8.8e-01 [ Info: 5 / 5 740 / 1000 2.1e+03 2.1e+03 7.8e-01 [ Info: 5 / 5 760 / 1000 2.1e+03 2.1e+03 7.8e-01 [ Info: 5 / 5 780 / 1000 2.1e+03 2.1e+03 7.9e-01 [ Info: 5 / 5 800 / 1000 2.0e+03 2.4e+03 8.9e-01 [ Info: 5 / 5 820 / 1000 2.0e+03 2.3e+03 8.5e-01 [ Info: 5 / 5 840 / 1000 2.0e+03 2.1e+03 7.7e-01 [ Info: 5 / 5 860 / 1000 2.0e+03 2.5e+03 9.3e-01 [ Info: 5 / 5 880 / 1000 2.0e+03 2.0e+03 7.6e-01 [ Info: 5 / 5 900 / 1000 2.0e+03 2.0e+03 7.6e-01 [ Info: 5 / 5 920 / 1000 2.0e+03 2.1e+03 7.9e-01 [ Info: 5 / 5 940 / 1000 2.0e+03 2.0e+03 7.6e-01 [ Info: 5 / 5 960 / 1000 2.0e+03 2.0e+03 7.5e-01 [ Info: 5 / 5 980 / 1000 2.0e+03 2.2e+03 8.4e-01 [ Info: 5 / 5 1000 / 1000 2.0e+03 2.1e+03 7.9e-01 1458.739189 seconds (1.61 G allocations: 308.094 GiB, 7.12% gc time, 0.39% compilation time)
FittedPumasModel
Dynamical system type: Nonlinear ODE
Solver(s): (OrdinaryDiffEqVerner.Vern7,OrdinaryDiffEqRosenbrock.Rodas5P)
Number of subjects: 32
Observation records: Active Missing
conc: 247 51
pca: 232 66
Total: 479 117
Number of parameters: Constant Optimized
0 23
Estimation method: VEM (EXPERIMENTAL)
-----------------------
Estimate
-----------------------
pop_CL 0.16289
pop_V 9.0283
pop_tabs 1.4096
pop_lag 0.89858
pk_Ω₁,₁ 0.14117
pk_Ω₂,₁ 0.074728
pk_Ω₃,₁ 0.33276
pk_Ω₂,₂ 0.20382
pk_Ω₃,₂ 0.011606
pk_Ω₃,₃ 1.6695
σ_prop 0.033996
σ_add 2.8624
pop_e0 95.31
pop_emax -1.14
pop_c50 1.8619
pop_tover 13.417
pd_Ω₁,₁ 0.0089951
pd_Ω₂,₁ 0.0034843
pd_Ω₃,₁ -0.0049231
pd_Ω₂,₂ 0.024728
pd_Ω₃,₂ -0.016465
pd_Ω₃,₃ 0.032257
σ_fx 0.70753
-----------------------
loglikelihood(fpm_vem_resample, FOCE())-2091.5530177086052
The number of iterations and phases have been restricted in the above example for demonstration purposes.
When presampling is enabled, the trace is deterministic and should show steady convergence. The trace printed during fitting has three columns:
| Column | Meaning |
|---|---|
| Iter | Current iteration within the phase. |
| Function value | The negative ELBO — the objective being minimized. Lower values indicate a better fit. |
| Gradient norm | The gradient norm — how steep the objective landscape is. Decreasing values indicate the optimizer is converging. |
When resampling is enabled, the trace is stochastic and may show more fluctuation. In the stochastic case, the trace printed during fitting has five columns:
| Column | Meaning |
|---|---|
| phase | Current phase / total phases. Each phase increases the sample count. |
| iter | Current iteration / max iterations within this phase. |
| obj | The negative ELBO — the objective being minimized. Lower values indicate a better fit. |
| ∇obj | The gradient norm — how steep the objective landscape is. Decreasing values indicate the optimizer is converging. |
| s∇obj | The scaled gradient norm — gradient norm relative to the first iteration of the current phase (normalized to 1.0). Shows relative convergence progress within each phase. |
A phase may terminate early with “Terminating phase due to sgtol” when the scaled gradient norm drops below the stopping tolerance (0.05), triggering the next phase, typically with an increase in the sample size and a decrease in the learning rate. For stochastic optimizers like Adam (resampling VEM), some fluctuation in ∇obj is normal because the gradients are estimated from random samples.
compare_estimates(;
VEM_presample = fpm_vem,
VEM_resample = fpm_vem_resample,
VEM_FOCE = fpm_foce_from_vem,
FOCE = fpm_foce_direct,
)| Row | parameter | VEM_presample | VEM_resample | VEM_FOCE | FOCE |
|---|---|---|---|---|---|
| String | Float64? | Float64? | Float64? | Float64? | |
| 1 | pop_CL | 0.13453 | 0.16289 | 0.135019 | 0.135055 |
| 2 | pop_V | 7.99776 | 9.02833 | 8.03022 | 7.14558 |
| 3 | pop_tabs | 0.396436 | 1.40958 | 0.536134 | 0.619069 |
| 4 | pop_lag | 0.88715 | 0.898585 | 0.879113 | 0.872328 |
| 5 | pk_Ω₁,₁ | 0.0666168 | 0.141171 | 0.0683831 | 0.0679948 |
| 6 | pk_Ω₂,₁ | 0.0087813 | 0.0747281 | 0.00789109 | 0.0280481 |
| 7 | pk_Ω₃,₁ | 0.0386503 | 0.33276 | 0.032845 | 0.0102071 |
| 8 | pk_Ω₂,₂ | 0.0203362 | 0.203824 | 0.0210109 | 0.464393 |
| 9 | pk_Ω₃,₂ | -0.0261285 | 0.0116061 | -0.00372611 | -0.527727 |
| 10 | pk_Ω₃,₃ | 0.98922 | 1.6695 | 0.848756 | 1.41271 |
| 11 | σ_prop | 0.0869357 | 0.0339964 | 0.0868518 | 0.0882328 |
| 12 | σ_add | 0.40221 | 2.86235 | 0.431393 | 0.445463 |
| 13 | pop_e0 | 96.334 | 95.3103 | 96.3617 | 96.3449 |
| 14 | pop_emax | -1.07 | -1.13996 | -1.08743 | -1.08655 |
| 15 | pop_c50 | 1.53989 | 1.8619 | 1.62901 | 1.62474 |
| 16 | pop_tover | 14.1233 | 13.4172 | 14.4208 | 14.406 |
| 17 | pd_Ω₁,₁ | 0.00305062 | 0.00899515 | 0.00285578 | 0.0028373 |
| 18 | pd_Ω₂,₁ | -0.00353783 | 0.00348428 | 0.0017457 | 0.00166261 |
| 19 | pd_Ω₃,₁ | 0.00128838 | -0.00492307 | 0.000531597 | 0.000553428 |
| 20 | pd_Ω₂,₂ | 0.151206 | 0.0247284 | 0.143078 | 0.137592 |
| 21 | pd_Ω₃,₂ | -0.0334376 | -0.0164645 | -0.0359802 | -0.0333467 |
| 22 | pd_Ω₃,₃ | 0.0131138 | 0.0322566 | 0.017667 | 0.0166972 |
| 23 | σ_fx | 3.83855 | 0.707526 | 3.56433 | 3.52277 |
DataFrame(
Method = [
"Deterministic VEM",
"Stochastic VEM",
"FOCE initialized from deterministic VEM",
"FOCE direct",
],
LogLikelihood = [
loglikelihood(fpm_vem, FOCE()),
loglikelihood(fpm_vem_resample, FOCE()),
loglikelihood(fpm_foce_from_vem),
loglikelihood(fpm_foce_direct),
],
) |> simple_table| Method | LogLikelihood |
| Deterministic VEM | -1063 |
| Stochastic VEM | -2092 |
| FOCE initialized from deterministic VEM | -1060 |
| FOCE direct | -1108 |
Note that stochastic VEM took much longer to converge and it only got close to the best parameters. Deterministic VEM will generally converge faster. Stochastic VEM is mostly useful for models with neural networks and large populations where subject_ratio < 1 (to be demonstrated below) can be used to get quick estimates by running the algorithm for a few iterations only.
VEM draws samples from a base distribution to estimate the ELBO gradient. More samples improve accuracy but increase computation time.
When nsamples is set to a nonzero value, it overrides init_nsamples and sets nsamples_increment to 0, so the sample count stays fixed across all phases:
# Fixed 30 samples per subject across all phases
@time fpm_vem_nsamples = fit(
warfarin_model,
pop,
warfarin_init,
VEM(; nsamples = 30);
optim_options = (; show_every = 20),
)[ Info: Checking the initial parameter values. [ Info: The initial negative log-likelihood and its gradient are finite. Check passed. Iter Function value Gradient norm 0 1.374727e+08 2.749450e+08 * time: 2.09808349609375e-5 20 1.550251e+03 5.815802e+02 * time: 31.568580865859985 40 1.082107e+03 1.660056e+02 * time: 64.77731084823608 60 9.610787e+02 2.306563e+01 * time: 92.59014391899109 80 9.228660e+02 2.706740e+01 * time: 123.87429785728455 100 8.818840e+02 1.738420e+01 * time: 161.69694185256958 120 8.493450e+02 4.231016e+01 * time: 201.65474390983582 140 8.043672e+02 1.933396e+01 * time: 239.0825219154358 160 7.603885e+02 1.282275e+01 * time: 273.10702896118164 180 7.324729e+02 1.321452e+01 * time: 306.0147759914398 200 7.226272e+02 3.851044e+00 * time: 353.58604097366333 220 7.178002e+02 1.130184e+00 * time: 401.90573501586914 240 7.168652e+02 7.567175e+00 * time: 445.86555886268616 458.667157 seconds (393.31 M allocations: 89.216 GiB, 7.22% gc time)
FittedPumasModel
Dynamical system type: Nonlinear ODE
Solver(s): (OrdinaryDiffEqVerner.Vern7,OrdinaryDiffEqRosenbrock.Rodas5P)
Number of subjects: 32
Observation records: Active Missing
conc: 247 51
pca: 232 66
Total: 479 117
Number of parameters: Constant Optimized
0 23
Estimation method: VEM (EXPERIMENTAL)
-----------------------
Estimate
-----------------------
pop_CL 0.13432
pop_V 7.9897
pop_tabs 0.39726
pop_lag 0.8834
pk_Ω₁,₁ 0.064757
pk_Ω₂,₁ 0.0083885
pk_Ω₃,₁ 0.030556
pk_Ω₂,₂ 0.020286
pk_Ω₃,₂ -0.027913
pk_Ω₃,₃ 0.91077
σ_prop 0.086739
σ_add 0.40521
pop_e0 96.306
pop_emax -1.0702
pop_c50 1.5436
pop_tover 14.117
pd_Ω₁,₁ 0.0032088
pd_Ω₂,₁ -0.0037275
pd_Ω₃,₁ 0.0010459
pd_Ω₂,₂ 0.14785
pd_Ω₃,₂ -0.033741
pd_Ω₃,₃ 0.014401
σ_fx 3.7984
-----------------------
loglikelihood(fpm_vem_nsamples, FOCE())-1062.68021696948
Notice that increasing the number of samples improved the quality of the ELBO however that does not guarantee an improvement in the FOCE-approximated log likelihood of the solution found. In this case, it improved it. Heuristically, setting nsamples to 15-30 seems to be a good starting point for most models, but the best value can vary based on the number of random effects in the model.
For multi-phase runs, the number of samples in phase p is: min(init_nsamples + (p - 1) * nsamples_increment, max_nsamples). More samples in later phases refine the estimate as the parameters approach their final values.
# Resampling with more phases and custom sample progression
fit(
model,
pop,
params,
VEM(;
presample = false,
nphases = 15,
init_nsamples = 5,
nsamples_increment = 5,
max_nsamples = 50,
),
)optim_options)Fine-grained control over the optimizer is passed through the optim_options named tuple. This is a keyword to fit, not to VEM(). The available options depend on which variant is used.
For presampling (L-BFGS):
| Option | Default | Description |
|---|---|---|
iterations |
3000 | Maximum iterations per phase |
show_trace |
true |
Print convergence information |
show_every |
5 | Print every N iterations |
f_reltol |
1e-5 | Relative tolerance on the objective |
g_tol |
1e-3 | Gradient norm tolerance |
fit(model, pop, params, VEM(); optim_options = (; iterations = 5000, show_every = 10))For resampling (Adam):
| Option | Default | Description |
|---|---|---|
iterations |
20000 | Maximum iterations per phase |
show_trace |
true |
Print convergence information |
show_every |
10 | Print every N iterations |
subject_ratio |
1.0 | Fraction of subjects per iteration (mini-batching) |
Setting subject_ratio < 1.0 enables mini-batching, where only a random subset of subjects is used per iteration. This can speed up each iteration for large datasets at the cost of noisier gradients:
fit(
model,
pop,
params,
VEM(; presample = false);
optim_options = (; iterations = 30000, subject_ratio = 0.5),
)prior)By default (prior=true), VEM adds the log prior on the population parameters, if defined in the @param block, to the ELBO. This allows for maximum a-posteriori (MAP) estimation of the population parameters, after marginalizing the random effects. Setting prior=false ignores the prior contribution, which means the ELBO depends only on the likelihood:
fit(model, pop, params, VEM(; prior = false))If the @param block has no prior distributions and only includes domains, this option is not used.
base_dist_std)VEM’s variational family uses a reparameterization trick: samples are drawn from a base distribution and transformed to approximate the posterior. The base_dist_std parameter controls the standard deviation of this base distribution (default: 0.1):
fit(model, pop, params, VEM(; base_dist_std = 0.05))Smaller values start the optimization with a tighter initial approximation to the posterior, which can help stability for sensitive models. Larger values allow broader initial exploration. The default of 0.1 works well for most models.
diagonal)VEM approximates each subject’s random effect posterior with a multivariate normal distribution. The diagonal keyword controls the covariance structure:
diagonal=true (default): assumes independent random effects in the posterior. Faster and requires fewer variational parameters.diagonal=false: captures correlations between random effects in the posterior. Slower but may be more accurate when random effects are correlated.fpm_full_cov = fit(
warfarin_model,
pop,
warfarin_init,
VEM(; diagonal = false);
optim_options = (; show_every = 20),
)[ Info: Checking the initial parameter values. [ Info: The initial negative log-likelihood and its gradient are finite. Check passed. Iter Function value Gradient norm 0 1.368292e+08 2.736580e+08 * time: 2.6941299438476562e-5 20 1.525485e+03 5.300085e+02 * time: 16.681507110595703 40 1.103091e+03 2.490318e+02 * time: 31.316335916519165 60 9.616820e+02 4.585779e+01 * time: 43.586803913116455 80 8.954688e+02 4.166251e+01 * time: 57.02774500846863 100 8.118614e+02 7.086030e+01 * time: 72.26464104652405 120 7.623553e+02 9.704948e+01 * time: 89.80058193206787 140 7.333012e+02 1.749287e+01 * time: 107.1308491230011 160 7.201056e+02 1.222130e+01 * time: 124.47755908966064 180 7.143766e+02 2.752501e+00 * time: 142.4066789150238 200 7.108468e+02 3.909115e+00 * time: 166.86034893989563 220 7.075539e+02 6.408487e+00 * time: 199.35051107406616 240 7.053211e+02 1.071369e+01 * time: 229.38029193878174 260 7.041626e+02 1.499914e+01 * time: 254.64632105827332 280 7.031582e+02 1.646967e+01 * time: 272.8636751174927 300 7.024212e+02 2.161227e+00 * time: 291.43470311164856 320 7.017475e+02 2.988461e+01 * time: 309.0804340839386 340 7.009747e+02 4.088582e+00 * time: 328.33753514289856 360 6.996141e+02 4.386131e+00 * time: 348.9258201122284 380 6.983768e+02 2.137981e+00 * time: 367.09574604034424 400 6.972393e+02 5.685682e+00 * time: 384.0184030532837 420 6.953632e+02 4.324135e+00 * time: 400.6962969303131 440 6.936934e+02 6.003071e+00 * time: 418.0892460346222 460 6.927456e+02 1.074300e+01 * time: 435.73579001426697 480 6.918063e+02 3.361402e+00 * time: 456.6340711116791 500 6.908235e+02 1.331224e+00 * time: 480.790687084198 520 6.901069e+02 3.909953e+00 * time: 501.35266399383545
FittedPumasModel
Dynamical system type: Nonlinear ODE
Solver(s): (OrdinaryDiffEqVerner.Vern7,OrdinaryDiffEqRosenbrock.Rodas5P)
Number of subjects: 32
Observation records: Active Missing
conc: 247 51
pca: 232 66
Total: 479 117
Number of parameters: Constant Optimized
0 23
Estimation method: VEM (EXPERIMENTAL)
-----------------------
Estimate
-----------------------
pop_CL 0.13422
pop_V 8.0044
pop_tabs 0.42091
pop_lag 0.88938
pk_Ω₁,₁ 0.067359
pk_Ω₂,₁ 0.0081471
pk_Ω₃,₁ 0.055683
pk_Ω₂,₂ 0.02143
pk_Ω₃,₂ -0.0063547
pk_Ω₃,₃ 1.0164
σ_prop 0.086551
σ_add 0.43006
pop_e0 96.402
pop_emax -1.0866
pop_c50 1.6153
pop_tover 14.37
pd_Ω₁,₁ 0.0020935
pd_Ω₂,₁ 0.0050701
pd_Ω₃,₁ 0.0010105
pd_Ω₂,₂ 0.13637
pd_Ω₃,₂ -0.035903
pd_Ω₃,₃ 0.012341
σ_fx 3.894
-----------------------
loglikelihood(fpm_full_cov, FOCE())-1064.1164257202418
Note that setting diagonal = false does not always lead to a better fit. It is generally recommended to start with diagonal = true and only switch to diagonal = false if there is reason to believe that the random effects are highly correlated.
constantcoef)You can hold specific parameters constant during estimation using the constantcoef keyword. This is a keyword to fit, not to VEM():
fit(model, pop, params, VEM(); constantcoef = (:σ_prop,))constantcoef is a keyword argument to fit, not to VEM(). This is consistent with how it works for FOCE and other algorithms.
rng)VEM uses random sampling internally, so results vary across runs by default. Set a specific random number generator for reproducible results. This is a keyword to fit, not to VEM():
rng = default_rng()
seed!(rng, 123)
fit(model, pop, params, VEM(); rng)Even with presampling, different seeds produce different results because the pre-drawn samples differ. For exact reproducibility across runs, always set the rng.
ensemblealg)Control parallelism across subjects with this keyword to fit:
EnsembleThreads() (default): uses multiple threads for gradient computation.EnsembleSerial(): single-threaded, useful for debugging.fit(model, pop, params, VEM(); ensemblealg = EnsembleSerial())diffeq_options)Pass differential equation solver options through diffeq_options, another keyword to fit:
fit(model, pop, params, VEM(); diffeq_options = (; abstol = 1e-8, reltol = 1e-8))This is useful when the default ODE tolerances are insufficient for a stiff or sensitive model.
Choose VEM when:
Choose FOCE when:
Choose LaplaceI when:
In this tutorial, we introduced VEM as a flexible estimation method in Pumas. We covered its conceptual foundations (ELBO maximization, presampling vs. resampling, and posterior covariance structure), demonstrated its use with a warfarin PK model, showed how to compare results with FOCE, and provided a reference for the main VEM options.