using Dates
using Pumas
using PumasUtilities
using DataFramesMeta
using PharmaDatasets
using CairoMakie
using AlgebraOfGraphics
using Random
Why are non-Gaussian random effects relevant?
1 Motivation - PK model
Why using a non-Gaussian distribution as the underlying distribution for the random effects? There are a couple of arguments.
First, the Gaussian distribution has unbounded support, i.e. it take any value in \((-\infty, \infty)\). While phamacokinetic parameters typically are (semi) bounded, e.g.:
- clearance and volumes, \((0, \infty)\)
- bioavailability, \([0, 1]\)
Additionally, in order for a Gaussian distribution to work as the underlying distribution, often we need to transform them (e.g. exponentiation and logistic transformation). But these transformations in some settings, when the random effects do not have a great impact, i.e. they do not have large values, may shift the mean of the typical values (\(\theta\)) so that the expectation of the typical values (\(\operatorname{E}\)) are not equal to the mean. For example, the following code block is a traditional 1-compartment PK model with a Gaussian random effect that needs to be constrained to positive values, \((0, \infty)\):
@random begin
ηCL ~ Normal(0.0, ωCL)
ηVc ~ Normal(0.0, ωVc)
end
@pre begin
CL = θCL * exp(ηCL)
Vc = θVc * exp(ηVc)
end
@dynamics begin
Central' = -CL / Vc * Central
endIf we recover the formula for the expectation of the log-normal distribution, we have that:
\[\operatorname{E}[CL] = \exp \left\{ \log(\theta_{CL}) + \frac{\omega^2_{CL}}{2} \right\} \approx \theta_{CL}\]
This approximation only holds for small \(\omega_{CL}\).
Hence, \(\theta_{CL}\) is only the typical value when \(\omega_{CL}\) is small.
Here is a small tabulation for \(\operatorname{E}[CL]\) when \(\theta_{CL} = 0.5\):
ωs = [0.1, 0.2, 0.4, 0.8, 1.6]
DataFrame(; ω_CL = ωs, E_CL = (ω -> exp(log(0.5) + ω^2 / 2)).(ωs))| Row | ω_CL | E_CL |
|---|---|---|
| Float64 | Float64 | |
| 1 | 0.1 | 0.502506 |
| 2 | 0.2 | 0.510101 |
| 3 | 0.4 | 0.541644 |
| 4 | 0.8 | 0.688564 |
| 5 | 1.6 | 1.79832 |
As you can see, the larger the \(\omega_{CL}\) the more \(\operatorname{E}[CL]\) deviates from \(\theta_{CL}\).
1.1 Gamma distribution for the rescue
We can use the gamma distribution which has the following parametrization:
\[\text{Gamma}(k, \theta)\]
where \(k\) is a shape parameter and \(\theta\) is a scale parameter.
Shape parameters generally control the shape of the distribution rather than shifting it (as a location parameter) of stretching/shrinking it (as a scale parameter)
We can use an alternative parametrization where the mean-value appears directly a parameter:
\[\text{Gamma}(\mu, \sigma)\]
where:
- \(\mu = \theta k\)
- \(\sigma = k^{-\frac{1}{2}}\)
The \(\sigma\) parameter is the coefficient of variation, i.e.
\[\sigma = \frac{\operatorname{Var} X}{\operatorname{E} X},\]
because that mimics the role of \(\sigma\) in the LogNormal(log(μ), σ) where for small values of \(\sigma\)
\[\sigma \approx \\frac{\operatorname{Var} X}{\operatorname{E} X}.\]
So, our previous PK model now becomes:
@random begin
_CL ~ Gamma(1 / ωCL^2, θCL * ωCL^2)
_Vc ~ Gamma(1 / ωVc^2, θVc * ωVc^2)
end
@pre begin
CL = _CL
Vc = _Vc
end
@dynamics begin
Central' = -CL / Vc * Central
endAs you can see the mean from the gamma distribution becomes:
\[\operatorname{E}[CL] = \theta k = \frac{1}{\omega^2_{CL}} \theta_{CL} \omega^2_{CL} = \theta_{CL}\]
It does not dependent on the between-subject variability \(\omega\)!
We are avoiding η notation here since we are modeling the subject-specific parameter directly.
1.2 Gamma versus Log-Nogmal Numerical Simulations
Before we dive into our PK examples, let us showcase the case for gamma versus log-normal with some numerical simulations.
First, let’s define a mean μ_PK value for a typical value along with an array of possible standard deviations σ values:
μ_PK = 1.0
σ = [0.1, 0.2, 0.5, 1.0, 1.5, 2.0]These will serve as the mean and standard deviations for our gamma and log-normal distributions.
Now let’s compare the coefficient of variation (CV) as a function of σ for LogNormal and Gamma:
num_df_gamma = DataFrame(;
μ = μ_PK,
σ = σ,
meanLogNormal = mean.(LogNormal.(log.(μ_PK), σ)),
cvLogNormal = std.(LogNormal.(log.(μ_PK), σ)) ./ mean.(LogNormal.(log.(μ_PK), σ)),
meanGamma = mean.(Gamma.(1 ./ σ .^ 2, μ_PK .* σ .^ 2)),
cvGamma = std.(Gamma.(1 ./ σ .^ 2, μ_PK .* σ .^ 2)) ./
mean.(Gamma.(1 ./ σ .^ 2, μ_PK .* σ .^ 2)),
)| Row | μ | σ | meanLogNormal | cvLogNormal | meanGamma | cvGamma |
|---|---|---|---|---|---|---|
| Float64 | Float64 | Float64 | Float64 | Float64 | Float64 | |
| 1 | 1.0 | 0.1 | 1.00501 | 0.100251 | 1.0 | 0.1 |
| 2 | 1.0 | 0.2 | 1.0202 | 0.202017 | 1.0 | 0.2 |
| 3 | 1.0 | 0.5 | 1.13315 | 0.53294 | 1.0 | 0.5 |
| 4 | 1.0 | 1.0 | 1.64872 | 1.31083 | 1.0 | 1.0 |
| 5 | 1.0 | 1.5 | 3.08022 | 2.91337 | 1.0 | 1.5 |
| 6 | 1.0 | 2.0 | 7.38906 | 7.32108 | 1.0 | 2.0 |
f, ax, plotobj = lines(
num_df_gamma.σ,
num_df_gamma.meanLogNormal;
label = "μ - LogNormal",
linewidth = 3,
axis = (; xlabel = L"\sigma", ylabel = "value"),
)
lines!(
num_df_gamma.σ,
num_df_gamma.meanGamma;
label = "μ - Gamma",
linewidth = 3,
linestyle = :dashdot,
)
lines!(num_df_gamma.σ, num_df_gamma.cvLogNormal; label = "CV - LogNormal", linewidth = 3)
lines!(
num_df_gamma.σ,
num_df_gamma.cvGamma;
label = "CV - Gamma",
linewidth = 3,
linestyle = :dashdot,
)
axislegend(ax; position = :lt)
fIn the graph above, the dashed lines correspond to the mean and CV for the gamma distribution, whereas the solid lines correspond to the log-normal distribution.
There is clearly a bias in both the log-normal’s mean and CV that we don’t see in the gamma distribution.
2 Motivation - Bioavailability
Here is a very common model that can benefit from a non-Gaussian random effects distribution.
The model has one-compartment elimination and oral absorption with modeled bioavailability based on a crossover design.
The following code is a traditional PK model with a Gaussian random effect that needs to be constrained to the unit interval, \([0, 1]\):
@param begin
θF ∈ RealDomain(lower = 0.0, upper = 1.0)
ωF ∈ RealDomain(lower = 0.0)
end
@random begin
ηF ~ Normal(0.0, ωF)
end
@dosecontrol begin
bioav = (Depot = logistic(logit(θF) + ηF),)
endThe expectation \(\operatorname{E}[F]\) doesn’t have closed form and is generally different from \(\theta_F\). However, we have that:
\[\operatorname{E}[F] \approx \theta_F\]
when \(ωF\) is small. I.e. \(\theta_F\) is only the typical value when \(ωF\) is small.
2.1 Beta versus Logit-Normal Numerical Simulations
Let’s perform the same type of simulations we did before, but now we will be using the numerical integrator quadgk from the QuadGK.jl package. This is because we don’t have a closed form solution for \(\operatorname{E}[F]\) in the logit-normal parameterization.
using QuadGK: quadgkμ_bioav = 0.7We’ll also reuse the same σ values for the CVs.
num_df_beta = DataFrame(;
μ = μ_bioav,
σ = σ,
meanLogitNormal = map(
σ -> quadgk(
t -> logistic(t) * pdf(Normal(logit(μ_bioav), σ), t),
-100 * σ,
100 * σ,
)[1],
σ,
),
meanBeta = mean.(Beta.(μ_bioav ./ σ, (1 - μ_bioav) ./ σ)),
)| Row | μ | σ | meanLogitNormal | meanBeta |
|---|---|---|---|---|
| Float64 | Float64 | Float64 | Float64 | |
| 1 | 0.7 | 0.1 | 0.699582 | 0.7 |
| 2 | 0.7 | 0.2 | 0.698345 | 0.7 |
| 3 | 0.7 | 0.5 | 0.690393 | 0.7 |
| 4 | 0.7 | 1.0 | 0.668971 | 0.7 |
| 5 | 0.7 | 1.5 | 0.646064 | 0.7 |
| 6 | 0.7 | 2.0 | 0.626038 | 0.7 |
f, ax, plotobj = lines(
num_df_beta.σ,
num_df_beta.meanLogitNormal;
label = "μ - LogitNormal",
linewidth = 3,
axis = (; xlabel = L"\sigma", ylabel = "value"),
)
lines!(
num_df_beta.σ,
num_df_beta.meanBeta;
label = "μ - Beta",
linewidth = 3,
linestyle = :dashdot,
)
axislegend(ax; position = :lb)
fIn the graph above, the dashed lines correspond to the mean for the beta distribution, whereas the solid lines correspond to the logit-normal distribution.
As before, there is clearly a bias in the logit-normal’s mean that we don’t see in the beta distribution.
3 Warfarin data
We’ll demonstrate those intuitions using the Warfarin dataset.
pop = read_pumas(dataset("pumas/warfarin"))Population
Subjects: 32
Observations: dv
4 Models and Simulations
Here we will provide a Gaussian and a non-Gaussian approach for:
- PK IV 1-compartment model fit for the Warfarin dataset
- Bioavaliability parallel absorption model simulation
4.1 Warfarin Gaussian and non-Gaussian PK model
The first model is a simple 1-compartment PK IV model with proportional error. This is for the Gaussian versus gamma random effects:
model_lognormal = @model begin
@param begin
θCL ∈ RealDomain(; lower = 0)
θVc ∈ RealDomain(; lower = 0)
ωCL ∈ RealDomain(; lower = 0)
ωVc ∈ RealDomain(; lower = 0)
σ ∈ RealDomain(; lower = 0)
end
@random begin
_CL ~ LogNormal(log(θCL), ωCL)
_Vc ~ LogNormal(log(θVc), ωVc)
end
# This is equivalent to defining
# CL = θCL*exp(ηCL)
# with
# ηCL = Normal(0, ωCL)
@pre begin
CL = _CL
Vc = _Vc
end
@dynamics Central1
@derived begin
μ := @. Central / Vc
dv ~ @. ProportionalNormal(μ, σ)
end
endPumasModel
Parameters: θCL, θVc, ωCL, ωVc, σ
Random effects: _CL, _Vc
Covariates:
Dynamical system variables: Central
Dynamical system type: Closed form
Derived: dv
Observed: dv
model_gamma = @model begin
@param begin
θCL ∈ RealDomain(; lower = 0)
θVc ∈ RealDomain(; lower = 0)
ωCL ∈ RealDomain(; lower = 0)
ωVc ∈ RealDomain(; lower = 0)
σ ∈ RealDomain(; lower = 0)
end
@random begin
_CL ~ Gamma(1 / ωCL^2, θCL * ωCL^2)
_Vc ~ Gamma(1 / ωVc^2, θVc * ωVc^2)
end
@pre begin
CL = _CL
Vc = _Vc
end
@dynamics begin
Central' = -CL / Vc * Central
end
@derived begin
μ := @. Central / Vc
dv ~ @. ProportionalNormal(μ, σ)
end
endPumasModel
Parameters: θCL, θVc, ωCL, ωVc, σ
Random effects: _CL, _Vc
Covariates:
Dynamical system variables: Central
Dynamical system type: Matrix exponential
Derived: dv
Observed: dv
We also need some initial values for the fitting:
iparams_pk = (; θCL = 1.0, θVc = 5.0, ωCL = 0.1, ωVc = 0.1, σ = 0.2)(θCL = 1.0, θVc = 5.0, ωCL = 0.1, ωVc = 0.1, σ = 0.2)
We proceed by fitting both models:
fit_lognormal = fit(model_lognormal, pop, iparams_pk, FOCE())[ Info: Checking the initial parameter values. [ Info: The initial negative log-likelihood and its gradient are finite. Check passed. Iter Function value Gradient norm 0 5.770212e+03 7.912060e+03 * time: 0.047376155853271484 1 9.433464e+02 6.079483e+02 * time: 6.3358049392700195 2 8.189627e+02 4.423725e+02 * time: 6.346259117126465 3 5.917683e+02 1.819248e+02 * time: 6.354077100753784 4 5.421783e+02 1.121313e+02 * time: 6.359986066818237 5 5.255651e+02 7.407230e+01 * time: 6.365612983703613 6 5.208427e+02 8.699271e+01 * time: 6.3705549240112305 7 5.174883e+02 8.974584e+01 * time: 6.375195026397705 8 5.138523e+02 7.328235e+01 * time: 6.380028963088989 9 5.109883e+02 4.155805e+01 * time: 6.384787082672119 10 5.094359e+02 3.170517e+01 * time: 6.3891520500183105 11 5.086172e+02 3.327331e+01 * time: 6.3931989669799805 12 5.080941e+02 2.942077e+01 * time: 6.397565126419067 13 5.074009e+02 2.839941e+01 * time: 6.401540040969849 14 5.059302e+02 3.330093e+01 * time: 6.40529203414917 15 5.036399e+02 3.172884e+01 * time: 6.40952205657959 16 5.017004e+02 3.160020e+01 * time: 6.413973093032837 17 5.008553e+02 2.599524e+01 * time: 6.418406009674072 18 5.005913e+02 2.139314e+01 * time: 6.422608137130737 19 5.003573e+02 2.134778e+01 * time: 6.426455974578857 20 4.997249e+02 2.069868e+01 * time: 6.430653095245361 21 4.984453e+02 1.859010e+01 * time: 6.4348649978637695 22 4.959584e+02 2.156209e+01 * time: 6.43953800201416 23 4.923347e+02 3.030833e+01 * time: 6.444739103317261 24 4.906916e+02 1.652278e+01 * time: 6.449924945831299 25 4.902955e+02 6.360800e+00 * time: 6.454648017883301 26 4.902248e+02 1.151521e+00 * time: 6.463510036468506 27 4.902191e+02 1.174457e+00 * time: 6.467755079269409 28 4.902189e+02 1.171564e+00 * time: 6.471411943435669 29 4.902184e+02 1.164556e+00 * time: 6.474977970123291 30 4.902169e+02 1.143206e+00 * time: 6.478655099868774 31 4.902134e+02 1.089000e+00 * time: 6.483134984970093 32 4.902052e+02 1.135407e+00 * time: 6.487934112548828 33 4.901885e+02 1.547068e+00 * time: 6.492768049240112 34 4.901641e+02 1.507949e+00 * time: 6.497530937194824 35 4.901455e+02 8.269239e-01 * time: 6.502656936645508 36 4.901410e+02 2.300152e-01 * time: 6.507610082626343 37 4.901406e+02 3.097984e-02 * time: 6.512316942214966 38 4.901405e+02 5.453246e-03 * time: 6.516749143600464 39 4.901405e+02 1.203230e-03 * time: 6.520477056503296 40 4.901405e+02 6.315365e-05 * time: 6.523711919784546
FittedPumasModel
Dynamical system type: Closed form
Number of subjects: 32
Observation records: Active Missing
dv: 256 0
Total: 256 0
Number of parameters: Constant Optimized
0 5
Likelihood approximation: FOCE
Likelihood optimizer: BFGS
Termination Reason: GradientNorm
Log-likelihood value: -490.14053
---------------
Estimate
---------------
θCL 0.16025
θVc 10.262
ωCL 0.23504
ωVc 0.10449
σ 0.3582
---------------
fit_gamma = fit(model_gamma, pop, iparams_pk, FOCE())[ Info: Checking the initial parameter values. [ Info: The initial negative log-likelihood and its gradient are finite. Check passed. Iter Function value Gradient norm 0 4.160707e+03 5.960578e+03 * time: 4.887580871582031e-5 1 9.056373e+02 5.541920e+02 * time: 3.6705398559570312 2 7.787963e+02 3.988904e+02 * time: 3.682436943054199 3 5.915054e+02 1.495777e+02 * time: 3.691776990890503 4 5.533120e+02 8.826583e+01 * time: 3.7005598545074463 5 5.389239e+02 9.144086e+01 * time: 3.709040880203247 6 5.323499e+02 1.000933e+02 * time: 3.7175159454345703 7 5.270252e+02 8.423844e+01 * time: 3.7256829738616943 8 5.233813e+02 5.194402e+01 * time: 3.7335400581359863 9 5.213366e+02 3.461331e+01 * time: 3.741499900817871 10 5.200972e+02 3.888113e+01 * time: 3.7493739128112793 11 5.191933e+02 3.556605e+01 * time: 3.7571399211883545 12 5.181335e+02 3.624436e+01 * time: 3.7650158405303955 13 5.161626e+02 4.322775e+01 * time: 3.7729949951171875 14 5.133202e+02 3.722515e+01 * time: 3.7807278633117676 15 5.107758e+02 3.401586e+01 * time: 3.788648843765259 16 5.095157e+02 2.854997e+01 * time: 3.796915054321289 17 5.090165e+02 2.644560e+01 * time: 3.804866075515747 18 5.085184e+02 2.744429e+01 * time: 3.812628984451294 19 5.074309e+02 2.793918e+01 * time: 3.820399045944214 20 5.053757e+02 2.616169e+01 * time: 3.828248977661133 21 5.018507e+02 2.257667e+01 * time: 3.8364288806915283 22 4.942495e+02 3.832878e+01 * time: 3.8456239700317383 23 4.916504e+02 3.167965e+01 * time: 3.8650810718536377 24 4.902302e+02 1.020055e+01 * time: 3.8760569095611572 25 4.898632e+02 6.068208e+00 * time: 3.8863558769226074 26 4.898142e+02 1.688453e+00 * time: 3.896475076675415 27 4.897942e+02 6.439288e-01 * time: 3.9066810607910156 28 4.897941e+02 6.464271e-01 * time: 3.9152488708496094 29 4.897939e+02 6.470322e-01 * time: 4.529684066772461 30 4.897933e+02 6.434520e-01 * time: 4.540042877197266 31 4.897921e+02 7.468240e-01 * time: 4.54696798324585 32 4.897892e+02 1.271556e+00 * time: 4.554199934005737 33 4.897836e+02 1.739405e+00 * time: 4.5614330768585205 34 4.897761e+02 1.644118e+00 * time: 4.568567991256714 35 4.897711e+02 7.983427e-01 * time: 4.57574200630188 36 4.897701e+02 1.702950e-01 * time: 4.582598924636841 37 4.897700e+02 1.208261e-02 * time: 4.5889599323272705 38 4.897700e+02 2.536842e-03 * time: 4.594806909561157 39 4.897700e+02 3.067685e-04 * time: 4.599622964859009
FittedPumasModel
Dynamical system type: Matrix exponential
Number of subjects: 32
Observation records: Active Missing
dv: 256 0
Total: 256 0
Number of parameters: Constant Optimized
0 5
Likelihood approximation: FOCE
Likelihood optimizer: BFGS
Termination Reason: GradientNorm
Log-likelihood value: -489.77002
---------------
Estimate
---------------
θCL 0.16466
θVc 10.329
ωCL 0.23348
ωVc 0.10661
σ 0.35767
---------------
Finally, let’s compare the estimates:
compare_estimates(; lognormal = fit_lognormal, gamma = fit_gamma)| Row | parameter | lognormal | gamma |
|---|---|---|---|
| String | Float64? | Float64? | |
| 1 | θCL | 0.160253 | 0.164658 |
| 2 | θVc | 10.2617 | 10.3288 |
| 3 | ωCL | 0.23504 | 0.233485 |
| 4 | ωVc | 0.104494 | 0.106612 |
| 5 | σ | 0.358204 | 0.357666 |
As mention above, the mean of a log-normal is \(\exp \left\{ \mu + \frac{\sigma^2}{2} \right\}\).
So let’s compare that with the gamma typical values:
DataFrame(;
parameter = ["θCL", "θVc"],
θLogNormal = [coef(fit_lognormal).θCL, coef(fit_lognormal).θVc],
ELogNormal = [
exp(log(coef(fit_lognormal).θCL) + coef(fit_lognormal).ωCL^2 / 2),
exp(log(coef(fit_lognormal).θVc) + coef(fit_lognormal).ωVc^2 / 2),
],
θGamma = [coef(fit_gamma).θCL, coef(fit_gamma).θVc],
)| Row | parameter | θLogNormal | ELogNormal | θGamma |
|---|---|---|---|---|
| String | Float64 | Float64 | Float64 | |
| 1 | θCL | 0.160253 | 0.164741 | 0.164658 |
| 2 | θVc | 10.2617 | 10.3178 | 10.3288 |
As you can see the Gaussian model has a slight bias in the estimation of both θCL and θVc.
Let’s also plot the two probability density functions (PDF) for θCL:
plotdataPK = @chain DataFrame(; x = range(0, 0.5; length = 1_000)) begin
@rtransform begin
:LogNormal =
pdf(LogNormal(log(coef(fit_lognormal).θCL), coef(fit_lognormal).ωCL), :x)
:Gamma = pdf(LogNormal(log(coef(fit_gamma).θCL), coef(fit_gamma).ωCL), :x)
end
end
first(plotdataPK, 5)| Row | x | LogNormal | Gamma |
|---|---|---|---|
| Float64 | Float64 | Float64 | |
| 1 | 0.0 | 0.0 | 0.0 |
| 2 | 0.000500501 | 5.20157e-128 | 5.26158e-131 |
| 3 | 0.001001 | 9.15989e-99 | 3.24782e-101 |
| 4 | 0.0015015 | 2.08275e-83 | 1.45738e-85 |
| 5 | 0.002002 | 2.69537e-73 | 2.98228e-75 |
data(stack(plotdataPK, [:LogNormal, :Gamma])) *
mapping(:x, :value; color = :variable) *
visual(Lines) |> draw4.2 Bioavaliability Parallel Absorption Simulation
This is a parallel absorption model with bioavaliabity in both the “fast” as the “slow” depots.
First, the traditional approach with a logistic transformation of a Gaussian random variable. This makes the individual relative bioavailibility logit-normally distributed.
model_logitnormal = @model begin
@param begin
θkaFast ∈ RealDomain(; lower = 0)
θkaSlow ∈ RealDomain(; lower = 0)
θCL ∈ RealDomain(; lower = 0)
θVc ∈ RealDomain(; lower = 0)
θbioav ∈ RealDomain(; lower = 0.0, upper = 1.0)
ωCL ∈ RealDomain(; lower = 0)
ωVc ∈ RealDomain(; lower = 0)
# I call this one ξ to distinguish it from ω since the interpretation is NOT a relative error (coefficient of variation)
ξbioav ∈ RealDomain(; lower = 0, init = 0.1)
σ ∈ RealDomain(; lower = 0)
end
@random begin
_CL ~ Gamma(1 / ωCL^2, θCL * ωCL^2)
_Vc ~ Gamma(1 / ωVc^2, θVc * ωVc^2)
# define the latent Gaussian random effect. Notice the logit transform
ηbioavLogit ~ Normal(logit(θbioav), ξbioav)
end
@pre begin
kaFast = θkaFast
kaSlow = θkaSlow
CL = _CL
Vc = _Vc
end
@dosecontrol begin
# _bioav is LogitNormal distributed
_bioav := logistic(ηbioavLogit)
bioav = (; DepotFast = _bioav, DepotSlow = 1 - _bioav)
end
@dynamics begin
DepotFast' = -kaFast * DepotFast
DepotSlow' = -kaSlow * DepotSlow
Central' = kaFast * DepotFast + kaSlow * DepotSlow - CL / Vc * Central
end
@derived begin
μ := @. Central / Vc
dv ~ @. ProportionalNormal(μ, σ)
end
endPumasModel
Parameters: θkaFast, θkaSlow, θCL, θVc, θbioav, ωCL, ωVc, ξbioav, σ
Random effects: _CL, _Vc, ηbioavLogit
Covariates:
Dynamical system variables: DepotFast, DepotSlow, Central
Dynamical system type: Matrix exponential
Derived: dv
Observed: dv
Now the same model but with the non-Gaussian random-effects using a beta distribution instead of the logit parameterization of the Gaussian distribution:
model_beta = @model begin
@param begin
θkaFast ∈ RealDomain(; lower = 0)
θkaSlow ∈ RealDomain(; lower = 0)
θCL ∈ RealDomain(; lower = 0)
θVc ∈ RealDomain(; lower = 0)
θbioav ∈ RealDomain(; lower = 0.0, upper = 1.0)
ωCL ∈ RealDomain(; lower = 0)
ωVc ∈ RealDomain(; lower = 0)
# We call this one n since the interpretation is like the length of a Binomial distribution
nbioav ∈ RealDomain(; lower = 0, init = 10)
σ ∈ RealDomain(; lower = 0)
end
@random begin
ηCL ~ Gamma(1 / ωCL^2, θCL * ωCL^2)
ηVc ~ Gamma(1 / ωVc^2, θVc * ωVc^2)
# The makes E(_bioav) = θbioav
# See https://en.wikipedia.org/wiki/Beta_distribution
ηbioav ~ Beta(θbioav * nbioav, (1 - θbioav) * nbioav)
end
@pre begin
kaFast = θkaFast
kaSlow = θkaSlow
CL = ηCL
Vc = ηVc
end
@dosecontrol begin
bioav = (; DepotFast = ηbioav, DepotSlow = 1 - ηbioav)
end
@dynamics begin
DepotFast' = -kaFast * DepotFast
DepotSlow' = -kaSlow * DepotSlow
Central' = kaFast * DepotFast + kaSlow * DepotSlow - CL / Vc * Central
end
@derived begin
μ := @. Central / Vc
dv ~ @. ProportionalNormal(μ, σ)
end
endPumasModel
Parameters: θkaFast, θkaSlow, θCL, θVc, θbioav, ωCL, ωVc, nbioav, σ
Random effects: ηCL, ηVc, ηbioav
Covariates:
Dynamical system variables: DepotFast, DepotSlow, Central
Dynamical system type: Matrix exponential
Derived: dv
Observed: dv
We have two types of random effects here.
First, as you are already familiar from the previous example, the clearance (CL), volume of concentration (Vc), and absorption rate (ka) have typical values (i.e. fixed effects) and between-subject variability (i.e. random effects) modelled as a gamma distribution.
Second, bioavailability Bioav is modelled as a beta distribution. Generally the beta distribution is parametrized as:
\[\text{Beta}(\alpha, \beta)\]
where both parameters \(\alpha\) and \(\beta\) are shape parameters.
One nice thing about the beta distribution is that it only takes values between and including 0 and 1, i.e. \([0, 1]\). This makes it the perfect candidate to model bioavailability parameters which are generally bounded in that interval. So, we don’t need to do a logistic transformation.
Another nice thing about the beta distribution is that we can use the alternative \((\mu, n)\)-parametrization with with \(\mu\) serving as a mean-value parameter:
\[\text{Beta}(\mu, n)\]
where in the original beta parametrization:
- \(\alpha = \mu n\)
- \(\beta = (1 - \mu) n\)
Hence, our mean is:
\[\operatorname{E}[F] = \mu = \theta_F\]
which, again, does not depend on any other parameters. The variance is
\[\operatorname{Var}(F) = \frac{\mu(1 - \mu)}{n}\]
so similar to the mean of Bernoulli trials.
Now let’s generate some data for the simulation:
dr = DosageRegimen(
DosageRegimen(100; cmt = :DepotFast),
DosageRegimen(100; cmt = :DepotSlow),
)| Row | time | cmt | amt | evid | ii | addl | rate | duration | ss |
|---|---|---|---|---|---|---|---|---|---|
| Float64 | Symbol | Float64 | Int8 | Float64 | Int64 | Float64 | Float64 | Int8 | |
| 1 | 0.0 | DepotFast | 100.0 | 1 | 0.0 | 0 | 0.0 | 0.0 | 0 |
| 2 | 0.0 | DepotSlow | 100.0 | 1 | 0.0 | 0 | 0.0 | 0.0 | 0 |
simtimes = [0.5, 1.0, 2.0, 4.0, 8.0, 24.0]6-element Vector{Float64}:
0.5
1.0
2.0
4.0
8.0
24.0
trueparam = (;
θkaFast = 0.9,
θkaSlow = 0.2,
θCL = 1.1,
θVc = 10.0,
θbioav = 0.7,
ωCL = 0.1,
ωVc = 0.1,
nbioav = 40,
σ = 0.1,
)(θkaFast = 0.9, θkaSlow = 0.2, θCL = 1.1, θVc = 10.0, θbioav = 0.7, ωCL = 0.1, ωVc = 0.1, nbioav = 40, σ = 0.1)
For simplicity, we just add 20% to the true values for initial values:
initparamBeta = map(t -> 1.2 * t, trueparam)(θkaFast = 1.08, θkaSlow = 0.24, θCL = 1.32, θVc = 12.0, θbioav = 0.84, ωCL = 0.12, ωVc = 0.12, nbioav = 48.0, σ = 0.12)
The initial values for the LogitNormal need to have ξbioav defined instead of nbioav:
initparamLogitNormal =
(Base.structdiff(initparamBeta, NamedTuple{(:nbioav,)})..., ξbioav = 0.1)(θkaFast = 1.08, θkaSlow = 0.24, θCL = 1.32, θVc = 12.0, θbioav = 0.84, ωCL = 0.12, ωVc = 0.12, σ = 0.12, ξbioav = 0.1)
Setup empty Subjects with the dose information:
skeletonpop = map(i -> Subject(; id = i, events = dr), 1:40)Population
Subjects: 40
Observations:
Next, we simulate the data (while setting the seed for reprocibility):
Random.seed!(128)
simpop = Subject.(simobs(model_beta, skeletonpop, trueparam; obstimes = simtimes))Finally let’s fit both models:
fit_logitnormal = fit(model_logitnormal, simpop, initparamLogitNormal, FOCE())[ Info: Checking the initial parameter values. [ Info: The initial negative log-likelihood and its gradient are finite. Check passed. Iter Function value Gradient norm 0 2.512964e+02 3.650794e+02 * time: 2.9087066650390625e-5 1 2.097177e+02 2.581791e+02 * time: 13.010613918304443 2 1.912461e+02 8.737199e+01 * time: 13.091485023498535 3 1.887358e+02 6.798862e+01 * time: 13.166548013687134 4 1.843518e+02 1.520669e+01 * time: 13.2478609085083 5 1.833150e+02 1.305346e+01 * time: 13.296052932739258 6 1.825488e+02 1.030554e+01 * time: 13.343508958816528 7 1.811422e+02 8.012892e+00 * time: 13.393826007843018 8 1.810538e+02 3.075453e+00 * time: 13.44311809539795 9 1.810370e+02 2.322234e+00 * time: 13.49193787574768 10 1.810214e+02 1.641742e+00 * time: 13.54140305519104 11 1.808623e+02 9.955746e+00 * time: 13.5954430103302 12 1.806361e+02 2.328715e+01 * time: 13.64899492263794 13 1.803677e+02 2.160846e+01 * time: 13.704339027404785 14 1.796608e+02 1.029944e+01 * time: 13.758073091506958 15 1.796130e+02 2.787719e+01 * time: 13.813677072525024 16 1.793909e+02 1.479965e+01 * time: 13.870265007019043 17 1.792909e+02 1.128584e+01 * time: 13.928252935409546 18 1.792571e+02 7.571530e+00 * time: 13.983803987503052 19 1.791694e+02 7.130415e+00 * time: 14.041668891906738 20 1.790947e+02 1.587148e+01 * time: 14.097861051559448 21 1.789919e+02 9.984290e+00 * time: 14.154147863388062 22 1.788772e+02 9.117968e+00 * time: 14.211390972137451 23 1.788396e+02 7.991913e+00 * time: 14.265094995498657 24 1.788017e+02 2.044978e+00 * time: 14.316598892211914 25 1.787978e+02 1.967777e+00 * time: 14.365660905838013 26 1.787814e+02 1.676898e+00 * time: 14.419565916061401 27 1.787796e+02 8.236497e-01 * time: 14.472259044647217 28 1.787780e+02 3.610259e-01 * time: 14.907938957214355 29 1.787762e+02 4.980138e-01 * time: 14.960705041885376 30 1.787739e+02 1.025037e+00 * time: 15.01301097869873 31 1.787721e+02 1.497724e+00 * time: 15.064081907272339 32 1.787704e+02 1.294175e+00 * time: 15.115048885345459 33 1.787674e+02 1.360458e+00 * time: 15.164702892303467 34 1.787612e+02 1.811390e+00 * time: 15.212805032730103 35 1.787464e+02 2.634680e+00 * time: 15.262541055679321 36 1.787141e+02 3.363820e+00 * time: 15.311946868896484 37 1.786632e+02 3.060364e+00 * time: 15.361937046051025 38 1.786277e+02 1.344290e+00 * time: 15.413615942001343 39 1.786192e+02 2.524868e-01 * time: 15.46520209312439 40 1.786166e+02 6.759174e-01 * time: 15.517277956008911 41 1.786136e+02 8.208435e-01 * time: 15.563062906265259 42 1.786106e+02 4.973292e-01 * time: 15.607578992843628 43 1.786088e+02 3.537716e-01 * time: 15.65540599822998 44 1.786083e+02 1.527595e-01 * time: 15.70212197303772 45 1.786080e+02 1.812419e-01 * time: 15.749612092971802 46 1.786078e+02 1.660844e-01 * time: 15.795186996459961 47 1.786077e+02 5.979074e-02 * time: 15.841912031173706 48 1.786076e+02 4.897633e-02 * time: 15.888545036315918 49 1.786076e+02 3.830216e-02 * time: 15.933785915374756 50 1.786076e+02 2.623948e-02 * time: 15.98161005973816 51 1.786076e+02 6.769983e-03 * time: 16.02837109565735 52 1.786076e+02 5.050303e-03 * time: 16.075736045837402 53 1.786076e+02 7.621588e-03 * time: 16.121201992034912 54 1.786076e+02 4.582334e-03 * time: 16.167496919631958 55 1.786076e+02 6.267388e-04 * time: 16.213690996170044
FittedPumasModel
Dynamical system type: Matrix exponential
Number of subjects: 40
Observation records: Active Missing
dv: 240 0
Total: 240 0
Number of parameters: Constant Optimized
0 9
Likelihood approximation: FOCE
Likelihood optimizer: BFGS
Termination Reason: GradientNorm
Log-likelihood value: -178.60759
---------------------
Estimate
---------------------
θkaFast 0.91098
θkaSlow 0.13112
θCL 1.0854
θVc 7.1008
θbioav 0.4802
ωCL 0.088113
ωVc 0.12133
ξbioav 0.00017116
σ 0.10545
---------------------
fit_beta = fit(model_beta, simpop, initparamBeta, FOCE())[ Info: Checking the initial parameter values. [ Info: The initial negative log-likelihood and its gradient are finite. Check passed. Iter Function value Gradient norm 0 2.523111e+02 3.644346e+02 * time: 2.6941299438476562e-5 1 2.098955e+02 2.554795e+02 * time: 2.530000925064087 2 1.914622e+02 7.988189e+01 * time: 2.614223003387451 3 1.877135e+02 8.496056e+01 * time: 2.6643588542938232 4 1.851001e+02 4.097104e+01 * time: 2.7422869205474854 5 1.838808e+02 1.323193e+01 * time: 2.7984750270843506 6 1.831532e+02 1.415278e+01 * time: 2.8543128967285156 7 1.816636e+02 1.862107e+01 * time: 2.911907911300659 8 1.815171e+02 3.953485e+00 * time: 3.3297810554504395 9 1.814961e+02 1.940229e+00 * time: 3.380063056945801 10 1.814647e+02 4.319730e+00 * time: 3.4285800457000732 11 1.812769e+02 1.765417e+01 * time: 3.4779770374298096 12 1.810778e+02 2.806019e+01 * time: 3.528217077255249 13 1.807917e+02 2.968381e+01 * time: 3.578618049621582 14 1.802819e+02 1.547090e+01 * time: 3.62937593460083 15 1.801090e+02 1.366244e+00 * time: 3.6788289546966553 16 1.800821e+02 6.329956e+00 * time: 3.7514379024505615 17 1.800545e+02 1.125074e+00 * time: 3.800477981567383 18 1.800446e+02 1.561431e+00 * time: 3.8490400314331055 19 1.800177e+02 2.451310e+00 * time: 3.899388074874878 20 1.799967e+02 2.539586e+00 * time: 3.949277877807617 21 1.799753e+02 1.523139e+00 * time: 4.0012829303741455 22 1.799469e+02 2.173685e+00 * time: 4.055161952972412 23 1.798979e+02 8.681997e+00 * time: 4.109477996826172 24 1.798362e+02 7.272561e+00 * time: 4.164207935333252 25 1.797469e+02 1.490516e+01 * time: 4.2184529304504395 26 1.796654e+02 8.718280e+00 * time: 4.292547941207886 27 1.795920e+02 9.410186e+00 * time: 4.347012042999268 28 1.795394e+02 5.212003e+00 * time: 4.400117874145508 29 1.794630e+02 4.927958e+00 * time: 4.453466892242432 30 1.793764e+02 5.159988e+00 * time: 4.507130861282349 31 1.793446e+02 7.307439e+00 * time: 4.66205096244812 32 1.793227e+02 1.964089e+00 * time: 4.733603000640869 33 1.793181e+02 1.829483e+00 * time: 4.782405853271484 34 1.793118e+02 1.092217e+00 * time: 4.832676887512207 35 1.793067e+02 1.256868e+00 * time: 4.879111051559448 36 1.792666e+02 4.665942e+00 * time: 4.925428867340088 37 1.792113e+02 7.305936e+00 * time: 4.971827030181885 38 1.791121e+02 8.705330e+00 * time: 5.025462865829468 39 1.789925e+02 7.398610e+00 * time: 5.077718019485474 40 1.788294e+02 5.245360e+00 * time: 5.129773855209351 41 1.787071e+02 9.501500e+00 * time: 5.182111024856567 42 1.786762e+02 3.851030e+00 * time: 5.236105918884277 43 1.786560e+02 1.039973e+00 * time: 5.291898965835571 44 1.786386e+02 2.307784e+00 * time: 5.344501972198486 45 1.786286e+02 1.623904e+00 * time: 5.398983955383301 46 1.786169e+02 7.950987e-01 * time: 5.454668045043945 47 1.786129e+02 1.210835e+00 * time: 5.510704040527344 48 1.786101e+02 6.972927e-01 * time: 5.563253879547119 49 1.786088e+02 1.656571e-01 * time: 5.614686965942383 50 1.786082e+02 1.785747e-01 * time: 5.669759035110474 51 1.786080e+02 2.576510e-01 * time: 5.724600076675415 52 1.786078e+02 1.991902e-01 * time: 5.8130738735198975 53 1.786077e+02 7.973891e-02 * time: 5.865697860717773 54 1.786076e+02 1.247772e-02 * time: 5.914049863815308 55 1.786076e+02 5.112672e-02 * time: 5.964370012283325 56 1.786076e+02 5.327180e-02 * time: 6.014150857925415 57 1.786076e+02 3.122404e-02 * time: 6.068072080612183 58 1.786076e+02 4.869464e-03 * time: 6.121379852294922 59 1.786076e+02 4.482959e-03 * time: 6.189954996109009 60 1.786076e+02 2.120120e-03 * time: 6.2413389682769775 61 1.786076e+02 2.553593e-03 * time: 6.286244869232178 62 1.786076e+02 3.407646e-03 * time: 6.363097906112671 63 1.786076e+02 7.386995e-04 * time: 6.412564039230347
FittedPumasModel
Dynamical system type: Matrix exponential
Number of subjects: 40
Observation records: Active Missing
dv: 240 0
Total: 240 0
Number of parameters: Constant Optimized
0 9
Likelihood approximation: FOCE
Likelihood optimizer: BFGS
Termination Reason: GradientNorm
Log-likelihood value: -178.60759
-------------------
Estimate
-------------------
θkaFast 0.91098
θkaSlow 0.13112
θCL 1.0854
θVc 7.1008
θbioav 0.4802
ωCL 0.088113
ωVc 0.12133
nbioav 2.8889e7
σ 0.10545
-------------------
As before, let’s compare the estimates:
compare_estimates(; logitnormal = fit_logitnormal, beta = fit_beta)| Row | parameter | logitnormal | beta |
|---|---|---|---|
| String | Float64? | Float64? | |
| 1 | θkaFast | 0.910979 | 0.910981 |
| 2 | θkaSlow | 0.131118 | 0.131118 |
| 3 | θCL | 1.0854 | 1.0854 |
| 4 | θVc | 7.10083 | 7.10079 |
| 5 | θbioav | 0.480204 | 0.4802 |
| 6 | ωCL | 0.0881133 | 0.0881134 |
| 7 | ωVc | 0.121334 | 0.121334 |
| 8 | σ | 0.105448 | 0.105449 |
| 9 | ξbioav | 0.000171162 | missing |
| 10 | nbioav | missing | 2.88887e7 |
Again, we’ll both PDFs from the estimated values:
plotdatabioav = @chain DataFrame(; x = range(0, 1; length = 1_000)) begin
@rtransform begin
:logitnormal =
1 / coef(fit_logitnormal).ξbioav / √(2π) / (:x * (1 - :x)) * exp(
-(logit(:x) - logit(coef(fit_logitnormal).θbioav))^2 /
(2 * coef(fit_logitnormal).ξbioav^2),
)
:beta = pdf(
Beta(
coef(fit_beta).θbioav * coef(fit_beta).nbioav,
(1 - coef(fit_beta).θbioav) * coef(fit_beta).nbioav,
),
:x,
)
end
end
first(plotdatabioav, 5)| Row | x | logitnormal | beta |
|---|---|---|---|
| Float64 | Float64 | Float64 | |
| 1 | 0.0 | NaN | 0.0 |
| 2 | 0.001001 | 0.0 | 0.0 |
| 3 | 0.002002 | 0.0 | 0.0 |
| 4 | 0.003003 | 0.0 | 0.0 |
| 5 | 0.004004 | 0.0 | 0.0 |
plt_pdf_bioav =
data(stack(plotdatabioav, [:logitnormal, :beta])) *
mapping(:x, :value; color = :variable) *
visual(Lines);
draw(plt_pdf_bioav; axis = (; xticks = 0.1:0.1:1.0))For this dataset, the two distributions differ significantly with the Beta model producing a distribution much closer to the truth but for other realizations of the simulated data they are closer to each other.