Why are non-Gaussian random effects relevant?

Authors

Jose Storopoli

Andreas Noack

using Dates
using Pumas
using PumasUtilities
using DataFramesMeta
using PharmaDatasets
using CairoMakie
using AlgebraOfGraphics
using Random

1 Motivation - PK model

Why using a non-Gaussian distribution as the underlying distribution for the random effects? There are a couple of arguments.

First, the Gaussian distribution has unbounded support, i.e. it take any value in \((-\infty, \infty)\). While phamacokinetic parameters typically are (semi) bounded, e.g.:

  • clearance and volumes, \((0, \infty)\)
  • bioavailability, \([0, 1]\)

Additionally, in order for a Gaussian distribution to work as the underlying distribution, often we need to transform them (e.g. exponentiation and logistic transformation). But these transformations in some settings, when the random effects do not have a great impact, i.e. they do not have large values, may shift the mean of the typical values (\(\theta\)) so that the expectation of the typical values (\(\operatorname{E}\)) are not equal to the mean. For example, the following code block is a traditional 1-compartment PK model with a Gaussian random effect that needs to be constrained to positive values, \((0, \infty)\):

@random begin
    ηCL ~ Normal(0.0, ωCL)
    ηVc ~ Normal(0.0, ωVc)
end
@pre begin
    CL = θCL * exp(ηCL)
    Vc = θVc * exp(ηVc)
end
@dynamics begin
    Central' = -CL / Vc * Central
end

If we recover the formula for the expectation of the log-normal distribution, we have that:

\[\operatorname{E}[CL] = \exp \left\{ \log(\theta_{CL}) + \frac{\omega^2_{CL}}{2} \right\} \approx \theta_{CL}\]

This approximation only holds for small \(\omega_{CL}\).

Hence, \(\theta_{CL}\) is only the typical value when \(\omega_{CL}\) is small.

Here is a small tabulation for \(\operatorname{E}[CL]\) when \(\theta_{CL} = 0.5\):

ωs = [0.1, 0.2, 0.4, 0.8, 1.6]
DataFrame(; ω_CL = ωs, E_CL = (ω -> exp(log(0.5) + ω^2 / 2)).(ωs))
5×2 DataFrame
Row ω_CL E_CL
Float64 Float64
1 0.1 0.502506
2 0.2 0.510101
3 0.4 0.541644
4 0.8 0.688564
5 1.6 1.79832

As you can see, the larger the \(\omega_{CL}\) the more \(\operatorname{E}[CL]\) deviates from \(\theta_{CL}\).

1.1 Gamma distribution for the rescue

We can use the gamma distribution which has the following parametrization:

\[\text{Gamma}(k, \theta)\]

where \(k\) is a shape parameter and \(\theta\) is a scale parameter.

Note

Shape parameters generally control the shape of the distribution rather than shifting it (as a location parameter) of stretching/shrinking it (as a scale parameter)

We can use an alternative parametrization where the mean-value appears directly a parameter:

\[\text{Gamma}(\mu, \sigma)\]

where:

  • \(\mu = \theta k\)
  • \(\sigma = k^{-\frac{1}{2}}\)
Tip

The \(\sigma\) parameter is the coefficient of variation, i.e.

\[\sigma = \frac{\operatorname{Var} X}{\operatorname{E} X},\]

because that mimics the role of \(\sigma\) in the LogNormal(log(μ), σ) where for small values of \(\sigma\)

\[\sigma \approx \\frac{\operatorname{Var} X}{\operatorname{E} X}.\]

So, our previous PK model now becomes:

@random begin
    _CL ~ Gamma(1 / ωCL^2, θCL * ωCL^2)
    _Vc ~ Gamma(1 / ωVc^2, θVc * ωVc^2)
end
@pre begin
    CL = _CL
    Vc = _Vc
end
@dynamics begin
    Central' = -CL / Vc * Central
end

As you can see the mean from the gamma distribution becomes:

\[\operatorname{E}[CL] = \theta k = \frac{1}{\omega^2_{CL}} \theta_{CL} \omega^2_{CL} = \theta_{CL}\]

It does not dependent on the between-subject variability \(\omega\)!

Note

We are avoiding η notation here since we are modeling the subject-specific parameter directly.

1.2 Gamma versus Log-Nogmal Numerical Simulations

Before we dive into our PK examples, let us showcase the case for gamma versus log-normal with some numerical simulations.

First, let’s define a mean μ_PK value for a typical value along with an array of possible standard deviations σ values:

μ_PK = 1.0
σ = [0.1, 0.2, 0.5, 1.0, 1.5, 2.0]

These will serve as the mean and standard deviations for our gamma and log-normal distributions.

Now let’s compare the coefficient of variation (CV) as a function of σ for LogNormal and Gamma:

num_df_gamma = DataFrame(;
    μ = μ_PK,
    σ = σ,
    meanLogNormal = mean.(LogNormal.(log.(μ_PK), σ)),
    cvLogNormal = std.(LogNormal.(log.(μ_PK), σ)) ./ mean.(LogNormal.(log.(μ_PK), σ)),
    meanGamma = mean.(Gamma.(1 ./ σ .^ 2, μ_PK .* σ .^ 2)),
    cvGamma = std.(Gamma.(1 ./ σ .^ 2, μ_PK .* σ .^ 2)) ./
              mean.(Gamma.(1 ./ σ .^ 2, μ_PK .* σ .^ 2)),
)
6×6 DataFrame
Row μ σ meanLogNormal cvLogNormal meanGamma cvGamma
Float64 Float64 Float64 Float64 Float64 Float64
1 1.0 0.1 1.00501 0.100251 1.0 0.1
2 1.0 0.2 1.0202 0.202017 1.0 0.2
3 1.0 0.5 1.13315 0.53294 1.0 0.5
4 1.0 1.0 1.64872 1.31083 1.0 1.0
5 1.0 1.5 3.08022 2.91337 1.0 1.5
6 1.0 2.0 7.38906 7.32108 1.0 2.0
f, ax, plotobj = lines(
    num_df_gamma.σ,
    num_df_gamma.meanLogNormal;
    label = "μ - LogNormal",
    linewidth = 3,
    axis = (; xlabel = L"\sigma", ylabel = "value"),
)
lines!(
    num_df_gamma.σ,
    num_df_gamma.meanGamma;
    label = "μ - Gamma",
    linewidth = 3,
    linestyle = :dashdot,
)
lines!(num_df_gamma.σ, num_df_gamma.cvLogNormal; label = "CV - LogNormal", linewidth = 3)
lines!(
    num_df_gamma.σ,
    num_df_gamma.cvGamma;
    label = "CV - Gamma",
    linewidth = 3,
    linestyle = :dashdot,
)
axislegend(ax; position = :lt)
f

In the graph above, the dashed lines correspond to the mean and CV for the gamma distribution, whereas the solid lines correspond to the log-normal distribution.

There is clearly a bias in both the log-normal’s mean and CV that we don’t see in the gamma distribution.

2 Motivation - Bioavailability

Here is a very common model that can benefit from a non-Gaussian random effects distribution.

The model has one-compartment elimination and oral absorption with modeled bioavailability based on a crossover design.

The following code is a traditional PK model with a Gaussian random effect that needs to be constrained to the unit interval, \([0, 1]\):

@param begin
    θF ∈ RealDomain(lower = 0.0, upper = 1.0)
    ωF ∈ RealDomain(lower = 0.0)
end
@random begin
    ηF ~ Normal(0.0, ωF)
end
@dosecontrol begin
    bioav = (Depot = logistic(logit(θF) + ηF),)
end

The expectation \(\operatorname{E}[F]\) doesn’t have closed form and is generally different from \(\theta_F\). However, we have that:

\[\operatorname{E}[F] \approx \theta_F\]

when \(ωF\) is small. I.e. \(\theta_F\) is only the typical value when \(ωF\) is small.

2.1 Beta versus Logit-Normal Numerical Simulations

Let’s perform the same type of simulations we did before, but now we will be using the numerical integrator quadgk from the QuadGK.jl package. This is because we don’t have a closed form solution for \(\operatorname{E}[F]\) in the logit-normal parameterization.

using QuadGK: quadgk
μ_bioav = 0.7

We’ll also reuse the same σ values for the CVs.

num_df_beta = DataFrame(;
    μ = μ_bioav,
    σ = σ,
    meanLogitNormal = map(
        σ -> quadgk(
            t -> logistic(t) * pdf(Normal(logit(μ_bioav), σ), t),
            -100 * σ,
            100 * σ,
        )[1],
        σ,
    ),
    meanBeta = mean.(Beta.(μ_bioav ./ σ, (1 - μ_bioav) ./ σ)),
)
6×4 DataFrame
Row μ σ meanLogitNormal meanBeta
Float64 Float64 Float64 Float64
1 0.7 0.1 0.699582 0.7
2 0.7 0.2 0.698345 0.7
3 0.7 0.5 0.690393 0.7
4 0.7 1.0 0.668971 0.7
5 0.7 1.5 0.646064 0.7
6 0.7 2.0 0.626038 0.7
f, ax, plotobj = lines(
    num_df_beta.σ,
    num_df_beta.meanLogitNormal;
    label = "μ - LogitNormal",
    linewidth = 3,
    axis = (; xlabel = L"\sigma", ylabel = "value"),
)
lines!(
    num_df_beta.σ,
    num_df_beta.meanBeta;
    label = "μ - Beta",
    linewidth = 3,
    linestyle = :dashdot,
)
axislegend(ax; position = :lb)
f

In the graph above, the dashed lines correspond to the mean for the beta distribution, whereas the solid lines correspond to the logit-normal distribution.

As before, there is clearly a bias in the logit-normal’s mean that we don’t see in the beta distribution.

3 Warfarin data

We’ll demonstrate those intuitions using the Warfarin dataset.

pop = read_pumas(dataset("pumas/warfarin"))
Population
  Subjects: 32
  Observations: dv

4 Models and Simulations

Here we will provide a Gaussian and a non-Gaussian approach for:

  • PK IV 1-compartment model fit for the Warfarin dataset
  • Bioavaliability parallel absorption model simulation

4.1 Warfarin Gaussian and non-Gaussian PK model

The first model is a simple 1-compartment PK IV model with proportional error. This is for the Gaussian versus gamma random effects:

model_lognormal = @model begin
    @param begin
        θCL ∈ RealDomain(; lower = 0)
        θVc ∈ RealDomain(; lower = 0)

        ωCL ∈ RealDomain(; lower = 0)
        ωVc ∈ RealDomain(; lower = 0)

        σ ∈ RealDomain(; lower = 0)
    end
    @random begin
        _CL ~ LogNormal(log(θCL), ωCL)
        _Vc ~ LogNormal(log(θVc), ωVc)
    end
    # This is equivalent to defining
    # CL = θCL*exp(ηCL)
    # with
    # ηCL = Normal(0, ωCL)
    @pre begin
        CL = _CL
        Vc = _Vc
    end
    @dynamics Central1
    @derived begin
        μ := @. Central / Vc
        dv ~ @. ProportionalNormal(μ, σ)
    end
end
PumasModel
  Parameters: θCL, θVc, ωCL, ωVc, σ
  Random effects: _CL, _Vc
  Covariates:
  Dynamical system variables: Central
  Dynamical system type: Closed form
  Derived: dv
  Observed: dv
model_gamma = @model begin
    @param begin
        θCL ∈ RealDomain(; lower = 0)
        θVc ∈ RealDomain(; lower = 0)

        ωCL ∈ RealDomain(; lower = 0)
        ωVc ∈ RealDomain(; lower = 0)

        σ ∈ RealDomain(; lower = 0)
    end
    @random begin
        _CL ~ Gamma(1 / ωCL^2, θCL * ωCL^2)
        _Vc ~ Gamma(1 / ωVc^2, θVc * ωVc^2)
    end
    @pre begin
        CL = _CL
        Vc = _Vc
    end
    @dynamics begin
        Central' = -CL / Vc * Central
    end
    @derived begin
        μ := @. Central / Vc
        dv ~ @. ProportionalNormal(μ, σ)
    end
end
PumasModel
  Parameters: θCL, θVc, ωCL, ωVc, σ
  Random effects: _CL, _Vc
  Covariates:
  Dynamical system variables: Central
  Dynamical system type: Matrix exponential
  Derived: dv
  Observed: dv

We also need some initial values for the fitting:

iparams_pk = (; θCL = 1.0, θVc = 5.0, ωCL = 0.1, ωVc = 0.1, σ = 0.2)
(θCL = 1.0, θVc = 5.0, ωCL = 0.1, ωVc = 0.1, σ = 0.2)

We proceed by fitting both models:

fit_lognormal = fit(model_lognormal, pop, iparams_pk, FOCE())
[ Info: Checking the initial parameter values.
[ Info: The initial negative log-likelihood and its gradient are finite. Check passed.
Iter     Function value   Gradient norm 
     0     5.770212e+03     7.912060e+03
 * time: 0.047376155853271484
     1     9.433464e+02     6.079483e+02
 * time: 6.3358049392700195
     2     8.189627e+02     4.423725e+02
 * time: 6.346259117126465
     3     5.917683e+02     1.819248e+02
 * time: 6.354077100753784
     4     5.421783e+02     1.121313e+02
 * time: 6.359986066818237
     5     5.255651e+02     7.407230e+01
 * time: 6.365612983703613
     6     5.208427e+02     8.699271e+01
 * time: 6.3705549240112305
     7     5.174883e+02     8.974584e+01
 * time: 6.375195026397705
     8     5.138523e+02     7.328235e+01
 * time: 6.380028963088989
     9     5.109883e+02     4.155805e+01
 * time: 6.384787082672119
    10     5.094359e+02     3.170517e+01
 * time: 6.3891520500183105
    11     5.086172e+02     3.327331e+01
 * time: 6.3931989669799805
    12     5.080941e+02     2.942077e+01
 * time: 6.397565126419067
    13     5.074009e+02     2.839941e+01
 * time: 6.401540040969849
    14     5.059302e+02     3.330093e+01
 * time: 6.40529203414917
    15     5.036399e+02     3.172884e+01
 * time: 6.40952205657959
    16     5.017004e+02     3.160020e+01
 * time: 6.413973093032837
    17     5.008553e+02     2.599524e+01
 * time: 6.418406009674072
    18     5.005913e+02     2.139314e+01
 * time: 6.422608137130737
    19     5.003573e+02     2.134778e+01
 * time: 6.426455974578857
    20     4.997249e+02     2.069868e+01
 * time: 6.430653095245361
    21     4.984453e+02     1.859010e+01
 * time: 6.4348649978637695
    22     4.959584e+02     2.156209e+01
 * time: 6.43953800201416
    23     4.923347e+02     3.030833e+01
 * time: 6.444739103317261
    24     4.906916e+02     1.652278e+01
 * time: 6.449924945831299
    25     4.902955e+02     6.360800e+00
 * time: 6.454648017883301
    26     4.902248e+02     1.151521e+00
 * time: 6.463510036468506
    27     4.902191e+02     1.174457e+00
 * time: 6.467755079269409
    28     4.902189e+02     1.171564e+00
 * time: 6.471411943435669
    29     4.902184e+02     1.164556e+00
 * time: 6.474977970123291
    30     4.902169e+02     1.143206e+00
 * time: 6.478655099868774
    31     4.902134e+02     1.089000e+00
 * time: 6.483134984970093
    32     4.902052e+02     1.135407e+00
 * time: 6.487934112548828
    33     4.901885e+02     1.547068e+00
 * time: 6.492768049240112
    34     4.901641e+02     1.507949e+00
 * time: 6.497530937194824
    35     4.901455e+02     8.269239e-01
 * time: 6.502656936645508
    36     4.901410e+02     2.300152e-01
 * time: 6.507610082626343
    37     4.901406e+02     3.097984e-02
 * time: 6.512316942214966
    38     4.901405e+02     5.453246e-03
 * time: 6.516749143600464
    39     4.901405e+02     1.203230e-03
 * time: 6.520477056503296
    40     4.901405e+02     6.315365e-05
 * time: 6.523711919784546
FittedPumasModel

Dynamical system type:                 Closed form

Number of subjects:                             32

Observation records:         Active        Missing
    dv:                         256              0
    Total:                      256              0

Number of parameters:      Constant      Optimized
                                  0              5

Likelihood approximation:                     FOCE
Likelihood optimizer:                         BFGS

Termination Reason:                   GradientNorm
Log-likelihood value:                   -490.14053

---------------
      Estimate
---------------
θCL    0.16025
θVc   10.262
ωCL    0.23504
ωVc    0.10449
σ      0.3582
---------------
fit_gamma = fit(model_gamma, pop, iparams_pk, FOCE())
[ Info: Checking the initial parameter values.
[ Info: The initial negative log-likelihood and its gradient are finite. Check passed.
Iter     Function value   Gradient norm 
     0     4.160707e+03     5.960578e+03
 * time: 4.887580871582031e-5
     1     9.056373e+02     5.541920e+02
 * time: 3.6705398559570312
     2     7.787963e+02     3.988904e+02
 * time: 3.682436943054199
     3     5.915054e+02     1.495777e+02
 * time: 3.691776990890503
     4     5.533120e+02     8.826583e+01
 * time: 3.7005598545074463
     5     5.389239e+02     9.144086e+01
 * time: 3.709040880203247
     6     5.323499e+02     1.000933e+02
 * time: 3.7175159454345703
     7     5.270252e+02     8.423844e+01
 * time: 3.7256829738616943
     8     5.233813e+02     5.194402e+01
 * time: 3.7335400581359863
     9     5.213366e+02     3.461331e+01
 * time: 3.741499900817871
    10     5.200972e+02     3.888113e+01
 * time: 3.7493739128112793
    11     5.191933e+02     3.556605e+01
 * time: 3.7571399211883545
    12     5.181335e+02     3.624436e+01
 * time: 3.7650158405303955
    13     5.161626e+02     4.322775e+01
 * time: 3.7729949951171875
    14     5.133202e+02     3.722515e+01
 * time: 3.7807278633117676
    15     5.107758e+02     3.401586e+01
 * time: 3.788648843765259
    16     5.095157e+02     2.854997e+01
 * time: 3.796915054321289
    17     5.090165e+02     2.644560e+01
 * time: 3.804866075515747
    18     5.085184e+02     2.744429e+01
 * time: 3.812628984451294
    19     5.074309e+02     2.793918e+01
 * time: 3.820399045944214
    20     5.053757e+02     2.616169e+01
 * time: 3.828248977661133
    21     5.018507e+02     2.257667e+01
 * time: 3.8364288806915283
    22     4.942495e+02     3.832878e+01
 * time: 3.8456239700317383
    23     4.916504e+02     3.167965e+01
 * time: 3.8650810718536377
    24     4.902302e+02     1.020055e+01
 * time: 3.8760569095611572
    25     4.898632e+02     6.068208e+00
 * time: 3.8863558769226074
    26     4.898142e+02     1.688453e+00
 * time: 3.896475076675415
    27     4.897942e+02     6.439288e-01
 * time: 3.9066810607910156
    28     4.897941e+02     6.464271e-01
 * time: 3.9152488708496094
    29     4.897939e+02     6.470322e-01
 * time: 4.529684066772461
    30     4.897933e+02     6.434520e-01
 * time: 4.540042877197266
    31     4.897921e+02     7.468240e-01
 * time: 4.54696798324585
    32     4.897892e+02     1.271556e+00
 * time: 4.554199934005737
    33     4.897836e+02     1.739405e+00
 * time: 4.5614330768585205
    34     4.897761e+02     1.644118e+00
 * time: 4.568567991256714
    35     4.897711e+02     7.983427e-01
 * time: 4.57574200630188
    36     4.897701e+02     1.702950e-01
 * time: 4.582598924636841
    37     4.897700e+02     1.208261e-02
 * time: 4.5889599323272705
    38     4.897700e+02     2.536842e-03
 * time: 4.594806909561157
    39     4.897700e+02     3.067685e-04
 * time: 4.599622964859009
FittedPumasModel

Dynamical system type:          Matrix exponential

Number of subjects:                             32

Observation records:         Active        Missing
    dv:                         256              0
    Total:                      256              0

Number of parameters:      Constant      Optimized
                                  0              5

Likelihood approximation:                     FOCE
Likelihood optimizer:                         BFGS

Termination Reason:                   GradientNorm
Log-likelihood value:                   -489.77002

---------------
      Estimate
---------------
θCL    0.16466
θVc   10.329
ωCL    0.23348
ωVc    0.10661
σ      0.35767
---------------

Finally, let’s compare the estimates:

compare_estimates(; lognormal = fit_lognormal, gamma = fit_gamma)
5×3 DataFrame
Row parameter lognormal gamma
String Float64? Float64?
1 θCL 0.160253 0.164658
2 θVc 10.2617 10.3288
3 ωCL 0.23504 0.233485
4 ωVc 0.104494 0.106612
5 σ 0.358204 0.357666

As mention above, the mean of a log-normal is \(\exp \left\{ \mu + \frac{\sigma^2}{2} \right\}\).

So let’s compare that with the gamma typical values:

DataFrame(;
    parameter = ["θCL", "θVc"],
    θLogNormal = [coef(fit_lognormal).θCL, coef(fit_lognormal).θVc],
    ELogNormal = [
        exp(log(coef(fit_lognormal).θCL) + coef(fit_lognormal).ωCL^2 / 2),
        exp(log(coef(fit_lognormal).θVc) + coef(fit_lognormal).ωVc^2 / 2),
    ],
    θGamma = [coef(fit_gamma).θCL, coef(fit_gamma).θVc],
)
2×4 DataFrame
Row parameter θLogNormal ELogNormal θGamma
String Float64 Float64 Float64
1 θCL 0.160253 0.164741 0.164658
2 θVc 10.2617 10.3178 10.3288

As you can see the Gaussian model has a slight bias in the estimation of both θCL and θVc.

Let’s also plot the two probability density functions (PDF) for θCL:

plotdataPK = @chain DataFrame(; x = range(0, 0.5; length = 1_000)) begin
    @rtransform begin
        :LogNormal =
            pdf(LogNormal(log(coef(fit_lognormal).θCL), coef(fit_lognormal).ωCL), :x)
        :Gamma = pdf(LogNormal(log(coef(fit_gamma).θCL), coef(fit_gamma).ωCL), :x)
    end
end
first(plotdataPK, 5)
5×3 DataFrame
Row x LogNormal Gamma
Float64 Float64 Float64
1 0.0 0.0 0.0
2 0.000500501 5.20157e-128 5.26158e-131
3 0.001001 9.15989e-99 3.24782e-101
4 0.0015015 2.08275e-83 1.45738e-85
5 0.002002 2.69537e-73 2.98228e-75
data(stack(plotdataPK, [:LogNormal, :Gamma])) *
mapping(:x, :value; color = :variable) *
visual(Lines) |> draw

4.2 Bioavaliability Parallel Absorption Simulation

This is a parallel absorption model with bioavaliabity in both the “fast” as the “slow” depots.

First, the traditional approach with a logistic transformation of a Gaussian random variable. This makes the individual relative bioavailibility logit-normally distributed.

model_logitnormal = @model begin
    @param begin
        θkaFast ∈ RealDomain(; lower = 0)
        θkaSlow ∈ RealDomain(; lower = 0)
        θCL ∈ RealDomain(; lower = 0)
        θVc ∈ RealDomain(; lower = 0)
        θbioav ∈ RealDomain(; lower = 0.0, upper = 1.0)

        ωCL ∈ RealDomain(; lower = 0)
        ωVc ∈ RealDomain(; lower = 0)
        # I call this one ξ to distinguish it from ω since the interpretation is NOT a relative error (coefficient of variation)
        ξbioav ∈ RealDomain(; lower = 0, init = 0.1)

        σ ∈ RealDomain(; lower = 0)
    end
    @random begin
        _CL ~ Gamma(1 / ωCL^2, θCL * ωCL^2)
        _Vc ~ Gamma(1 / ωVc^2, θVc * ωVc^2)
        # define the latent Gaussian random effect. Notice the logit transform
        ηbioavLogit ~ Normal(logit(θbioav), ξbioav)
    end
    @pre begin
        kaFast = θkaFast
        kaSlow = θkaSlow
        CL = _CL
        Vc = _Vc
    end
    @dosecontrol begin
        # _bioav is LogitNormal distributed
        _bioav := logistic(ηbioavLogit)
        bioav = (; DepotFast = _bioav, DepotSlow = 1 - _bioav)
    end
    @dynamics begin
        DepotFast' = -kaFast * DepotFast
        DepotSlow' = -kaSlow * DepotSlow
        Central' = kaFast * DepotFast + kaSlow * DepotSlow - CL / Vc * Central
    end
    @derived begin
        μ := @. Central / Vc
        dv ~ @. ProportionalNormal(μ, σ)
    end
end
PumasModel
  Parameters: θkaFast, θkaSlow, θCL, θVc, θbioav, ωCL, ωVc, ξbioav, σ
  Random effects: _CL, _Vc, ηbioavLogit
  Covariates:
  Dynamical system variables: DepotFast, DepotSlow, Central
  Dynamical system type: Matrix exponential
  Derived: dv
  Observed: dv

Now the same model but with the non-Gaussian random-effects using a beta distribution instead of the logit parameterization of the Gaussian distribution:

model_beta = @model begin
    @param begin
        θkaFast ∈ RealDomain(; lower = 0)
        θkaSlow ∈ RealDomain(; lower = 0)
        θCL ∈ RealDomain(; lower = 0)
        θVc ∈ RealDomain(; lower = 0)
        θbioav ∈ RealDomain(; lower = 0.0, upper = 1.0)

        ωCL ∈ RealDomain(; lower = 0)
        ωVc ∈ RealDomain(; lower = 0)
        # We call this one n since the interpretation is like the length of a Binomial distribution
        nbioav ∈ RealDomain(; lower = 0, init = 10)

        σ ∈ RealDomain(; lower = 0)
    end
    @random begin
        ηCL ~ Gamma(1 / ωCL^2, θCL * ωCL^2)
        ηVc ~ Gamma(1 / ωVc^2, θVc * ωVc^2)
        # The makes E(_bioav) = θbioav
        # See https://en.wikipedia.org/wiki/Beta_distribution
        ηbioav ~ Beta(θbioav * nbioav, (1 - θbioav) * nbioav)
    end
    @pre begin
        kaFast = θkaFast
        kaSlow = θkaSlow
        CL = ηCL
        Vc = ηVc
    end
    @dosecontrol begin
        bioav = (; DepotFast = ηbioav, DepotSlow = 1 - ηbioav)
    end
    @dynamics begin
        DepotFast' = -kaFast * DepotFast
        DepotSlow' = -kaSlow * DepotSlow
        Central' = kaFast * DepotFast + kaSlow * DepotSlow - CL / Vc * Central
    end
    @derived begin
        μ := @. Central / Vc
        dv ~ @. ProportionalNormal(μ, σ)
    end
end
PumasModel
  Parameters: θkaFast, θkaSlow, θCL, θVc, θbioav, ωCL, ωVc, nbioav, σ
  Random effects: ηCL, ηVc, ηbioav
  Covariates:
  Dynamical system variables: DepotFast, DepotSlow, Central
  Dynamical system type: Matrix exponential
  Derived: dv
  Observed: dv

We have two types of random effects here.

First, as you are already familiar from the previous example, the clearance (CL), volume of concentration (Vc), and absorption rate (ka) have typical values (i.e. fixed effects) and between-subject variability (i.e. random effects) modelled as a gamma distribution.

Second, bioavailability Bioav is modelled as a beta distribution. Generally the beta distribution is parametrized as:

\[\text{Beta}(\alpha, \beta)\]

where both parameters \(\alpha\) and \(\beta\) are shape parameters.

One nice thing about the beta distribution is that it only takes values between and including 0 and 1, i.e. \([0, 1]\). This makes it the perfect candidate to model bioavailability parameters which are generally bounded in that interval. So, we don’t need to do a logistic transformation.

Another nice thing about the beta distribution is that we can use the alternative \((\mu, n)\)-parametrization with with \(\mu\) serving as a mean-value parameter:

\[\text{Beta}(\mu, n)\]

where in the original beta parametrization:

  • \(\alpha = \mu n\)
  • \(\beta = (1 - \mu) n\)

Hence, our mean is:

\[\operatorname{E}[F] = \mu = \theta_F\]

which, again, does not depend on any other parameters. The variance is

\[\operatorname{Var}(F) = \frac{\mu(1 - \mu)}{n}\]

so similar to the mean of Bernoulli trials.

Now let’s generate some data for the simulation:

dr = DosageRegimen(
    DosageRegimen(100; cmt = :DepotFast),
    DosageRegimen(100; cmt = :DepotSlow),
)
2×9 DataFrame
Row time cmt amt evid ii addl rate duration ss
Float64 Symbol Float64 Int8 Float64 Int64 Float64 Float64 Int8
1 0.0 DepotFast 100.0 1 0.0 0 0.0 0.0 0
2 0.0 DepotSlow 100.0 1 0.0 0 0.0 0.0 0
simtimes = [0.5, 1.0, 2.0, 4.0, 8.0, 24.0]
6-element Vector{Float64}:
  0.5
  1.0
  2.0
  4.0
  8.0
 24.0
trueparam = (;
    θkaFast = 0.9,
    θkaSlow = 0.2,
    θCL = 1.1,
    θVc = 10.0,
    θbioav = 0.7,
    ωCL = 0.1,
    ωVc = 0.1,
    nbioav = 40,
    σ = 0.1,
)
(θkaFast = 0.9, θkaSlow = 0.2, θCL = 1.1, θVc = 10.0, θbioav = 0.7, ωCL = 0.1, ωVc = 0.1, nbioav = 40, σ = 0.1)

For simplicity, we just add 20% to the true values for initial values:

initparamBeta = map(t -> 1.2 * t, trueparam)
(θkaFast = 1.08, θkaSlow = 0.24, θCL = 1.32, θVc = 12.0, θbioav = 0.84, ωCL = 0.12, ωVc = 0.12, nbioav = 48.0, σ = 0.12)

The initial values for the LogitNormal need to have ξbioav defined instead of nbioav:

initparamLogitNormal =
    (Base.structdiff(initparamBeta, NamedTuple{(:nbioav,)})..., ξbioav = 0.1)
(θkaFast = 1.08, θkaSlow = 0.24, θCL = 1.32, θVc = 12.0, θbioav = 0.84, ωCL = 0.12, ωVc = 0.12, σ = 0.12, ξbioav = 0.1)

Setup empty Subjects with the dose information:

skeletonpop = map(i -> Subject(; id = i, events = dr), 1:40)
Population
  Subjects: 40
  Observations: 

Next, we simulate the data (while setting the seed for reprocibility):

Random.seed!(128)
simpop = Subject.(simobs(model_beta, skeletonpop, trueparam; obstimes = simtimes))

Finally let’s fit both models:

fit_logitnormal = fit(model_logitnormal, simpop, initparamLogitNormal, FOCE())
[ Info: Checking the initial parameter values.
[ Info: The initial negative log-likelihood and its gradient are finite. Check passed.
Iter     Function value   Gradient norm 
     0     2.512964e+02     3.650794e+02
 * time: 2.9087066650390625e-5
     1     2.097177e+02     2.581791e+02
 * time: 13.010613918304443
     2     1.912461e+02     8.737199e+01
 * time: 13.091485023498535
     3     1.887358e+02     6.798862e+01
 * time: 13.166548013687134
     4     1.843518e+02     1.520669e+01
 * time: 13.2478609085083
     5     1.833150e+02     1.305346e+01
 * time: 13.296052932739258
     6     1.825488e+02     1.030554e+01
 * time: 13.343508958816528
     7     1.811422e+02     8.012892e+00
 * time: 13.393826007843018
     8     1.810538e+02     3.075453e+00
 * time: 13.44311809539795
     9     1.810370e+02     2.322234e+00
 * time: 13.49193787574768
    10     1.810214e+02     1.641742e+00
 * time: 13.54140305519104
    11     1.808623e+02     9.955746e+00
 * time: 13.5954430103302
    12     1.806361e+02     2.328715e+01
 * time: 13.64899492263794
    13     1.803677e+02     2.160846e+01
 * time: 13.704339027404785
    14     1.796608e+02     1.029944e+01
 * time: 13.758073091506958
    15     1.796130e+02     2.787719e+01
 * time: 13.813677072525024
    16     1.793909e+02     1.479965e+01
 * time: 13.870265007019043
    17     1.792909e+02     1.128584e+01
 * time: 13.928252935409546
    18     1.792571e+02     7.571530e+00
 * time: 13.983803987503052
    19     1.791694e+02     7.130415e+00
 * time: 14.041668891906738
    20     1.790947e+02     1.587148e+01
 * time: 14.097861051559448
    21     1.789919e+02     9.984290e+00
 * time: 14.154147863388062
    22     1.788772e+02     9.117968e+00
 * time: 14.211390972137451
    23     1.788396e+02     7.991913e+00
 * time: 14.265094995498657
    24     1.788017e+02     2.044978e+00
 * time: 14.316598892211914
    25     1.787978e+02     1.967777e+00
 * time: 14.365660905838013
    26     1.787814e+02     1.676898e+00
 * time: 14.419565916061401
    27     1.787796e+02     8.236497e-01
 * time: 14.472259044647217
    28     1.787780e+02     3.610259e-01
 * time: 14.907938957214355
    29     1.787762e+02     4.980138e-01
 * time: 14.960705041885376
    30     1.787739e+02     1.025037e+00
 * time: 15.01301097869873
    31     1.787721e+02     1.497724e+00
 * time: 15.064081907272339
    32     1.787704e+02     1.294175e+00
 * time: 15.115048885345459
    33     1.787674e+02     1.360458e+00
 * time: 15.164702892303467
    34     1.787612e+02     1.811390e+00
 * time: 15.212805032730103
    35     1.787464e+02     2.634680e+00
 * time: 15.262541055679321
    36     1.787141e+02     3.363820e+00
 * time: 15.311946868896484
    37     1.786632e+02     3.060364e+00
 * time: 15.361937046051025
    38     1.786277e+02     1.344290e+00
 * time: 15.413615942001343
    39     1.786192e+02     2.524868e-01
 * time: 15.46520209312439
    40     1.786166e+02     6.759174e-01
 * time: 15.517277956008911
    41     1.786136e+02     8.208435e-01
 * time: 15.563062906265259
    42     1.786106e+02     4.973292e-01
 * time: 15.607578992843628
    43     1.786088e+02     3.537716e-01
 * time: 15.65540599822998
    44     1.786083e+02     1.527595e-01
 * time: 15.70212197303772
    45     1.786080e+02     1.812419e-01
 * time: 15.749612092971802
    46     1.786078e+02     1.660844e-01
 * time: 15.795186996459961
    47     1.786077e+02     5.979074e-02
 * time: 15.841912031173706
    48     1.786076e+02     4.897633e-02
 * time: 15.888545036315918
    49     1.786076e+02     3.830216e-02
 * time: 15.933785915374756
    50     1.786076e+02     2.623948e-02
 * time: 15.98161005973816
    51     1.786076e+02     6.769983e-03
 * time: 16.02837109565735
    52     1.786076e+02     5.050303e-03
 * time: 16.075736045837402
    53     1.786076e+02     7.621588e-03
 * time: 16.121201992034912
    54     1.786076e+02     4.582334e-03
 * time: 16.167496919631958
    55     1.786076e+02     6.267388e-04
 * time: 16.213690996170044
FittedPumasModel

Dynamical system type:          Matrix exponential

Number of subjects:                             40

Observation records:         Active        Missing
    dv:                         240              0
    Total:                      240              0

Number of parameters:      Constant      Optimized
                                  0              9

Likelihood approximation:                     FOCE
Likelihood optimizer:                         BFGS

Termination Reason:                   GradientNorm
Log-likelihood value:                   -178.60759

---------------------
          Estimate
---------------------
θkaFast   0.91098
θkaSlow   0.13112
θCL       1.0854
θVc       7.1008
θbioav    0.4802
ωCL       0.088113
ωVc       0.12133
ξbioav    0.00017116
σ         0.10545
---------------------
fit_beta = fit(model_beta, simpop, initparamBeta, FOCE())
[ Info: Checking the initial parameter values.
[ Info: The initial negative log-likelihood and its gradient are finite. Check passed.
Iter     Function value   Gradient norm 
     0     2.523111e+02     3.644346e+02
 * time: 2.6941299438476562e-5
     1     2.098955e+02     2.554795e+02
 * time: 2.530000925064087
     2     1.914622e+02     7.988189e+01
 * time: 2.614223003387451
     3     1.877135e+02     8.496056e+01
 * time: 2.6643588542938232
     4     1.851001e+02     4.097104e+01
 * time: 2.7422869205474854
     5     1.838808e+02     1.323193e+01
 * time: 2.7984750270843506
     6     1.831532e+02     1.415278e+01
 * time: 2.8543128967285156
     7     1.816636e+02     1.862107e+01
 * time: 2.911907911300659
     8     1.815171e+02     3.953485e+00
 * time: 3.3297810554504395
     9     1.814961e+02     1.940229e+00
 * time: 3.380063056945801
    10     1.814647e+02     4.319730e+00
 * time: 3.4285800457000732
    11     1.812769e+02     1.765417e+01
 * time: 3.4779770374298096
    12     1.810778e+02     2.806019e+01
 * time: 3.528217077255249
    13     1.807917e+02     2.968381e+01
 * time: 3.578618049621582
    14     1.802819e+02     1.547090e+01
 * time: 3.62937593460083
    15     1.801090e+02     1.366244e+00
 * time: 3.6788289546966553
    16     1.800821e+02     6.329956e+00
 * time: 3.7514379024505615
    17     1.800545e+02     1.125074e+00
 * time: 3.800477981567383
    18     1.800446e+02     1.561431e+00
 * time: 3.8490400314331055
    19     1.800177e+02     2.451310e+00
 * time: 3.899388074874878
    20     1.799967e+02     2.539586e+00
 * time: 3.949277877807617
    21     1.799753e+02     1.523139e+00
 * time: 4.0012829303741455
    22     1.799469e+02     2.173685e+00
 * time: 4.055161952972412
    23     1.798979e+02     8.681997e+00
 * time: 4.109477996826172
    24     1.798362e+02     7.272561e+00
 * time: 4.164207935333252
    25     1.797469e+02     1.490516e+01
 * time: 4.2184529304504395
    26     1.796654e+02     8.718280e+00
 * time: 4.292547941207886
    27     1.795920e+02     9.410186e+00
 * time: 4.347012042999268
    28     1.795394e+02     5.212003e+00
 * time: 4.400117874145508
    29     1.794630e+02     4.927958e+00
 * time: 4.453466892242432
    30     1.793764e+02     5.159988e+00
 * time: 4.507130861282349
    31     1.793446e+02     7.307439e+00
 * time: 4.66205096244812
    32     1.793227e+02     1.964089e+00
 * time: 4.733603000640869
    33     1.793181e+02     1.829483e+00
 * time: 4.782405853271484
    34     1.793118e+02     1.092217e+00
 * time: 4.832676887512207
    35     1.793067e+02     1.256868e+00
 * time: 4.879111051559448
    36     1.792666e+02     4.665942e+00
 * time: 4.925428867340088
    37     1.792113e+02     7.305936e+00
 * time: 4.971827030181885
    38     1.791121e+02     8.705330e+00
 * time: 5.025462865829468
    39     1.789925e+02     7.398610e+00
 * time: 5.077718019485474
    40     1.788294e+02     5.245360e+00
 * time: 5.129773855209351
    41     1.787071e+02     9.501500e+00
 * time: 5.182111024856567
    42     1.786762e+02     3.851030e+00
 * time: 5.236105918884277
    43     1.786560e+02     1.039973e+00
 * time: 5.291898965835571
    44     1.786386e+02     2.307784e+00
 * time: 5.344501972198486
    45     1.786286e+02     1.623904e+00
 * time: 5.398983955383301
    46     1.786169e+02     7.950987e-01
 * time: 5.454668045043945
    47     1.786129e+02     1.210835e+00
 * time: 5.510704040527344
    48     1.786101e+02     6.972927e-01
 * time: 5.563253879547119
    49     1.786088e+02     1.656571e-01
 * time: 5.614686965942383
    50     1.786082e+02     1.785747e-01
 * time: 5.669759035110474
    51     1.786080e+02     2.576510e-01
 * time: 5.724600076675415
    52     1.786078e+02     1.991902e-01
 * time: 5.8130738735198975
    53     1.786077e+02     7.973891e-02
 * time: 5.865697860717773
    54     1.786076e+02     1.247772e-02
 * time: 5.914049863815308
    55     1.786076e+02     5.112672e-02
 * time: 5.964370012283325
    56     1.786076e+02     5.327180e-02
 * time: 6.014150857925415
    57     1.786076e+02     3.122404e-02
 * time: 6.068072080612183
    58     1.786076e+02     4.869464e-03
 * time: 6.121379852294922
    59     1.786076e+02     4.482959e-03
 * time: 6.189954996109009
    60     1.786076e+02     2.120120e-03
 * time: 6.2413389682769775
    61     1.786076e+02     2.553593e-03
 * time: 6.286244869232178
    62     1.786076e+02     3.407646e-03
 * time: 6.363097906112671
    63     1.786076e+02     7.386995e-04
 * time: 6.412564039230347
FittedPumasModel

Dynamical system type:          Matrix exponential

Number of subjects:                             40

Observation records:         Active        Missing
    dv:                         240              0
    Total:                      240              0

Number of parameters:      Constant      Optimized
                                  0              9

Likelihood approximation:                     FOCE
Likelihood optimizer:                         BFGS

Termination Reason:                   GradientNorm
Log-likelihood value:                   -178.60759

-------------------
          Estimate
-------------------
θkaFast   0.91098
θkaSlow   0.13112
θCL       1.0854
θVc       7.1008
θbioav    0.4802
ωCL       0.088113
ωVc       0.12133
nbioav    2.8889e7
σ         0.10545
-------------------

As before, let’s compare the estimates:

compare_estimates(; logitnormal = fit_logitnormal, beta = fit_beta)
10×3 DataFrame
Row parameter logitnormal beta
String Float64? Float64?
1 θkaFast 0.910979 0.910981
2 θkaSlow 0.131118 0.131118
3 θCL 1.0854 1.0854
4 θVc 7.10083 7.10079
5 θbioav 0.480204 0.4802
6 ωCL 0.0881133 0.0881134
7 ωVc 0.121334 0.121334
8 σ 0.105448 0.105449
9 ξbioav 0.000171162 missing
10 nbioav missing 2.88887e7

Again, we’ll both PDFs from the estimated values:

plotdatabioav = @chain DataFrame(; x = range(0, 1; length = 1_000)) begin
    @rtransform begin
        :logitnormal =
            1 / coef(fit_logitnormal).ξbioav / √(2π) / (:x * (1 - :x)) * exp(
                -(logit(:x) - logit(coef(fit_logitnormal).θbioav))^2 /
                (2 * coef(fit_logitnormal).ξbioav^2),
            )
        :beta = pdf(
            Beta(
                coef(fit_beta).θbioav * coef(fit_beta).nbioav,
                (1 - coef(fit_beta).θbioav) * coef(fit_beta).nbioav,
            ),
            :x,
        )
    end
end
first(plotdatabioav, 5)
5×3 DataFrame
Row x logitnormal beta
Float64 Float64 Float64
1 0.0 NaN 0.0
2 0.001001 0.0 0.0
3 0.002002 0.0 0.0
4 0.003003 0.0 0.0
5 0.004004 0.0 0.0
plt_pdf_bioav =
    data(stack(plotdatabioav, [:logitnormal, :beta])) *
    mapping(:x, :value; color = :variable) *
    visual(Lines);
draw(plt_pdf_bioav; axis = (; xticks = 0.1:0.1:1.0))

For this dataset, the two distributions differ significantly with the Beta model producing a distribution much closer to the truth but for other realizations of the simulated data they are closer to each other.