Mixed effects models and the marginal likelihood

What are mixed effects models?

• Population parameters \(\theta\)
  • Model parameters modelled as deterministic quantities.
  • A subset of these are commonly referred to as fixed effects.
• Random effects \(\eta\)
  • Subject-specific parameters.
  • Modelled as random variables.
• Covariates \(x\)
• Responses \(y\)

What are mixed effects models?

Likelihood

Conditional likelihood

Probability of the response \(y\) according to the model given specific values of \(\theta\), \(\eta\) and \(x\).

\[ p_c(y \mid \theta, \eta, x) \]

Fit model by simply finding the values of \(\theta\) and \(\eta\) that maximizes the conditional probability?

This can horribly overfit the data, too many parameters!

Marginal likelihood

Integrates out the effect of the random effects

\[ p_m (y \mid \theta, x) = \int \overbrace{p_c (y \mid \theta, \eta, x) \cdot p_{\text{prior}} (\eta \mid \theta)}^{\text{joint probability}} \, d\eta \]

Average conditional probability weighted by a prior.

Comparing models

Which model has the largest area under the curve?

Comparing models

A good model should have:

1. A high peak: high joint probability with the data, and
2. A wide base: low sensitivity to the value of the random effect.

Comparing models