A Course on Bioequivalence: Unit 13 - Reference Scaling Part II

This is Unit 13 of a complete bioequivalence analysis course using Pumas. There are 15 units in the course in total.

1 Unit overview

In the previous unit we saw the FDA RSABE (reference scaled average bioequivalence) approach for highly variable drugs (HVD). In this unit we see how this approach is used for NTID (narrow therapeutic index drugs). The general topic of NTID was introduced in Unit 11.

The FDA approach to NTID is to employ reference scaling for potential effective narrowing of the limits. There are also other checks as part of the approach involving the within subject variability of the test and reference products.

We use the following packages.

using Pumas # available with Pumas products - also reexports Distributions
using Bioequivalence  # available with Pumas products
using BioequivalencePower  # available with Pumas products
using PharmaDatasets # available with Pumas products
using SummaryTables

2 Different requirements and criteria in comparison to HVD.

For NTID we must use a fully replicate design. This is so that the within subject variability for both R and T be estimated properly. This means a design of the type: RTRT|TRTR, RTTR|TRRT, or RRTT|TTRR.

Three criteria

To conclude bioequivalence we must pass three criteria:

1. Pass standard ABE (80-125 bounds). This condition is present in case the effective bounds via reference scaling are expanded beyond the 80-125 bounds. Ensuring that we pass standard ABE ensures that if such an expansion happens, then we essentially limit the expansion to 80-125.
2. Pass (reject) the key reference scaling hypothesis (Howe's Approx RSABE Stat (95%)) in the same way as it was for HVD, only with a different reference scaling constant (\(\theta\)). As seen in the previous unit, with this condition we consider this hypothesis test for the “linearized criterion”:

\[ {\cal T}_{\text{RSABE}} = \begin{cases} H_{0}:& (\mu_T - \mu_R)^2 - \theta \sigma^2_{WR} \ge 0, \\[5pt] H_{1}:& (\mu_T - \mu_R)^2 - \theta \sigma^2_{WR} < 0.\\ \end{cases} \]

In contrast to the HVD case where Δ = 1.25 and σ_W₀ = 0.25, in this case we have different parameters which yield a different θ:

Δ = 1 / 0.9 #1.11111
σ_W₀ = 0.1
θ = (log(Δ) / σ_W₀)^2

1.1100838259683068

3. Pass the ratio of variability. This is an additional criterion for safeguarding that the variability of the test product does not exceedingly surpass the variability of the reference product. It is a threshold based approach related to the Variability Ratio (%) and implemented via the Variability Ratio Quantile (95%). As a reference for this criteria see “Step 4” in FDA (2012). It does not appear in the HVD (RSABE) case or with other approaches for NTID as presented in Unit 11. The criteria requires the Variability Ratio Quantile (95%) to be not more than 2.5. We explain more about this criteria in the next section after we see an example.

An example

Consider this example dataset. As you can see it is a fully replicate RTTR|TRRT design and is hence suitable for RSABE for NTID. We use the FDA_NarrowTherapeuticIndex as a second positional argument to pumas_be:

data = dataset(joinpath("bioequivalence", "RTTR_TRRT", "PJ2017_4_3"))
be_result = pumas_be(data, FDA_NarrowTherapeuticIndex)

Observation Counts
Sequence ╲ Period	1	2	3	4
RTTR	8	8	8	8
TRRT	9	9	9	8

Paradigm: Replicated crossover that supports reference scaling
Model: Mixed model (unequal variance)
Criteria: FDA RSABE for NTI
Endpoint: AUC
Formulations: Reference(R), Test(T)
		Results(AUC)	Assessment	Criteria
R	Geometric Marginal Mean	7152
	Geometric Naive Mean	7131
T	Geometric Marginal Mean	7412
	Geometric Naive Mean	7454
	Geometric Mean T/R Ratio (%)	103.6
	Degrees of Freedom	15.24
	90% Confidence Interval (%)	[99.31, 108.2]	Pass	CI ⊆ [80, 125]
Variability	CVᵣ (%) | σ̂ᵣ	8.02 | 0.0801
	CVₜ (%) | σ̂ₜ	10.84 | 0.1081
	Variability Ratio (%)	135
ANOVA	Formulation (p-value)	0.1624
	Sequence (p-value)	0.3184
	Period (p-value)	0.665
Reference Scaling Params	Reference Scaling Constant	1.11
Reference Scaling Analysis	Geometric Mean T/R Ratio (%)	103.7
	Standard Error (Log Scale)	0.0253
	90% Confidence Interval (%)	[99.2, 108.4]
	Degrees of Freedom	15
	Howe's Approx RSABE Stat (95%)	0.0001105	Fail	≤ 0
	Variability Ratio Quantile (95%)	2.118	Pass	≤ 2.5

Since we do not have Pass in all three cases, we fail to show bioequivalence in this case.

In particular, while we pass the basic ABE (criterion 1 above) with [99.31, 108.2] and pass the ratio of variability (criterion 3 above) with 2.118, we do not reject \(H_0\) with \({\cal T}_{\text{RSABE}}\) since the Howe's Approx RSABE Stat (95%) is at 0.0001105 (and is greater than 0).

Note that in another (hypothetical circumstance) where we only consider the first 50 rows from the data we pass all three cases:

data2 = data[1:50, :]
be_result = pumas_be(data2, FDA_NarrowTherapeuticIndex)

Observation Counts
Sequence ╲ Period	1	2	3	4
RTTR	6	6	6	6
TRRT	7	7	6	6

Paradigm: Replicated crossover that supports reference scaling
Model: Mixed model (unequal variance)
Criteria: FDA RSABE for NTI
Endpoint: AUC
Formulations: Reference(R), Test(T)
		Results(AUC)	Assessment	Criteria
R	Geometric Marginal Mean	7012
	Geometric Naive Mean	7050
T	Geometric Marginal Mean	7251
	Geometric Naive Mean	7287
	Geometric Mean T/R Ratio (%)	103.4
	Degrees of Freedom	30.29
	90% Confidence Interval (%)	[98.92, 108.1]	Pass	CI ⊆ [80, 125]
Variability	CVᵣ (%) | σ̂ᵣ	9.24 | 0.0922
	CVₜ (%) | σ̂ₜ	9.72 | 0.097
	Variability Ratio (%)	105.2
ANOVA	Formulation (p-value)	0.2098
	Sequence (p-value)	0.5551
	Period (p-value)	0.154
Reference Scaling Params	Reference Scaling Constant	1.11
Reference Scaling Analysis	Geometric Mean T/R Ratio (%)	103.6
	Standard Error (Log Scale)	0.0265
	90% Confidence Interval (%)	[98.7, 108.6]
	Degrees of Freedom	10
	Howe's Approx RSABE Stat (95%)	-0.001253	Pass	≤ 0
	Variability Ratio Quantile (95%)	1.815	Pass	≤ 2.5

In another hypothetical case where we omit the first 20 rows we fail on the RSABE and the variability criteria. In particular the variability ratio quantile at 2.721 exceeds the threshold 2.5 and hence we have Fail for that criterion.

data3 = data[21:end, :]
be_result = pumas_be(data3, FDA_NarrowTherapeuticIndex)

Observation Counts
Sequence ╲ Period	1	2	3	4
RTTR	6	6	6	6
TRRT	6	6	6	5

Paradigm: Replicated crossover that supports reference scaling
Model: Mixed model (unequal variance)
Criteria: FDA RSABE for NTI
Endpoint: AUC
Formulations: Reference(R), Test(T)
		Results(AUC)	Assessment	Criteria
R	Geometric Marginal Mean	6913
	Geometric Naive Mean	6913
T	Geometric Marginal Mean	7197
	Geometric Naive Mean	7275
	Geometric Mean T/R Ratio (%)	104.1
	Degrees of Freedom	21.45
	90% Confidence Interval (%)	[99.17, 109.3]	Pass	CI ⊆ [80, 125]
Variability	CVᵣ (%) | σ̂ᵣ	7.68 | 0.0767
	CVₜ (%) | σ̂ₜ	11.82 | 0.1178
	Variability Ratio (%)	153.6
ANOVA	Formulation (p-value)	0.1681
	Sequence (p-value)	0.8862
	Period (p-value)	0.6426
Reference Scaling Params	Reference Scaling Constant	1.11
Reference Scaling Analysis	Geometric Mean T/R Ratio (%)	104.4
	Standard Error (Log Scale)	0.0267
	90% Confidence Interval (%)	[99.46, 109.7]
	Degrees of Freedom	10
	Howe's Approx RSABE Stat (95%)	0.002579	Fail	≤ 0
	Variability Ratio Quantile (95%)	2.721	Fail	≤ 2.5

3 More on the variability ratio criterion

Let us now understand the variability ratio criterion a bit further. The idea is to make sure that the within subject variability of the test product does excessively surpass that of the reference product.

The variability ratio

As a rough guide one may consider the Variability Ratio (%) as in the last output above. Naively one may suspect that this ratio, 153.6 in percentage, is simply CVₜ (%) divided by CVᵣ (%). However, this is not the case:

CVₜ = 11.82
CVᵣ = 7.68
ratio_of_cvs = CVₜ / CVᵣ
round(100 * ratio_of_cvs, digits = 1)

153.9

Instead, the Variability Ratio (%) is the slightly different value which is the ratio of the standard deviation estimates:

σ̂ₜ = 0.1178
σ̂ᵣ = 0.0767
variability_ratio = σ̂ₜ / σ̂ᵣ
round(100variability_ratio, digits = 1)

153.6

Note that in general standard deviations and CVs are quite close in value, but not identical.

An F-distribution based confidence interval

To make sure that the within subject variability of the test is not too high, a first approach may be to consider having a threshold based on the variability ratio. However, considering such a point estimate of this ratio does not take uncertainty of the estimates into consideration.

Instead we rely on the fact that we can compute a confidence interval for the actual ratio, \(\frac{\sigma_t}{\sigma_r}\), composed of the underlying within subject standard deviations of the test and reference product. Such a confidence interval relies on an F distribution and depends on the degrees of freedom used to estimate \(\sigma_t\) and \(\sigma_t\) (via the estimators \(σ̂ₜ\) and \(σ̂ᵣ\) respectively). These degrees of freedom were first introduced in Section 4 of Unit 10 dealing with estimation of within subject variability.

The standard pumas_be output does not present these degrees of freedom, but you may access them via the model_stats.within_subject_variability field:

be_result.model_stats.within_subject_variability |> simple_table

formulation	σw	k	σ_ratio	σ⁺
'R'	0.0767	10	missing	missing
'T'	0.118	9	1.54	2.72

Here we see that the degrees of freedom for the test are k=9 and for the reference are k=10. For this we rely on an F distribution with numerator and denominator degrees of freedom that are 9 and 10 respectively.

distribution = FDist(9, 10)

Distributions.FDist{Float64}(ν1=9.0, ν2=10.0)

Now for a \(90\%\) confidence interval for the ratio of standard deviations we need the 0.05 and 0.95 quantiles of this distribution

F_low = quantile(distribution, 0.05)
F_high = quantile(distribution, 0.95)
F_low, F_high

(0.31874743905891534, 3.020382947021373)

The confidence interval is then “around” the point estimate by scaling by these quantiles (taking square roots as this is a confidence interval for the standard deviation ratio):

confidence_interval = variability_ratio * [1 / √F_high, 1 / √F_low]
round.(confidence_interval, digits = 4) |> println

[0.8837, 2.7204]

We see that the confidence interval has an upper bound of 2.7204. This (up to rounding errors) is the Variability Ratio Quantile (95%) which appears in the pumas_be output and used for the third criterion.

The idea of failing (Fail) bioequivalence for NTID is to fail if that quantile exceeds \(2.5\) since this would indicate that there is a non-negligible possibility of having the within subject variability of the test to be more than \(2.5\) times that of the reference. Otherwise we Pass.

4 Conclusion

In this unit, we delved into the application of Reference Scaled Average Bioequivalence (RSABE) for Narrow Therapeutic Index Drugs (NTIDs), highlighting its distinctions from HVD analysis. We established the critical requirement of a fully replicate study design for NTIDs to accurately assess within-subject variability. The core of NTID bioequivalence involves satisfying three stringent criteria: passing standard ABE bounds (80-125%), rejecting the specific NTID RSABE hypothesis with a unique \(\theta\) value, and crucially, demonstrating an acceptable Variability Ratio Quantile (95%) that does not exceed 2.5. We explored how pumas_be facilitates this complex assessment and illustrated, through examples, that failing any one of these criteria leads to non-bioequivalence, underscoring the strict regulatory oversight for these critical medications.

5 Unit exercises

1. NTID RSABE Criteria

   1. List the three criteria that must be met for bioequivalence to be declared for an NTID using the FDA RSABE approach.
   2. What specific type of study design is required for NTID RSABE, and why?
   3. How does the reference scaling constant (\(\theta\)) for NTIDs differ from that used for HVDs (as seen in Unit 12)?

2. Understanding the Variability Ratio

   1. Explain the purpose of the Variability Ratio Quantile (95%) criterion in NTID bioequivalence.
   2. What is the numerical threshold for the Variability Ratio Quantile (95%) that must not be exceeded for an NTID to pass this criterion?
   3. Given the within-subject standard deviation estimates \(\sigmâₜ = 0.12\) and \(\sigmâᵣ = 0.06\), calculate the Variability Ratio (%).

3. Interpreting Results Consider a Pumas output for an NTID study:

   • Point Estimate: Pass
   • Howe’s Approx RSABE Stat (95%): 0.00008
   • Variability Ratio Quantile (95%): 2.1
   1. Based on these results, would the drug be considered bioequivalent? Explain why or why not for each criterion.
   2. If the Howe's Approx RSABE Stat (95%) was -0.00008 instead, how would the conclusion change?
   3. What would be the overall conclusion if the Variability Ratio Quantile (95%) was 2.6 (assuming the other two criteria passed)?

4. F-Distribution and Confidence Intervals

   1. Briefly describe how the F-distribution is used to construct the confidence interval for the ratio of standard deviations (\(\sigma_t / \sigma_r\)).
   2. Why is it important to use a confidence interval for the variability ratio, rather than just the point estimate of the ratio, in regulatory assessment for NTIDs?
   3. If you have degrees of freedom df_T = 15 and df_R = 12, and you want a 90% confidence interval, what quantiles of the F-distribution would you need?

5. Hypothetical Scenarios

   1. Describe a scenario where a drug passes the standard ABE criteria and the variability ratio criterion, but fails the NTID RSABE hypothesis test. What does this imply about the drug?
   2. Why is the Pass standard ABE (80-125 bounds) criterion still necessary for NTIDs, even when reference scaling is applied?
   3. If a hypothetical NTID study showed a Variability Ratio (%) of 180% and a Variability Ratio Quantile (95%) of 2.8, what would be the practical consequence for establishing bioequivalence?

References

FDA. 2012. “FDA, Draft Guidance on Warfarin Sodium. Recommended Dec 2012.” 2012. https://www.accessdata.fda.gov/drugsatfda_docs/psg/Warfarin_Sodium_tab_09218_RC12-12.pdf.