This simulation study examines how frailty, or heterogeneity in HIV infection risk within a study population, may impact the apparent declining efficacy seen in some randomized HIV intervention trials over time. The study uses mathematical modeling to simulate different trial scenarios varying factors like frailty level, intervention efficacy waning, and population risk distribution. The goal is to quantify how frailty alone could cause efficacy measures like the risk ratio to approach 1 at later time points even if the intervention effectiveness remains constant.

1. “Measuring the Potential Impact of Frailty on the Apparent
Declining Efficacy in Randomized Trials of HIV Interventions: A
Simulation Study”
Felicia P. Hardnett
Mathematical Statistician
Quantitative Sciences and Data Management Branch

2. Motivation
 Recent advancements in HIV prevention have
given researchers hope that effective HIV
interventions might soon become widely available
 Additional advancements in clinical trials
methodology have also occurred to measure the
efficacy of these interventions more accurately

3. Problem
 The results of recent HIV intervention trials have
been somewhat disappointing and difficult to
explain
 The efficacy of the interventions appears to
decline over time

4. Recently Published Trials
 Two recently published trials (1 vaccine trial and
1 microbicide trial) concluded that intervention
effectiveness decreased over time1,2.
 The investigators attributed this to:
 Waning vaccine efficacy (vaccine trial)
 Decreasing adherence (microbicide trial)
1Abdool Karim Q, Abdool Karim SS, Frohlich JA, Grobler AC, Baxter C, Mansoor LE, et al. Effectiveness and safety of tenofovir
gel, an antiretroviral microbicide, for the prevention of HIV infection in women. Science 2010; 329:1168–1174.
2Michael N. RV 144 update: vaccination with ALVAC and AIDSVAX to prevent hiv-1 infection in thai adults. 17th conference on
retroviruses & opportunistic infections, (2010).
http://app2.capitalreach.com/esp1204/servlet/tc?c¼10164&cn¼retro&e¼12354&m¼1&s¼20431&&espmt¼2&mp3file
¼12354&m4bfile¼12354.

5. Alternative Explanation
 In addition to these phenomena, the authors of a
recently published opinion piece assert that
frailty (due to heterogeneity in infection risk) is
another possible explanation1.
 This explanation is rarely cited in the literature as
a possible explanation for declining efficacy.
1O’Hagan JJ, Hernan MA, Walensky RP, Lipstitch M. Apparent declining efficacy in randomized trials: examples
of the Thai RV144 HIV vaccine and South African CAPRISA 004 microbicide trials. AIDS 2012, 26:123-126.

6. The Potential Impact of Frailty
 Even if the efficacy of an intervention remained
constant, frailty could give the appearance that its
declining
 This could cause researchers to reject an
effective intervention

7. Purpose
To explore the potential impact of frailty on the
results of randomized trials of HIV interventions

8. What is Frailty?

9. What is Frailty?
 Heterogeneity in infection risk within a study
population
 Causes change in the composition of the study
population over time
 Causes the measure of effect (risk ratio) to
approach 1 over time

10. Illustration of a hypothetical disease process
within a population
Population at risk
# persons who never
develop disease
Time
• As people become infected, the population at risk decreases over time and
eventually plateaus
• The rate of decline depends on disease incidence
• The curve plateaus at the number of persons who will never develop the
disease (low/no risk people)

11. Population at risk Illustration of Frailty as presented in the paper
High risk
Low risk
Time
The opinion piece asserts:
• High risk individuals will be infected early on and will be removed from
the population at risk first.
• This will leave lower risk individuals in the risk population resulting in lower
disease incidence at later time points.

12. Graphical representation of disease incidence
N0
High risk
Population at risk (N)
n1
Low risk
n2
# persons who never
develop disease
Time
t0 t1 t2 t3
Incidence=Number who become infected (n)
From t0 to t1
Number intially at risk (N0)

13. Graphical representation of disease incidence
N0
Population at risk (N)
n1
n2
# persons who never
develop disease
Time
t0 t1 t2 t3
• Fewer cases diagnosed at a later time point because the high risk
people are gone.
• Incidence, therefore decreases.

14. Intervention Scenario
Population at risk
Treatment arm
Placebo
RR=1
# persons who never
develop disease Time
• If the intervention is effective, it will prolong the time before high-risk individuals in
the treatment arm will become infected.
• Incidence decline in the placebo group will be larger because those at high risk will
be quickly removed from the population at risk.

15. Intervention Scenario
Rate ratio= incidence (treatment arm)
incidence(placebo)
Population at risk
Treatment arm
Placebo
RR=1
# persons who never
develop disease Time
• As a result, the time-specific rate ratio will increase from a value of less than one
to a value of one or greater.
• This process is termed “frailty”,“survivor bias”, “survivor cohort effect”,
“crossing of hazards” or “depletion of susceptibles”.

16. Intervention Scenario
Rate ratio= incidence (treatment arm)
incidence(placebo)
Population at risk
Treatment arm
Placebo
RR=1
# persons who never
develop disease Time
• As frailty increases, the curve becomes more steep early on and less steep towards
the end.
• RR approaches 1 sooner.

17. Intervention Scenario
Rate ratio= incidence (treatment arm)
incidence(placebo)
Population at risk
Treatment arm
Placebo
RR=1
# persons who never
develop disease Time
• As frailty increases, the curve becomes more steep early on and less steep towards
the end.
• RR approaches 1 sooner.

18. Possible Impact on Rate
Measures
 This risk ratio comparing the incidence in placebo
and treatment group becomes increasingly
attenuated as follow-up time increases.
 This occurs even if risk factors were balanced
between study arms at baseline and if effect of
intervention is constant over time.

19. Is Frailty Really Important?
Based on the information presented in the paper:
With competing factors such as waning vaccine
efficacy and decreasing adherence, it’s not clear
how important frailty is in explaining declining
efficacy in the two trials.

20. Current Study Objective
To quantify the potential impact of frailty under
various study scenarios.

21. Current Study Approach
 We designed several study scenarios using
study-related, intervention-related and
population-related parameters.
 We held study-related factors constant (e.g.,
sample size, follow-up time, intervention
effectiveness).

22. Current Study Approach
 We varied population and intervention-related
parameters (e.g. waning and frailty)
 We estimated the risk ratio at each time point for
each scenario and quantified the change that is
attributable to frailty.

23. Modeling Description
 Definition of Parameters
 Model Assumptions
 Scenario Design
 Results
 Conclusions

24. Modeling Description
 Definition of Parameters
 Model Assumptions
 Scenario Design
 Results
 Conclusions

25. Model Parameters
Study Intervention Population
Fixed • Sample Size Intervention Distribution of
• Follow-up Efficacy Population
Period across Risk
• Number of Groups
Risk Groups
Varied Waning Probability of
-- Disease
25

26. Risk Groups
The study population is divided into mutually-
exclusive groups ranging from very high risk to very
low risk.

27. Risk Distribution


28. The probability of an individual within a risk group
becoming infected within a given study period.

29. Frailty
The degree of heterogeneity in the probability of
disease within the study population.

30. Intervention Effectiveness (E)
The percent reduction in the probability of disease
that is conferred by the intervention.

31. Waning (W)
The proportional reduction in intervention
effectiveness that occurs over time.
Et= E0 * (1-W)t

32. Measures of Effect
32

33. Disease Incidence
The proportion of the population that becomes
infected within a given time frame.

34. Risk Ratio (Outcome Measure)


35. Modeling Description
 Definition of Parameters
 Model Assumptions
 Scenario Design
 Results
 Conclusions

36. Model Assumptions
 Sufficient sample size
 The treatment arms have equal sample sizes.
 Disease risk is balanced between both treatment
arms at the beginning of the study.
 Non-differential loss to follow up.

37. Model Assumptions
 The intervention is effective at reducing the
probability of disease and presents no adverse
effects (i.e., increasing in the probability of
infection) at any point in time.
 Intervention efficacy is constant across all risk
groups.
 Intervention waning/non-adherence is constant
across all risk groups.

38. Modeling Description
 Definition of Parameters
 Model Assumptions
 Scenario Design
 Results
 Conclusions

39. Features of Study Scenarios that
remain fixed
 Equal sample size in each treatment arm
 Ten-year follow-up time
 Five HIV risk groups ranging from very high risk
to very low risk
 Intervention effectiveness - 50%

40. Features of Study Scenarios that
remain fixed
Distribution of study population across risk groups
0.6
0.5
0.4
0.3
0.2
0.1
0
Very High High Moderate Low Very Low

41. Features of Study Scenarios that
are varied
 Waning- the rate at which the intervention loses
its effectiveness
 Frailty- heterogeneity in disease risk across the
5 risk groups

42. Intervention Efficacy
Waning Parameter
Time

43. Frailty Parameter
Probability of Infection
0% 20% 30% 50% 80%
43
Degree of Heterogeneity

44. Frailty Parameter
Probability of Infection
0% 20% 30% 50% 80%
44
Degree of Heterogeneity

45. Frailty Parameter
Probability of Infection
0% 20% 30% 50% 80%
45
Degree of Heterogeneity

46. Placebo Group No Frailty/ No Waning
Baseline Time 1 Time 2 Time 3
proportion proportion proportion proportion proportion proportion
that that that that that that
become remains Population become remains Population become remains Population
Population Probability infected uninfected Proportion infected uninfected Proportion infected uninfected Proportion
Risk Group proportion of disease (time 1) (time 1) (time 1) (time 2) (time 2) (time 2) (time 3) (time 3) (time 3)
1 .05 .05 0.025 0.025 0.05 0.025 0.025 0.05 0.025 0.025 0.05
2 .2 .05 0.1 0.1 0.2 0.1 0.1 0.2 0.1 0.1 0.2
3 .05 .05 0.25 0.25 0.5 0.25 0.25 0.5 0.25 0.25 0.5
4 .2 .05 0.1 0.1 0.2 0.1 0.1 0.2 0.1 0.1 0.2
5 .05 .05 0.025 0.025 0.05 0.025 0.025 0.05 0.025 0.025 0.05
0.5 0.5 1 0.5 0.5 1 0.5 0.5 1
Treatment E=50% E=50% E=50%
Group
proportion proportion proportion proportion proportion proportion
Interven that that that that that that
tion become remains Population become remains Population become remains Population
Underlying Probability Effective infected uninfected Proportion infected uninfected Proportion infected uninfected Proportion
Risk Group starting pop of disease ness (time 1) (time 1) (time 1) (time 2) (time 2) (time 2) (time 3) (time 3) (time 3)
1 .05 .05 .5 0.0125 0.0375 0.05 0.0125 0.0375 0.05 0.0125 0.0375 0.05
2 .2 .05 .5 0.05 0.15 0.2 0.05 0.15 0.2 0.05 0.15 0.2
3 .05 .05 .5 0.125 0.375 0.5 0.125 0.375 0.5 0.125 0.375 0.5
4 .2 .05 .5 0.05 0.15 0.2 0.05 0.15 0.2 0.05 0.15 0.2
5 .05 .05 .5 0.0125 0.0375 0.05 0.0125 0.0375 0.05 0.0125 0.0375 0.05
.25 .75 1 0.25 0.75 1 0.25 0.75 1
IR=.5 IR=.5 IR=.5 46

47. Placebo Group Moderate Frailty/ 10% Waning
Baseline Time 1 Time 2 Time 3
proportion proportion proportion proportion proportion proportion
that that that that that that
become remains Population become remains Population become remains Population
Population Probability infected uninfected Proportion infected uninfected Proportion infected uninfected Proportion
Risk Group proportion of disease (time 1) (time 1) (time 1) (time 2) (time 2) (time 2) (time 3) (time 3) (time 3)
1 .05 0.3 0.015 0.035 0.04140 0.01242 0.02898 0.0342 0.0102 0.0239 0.0281
2 .2 0.21 0.042 0.158 0.18691 0.03925 0.14766 0.1741 0.0366 0.1375 0.1615
3 .05 0.147 0.0735 0.4265 0.50454 0.07417 0.43038 0.5073 0.0746 0.4327 0.5084
4 .2 0.1029 0.0205 0.1794 0.21225 0.02184 0.19041 0.2244 0.0231 0.2013 0.2365
5 .05 0.07203 0.0036 0.0464 0.05488 0.00395 0.05094 0.0600 0.0043 0.0557 0.0655
0.1546 0.8453 1 0.15164 0.84836 1 0.1488 0.8512 1
Treatment E=50% E=45% E=40.5%
Group
proportion proportion proportion proportion proportion proportion
Intervent that that that that that that
ion become remains Population become remains Population become remains Population
Underlying Probability Effectiven infected uninfected Proportion infected uninfected Proportion infected uninfected Proportion
Risk Group starting pop of disease ess (time 1) (time 1) (time 1) (time 2) (time 2) (time 2) (time 3) (time 3) (time 3)
1 .05 0.3 .5 0.0075 0.0425 0.0461 0.0076 0.0385 0.0420 0.0075 0.0345 0.0379
2 .2 0.21 .5 0.0210 0.1790 0.1940 0.0224 0.1716 0.1874 0.0234 0.1640 0.1803
3 .05 0.147 .5 0.0368 0.4633 0.5021 0.0406 0.4615 0.5040 0.0441 0.4599 0.5056
4 .2 0.1029 .5 0.0103 0.1897 0.2056 0.0116 0.1940 0.2118 0.0130 0.1989 0.2186
5 .05 0.07203 .5 0.0018 0.0482 0.0522 0.0021 0.0502 0.0548 0.0023 0.0524 0.0576
0.0773 0.9227 1 0.0843 0.9157 1 0.0903 0.9097 1
IR=.5 IR=.56 IR=.61 47

48. Modeling Description
 Definition of Parameters
 Model Assumptions
 Scenario Design
 Results
 Conclusions

49. 49

50. 50

51. 51

52. 52

53. 53

54. 54

55. 55

56. 56

57. Modeling Description
 Definition of Parameters
 Model Assumptions
 Scenario Design
 Results
 Conclusions

58. Conclusions
 With the exception of the most extreme cases,
frailty (heterogeneity in disease risk) doesn’t
appear to have much of an impact on outcome
measures in randomized trials of HIV
interventions
 The impact of frailty appears substantial in
scenarios when HIV infection is a virtual
certainty in the highest risk group and negligible
in the lowest risk group

59. Conclusions
 This study condition is unlikely to occur in most
trials where higher risk individuals are commonly
recruited.
 Therefore, frailty is less likely to explain a
substantial portion of the declining efficacy in
many HIV intervention trials

60. QUESTIONS?