Hsini (Terry) Liao, Ph.D., Yun Lu, Hong Wang, “Comparison of Individual Patient-Level and Study-Level Meta-Analyses Using time to Event Analysis in Drug-Eluting Stent Data”, Abstract No 301037, Joint Statistical Meetings, Session No 90, Denver, CO, August 2008
1. Comparison of Individual Patient-Level and
Study-Level Meta-Analyses Using Time to
Event Analysis in Drug-Eluting Stent Data
Hsini Liao, Yun Lu, and Hong Wang
Presented to JSM 2008
Boston Scientific Corporation
JSM 2008 1
2. Conflict of Interest Disclosures
DISCLOSURE INFORMATION:
The following relationships exist related
to this presentation:
Hsini Liao, Yun Lu, Hong Wang
Full time employees of
Boston Scientific Corporation
JSM 2008 2
3. Outlines
• Motivation
• Meta-Analysis
• Time to Event Analysis
• Stent Data
• References
JSM 2008 3
4. Motivation
• Meta-analysis provides a structure of consolidating
the outcomes from several studies and deriving
statistical inferences of the outcomes.
• Meta-analysis of time-to-event data is less common
than of binary or continuous data.
• Aggregate Data (AD) – Two-Step Approach
• Individual Patient Data (IPD) – One-Step Approach
• Comparison of the analysis of AD and IPD should be
found discrepant results, but no clear general
systematic differences, especially when homogeneity
is assumed.
• AD vs IPD in TTE DES Data
JSM 2008 4
5. Meta-Analysis
• A Systematic Review of Literature to Measure the
Effect Size
• Single Study/Effect
• Many Studies/Narrative Review
• Effect Magnitude/Adequate Precision
• Combine the Effects to Give Overall Mean Effect
• A Recent Survey in Practice (Simmonds et al,
2005): Majority used simple fixed-effect model;
only small proportion considered among-study
heterogeneity
JSM 2008 5
6. Meta-Analysis
(AD vs IPD)
• Effect Size: Event Rate, • Ensuring Data Quality
OR, RR, HR, Corr, etc. (e.g. Date of Outcome)
• Sample Size/Standard • Detailed Data Checking
Error to Assign Weight (e.g. Randomization)
• Limit Analyses (e.g. • Ensuring the
Fixed/Random Effect, Appropriateness of the
Meta-Regression) Analyses (e.g. KM)
• Less Time and Costs • More Time and costs
JSM 2008 6
7. Fixed Effect Model
(AD: FEM)
• The FEM assumes that all studies in the
meta-analysis are drawn from a common
population.
• The observed effect size varies from one
study to the next only because of the
random error inherent in each study.
• Under the FEM there is one true effect
size. The combined effect is an estimate
of this value.
JSM 2008 7
8. FEM (Cont’d)
εA FEM with
Study A sampling error
TA (real world)
εB
Study B
TB
εC
Study C
e.g. TA= 0.6 – 0.1 = 0.5
TC
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2
JSM 2008 8
9. FEM (Cont’d)
• More generally, the observed effect Tj
for any study is given by the
population mean plus the sampling
error in that study. That is,
Tj = µ + εj
εj~N(0,σ2)
JSM 2008 j = 1,…,k 9
10. Random Effects Model
(AD: REM)
• The REM assumes that the studies are drawn from
populations that differ from each other in ways that
could impact on the treatment effect.
• The observed effect size varies from one study to
the next for two reasons. The first is random error
within studies, as in the FEM. The second is true
variation in effect size from one study to the next.
• Under the REM there is not one true effect size, but
a distribution of effect sizes. The combined effect is
not an estimate of one value, but is meant to be
the mean of a distribution of values.
JSM 2008 10
11. REM (Cont’d)
εA µA
REM, true effect and
Study A observed effect in one
TA study (real world)
ξA
e.g. TA= 0.60 – 0.05 – 0.15 = 0.40
µ
0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 1.10 1.20
JSM 2008 11
12. REM (Cont’d)
• More generally, the observed effect Tj for any study
is given by the grand mean µ, the deviation of this
study’s true effect from the grand mean, and the
deviation of this study’s observed effect from this
study’s true effect.
Tj = µ + ξj + εj
ξj ~ N(0,τ2); εj~N(0,σj2)
JSM 2008
j = 1,…,k 12
13. Heterogeneity
• Using FEM/REM is based on assumption.
• FEM
• Ignore heterogeneity
• Removing outliers causes biasness.
• REM
• Q Statistic is defined as total sum of squares.
• I2 Statistics to describe the ratio of true/total variance
• Subgroup/Sensitivity Analysis, ANOVA, Meta-
Regression
JSM 2008 13
14. Heterogeneity (Cont’d)
• Total Sums of Squares: k
k
(∑ wiTi ) 2
Q = ∑ wiTi 2 −
k
Q = ∑ wi (Ti − T• )
i =1
2 k
i =1
i =1
∑w i =1
i
• Test of Heterogeneity ( H 0 : θ1 = θ 2 = ... = θ k )
Q ~ χ 2
k −1
• Heterogeneity (Between-Studies Variance):
⎧ Q − df , if Q > df
⎪
τ =⎨ C
2
JSM 2008
⎪ 0 , if Q ≤ df
⎩ 14
15. Time to Event Analysis
(AD vs IPD)
• HR derived from • Model time to event
published data. • Account for censoring
• Always compare 2 • Compare survival curve
groups between 2+ groups
• Estimate HR from KM • Assess relationship
between survival time and
• Not able to adjust for covariates.
covariates, but able to • Univariate Method:
select HR being Kaplan-Meier (KM) Curves
adjusted for same
covariates. • Multivariate Method: Cox-
Proportional Hazards
Model
h(t) = λ(t)*exp(Xiβi)
i = 1,…,k
JSM 2008 15
16. Stent Data
• Study Outcome: Target Vessel Revascularization
• Treatment: Drug-Eluting Stent vs. Bare-Metal Stent
• Data: total 825 diabetic randomized patients over 5
studies
• Study A: (n=11) 5 years
• Study B: (n=51) 5 years
• Study C: (n=318) 5 years
• Study D: (n=356) 3 years
• Study E: (n=89) 5 years
• The hazard ratio estimate of study outcomes from
pooled IPD is compared with that from the AD to
assess the treatment effect in diabetic patients.
JSM 2008 16
17. Study Outcome: TVR
In-segment
5 mm 5 mm
In-stent
proximal distal
edge Covers entire stented length edge
JSM 2008 17
20. HR for Pooled IPD
Treatment Effect without Adjustment
JSM 2008 20
21. Meta-Analysis of TVR for Diabetic Patients (AD)
Study name Statistics for each study Hazard ratio and 95% CI
Hazard Lower Upper Relative
ratio limit limit p-Value weight
Study A 1.172 0.106 12.978 0.897 1.00
Study B 0.768 0.357 1.651 0.499 9.88
Study C 0.689 0.488 0.971 0.034 49.01
Study D 0.818 0.538 1.242 0.345 33.09
Study E 0.553 0.223 1.372 0.202 7.02
0.729 0.573 0.928 0.010
0.1 0.2 0.5 1 2 5 10
DES Better BMS Better
Fixed Effect Model
JSM 2008 21
22. Test of Heterogeneity
Heterogeneity (AD)
Q-value DF (Q) P-value I-squared
0.914 4 0.923 0.000
JSM 2008 22
23. AD vs IPD in DES Data
Treatment Effect without Adjustment
Hazard Log(HR) StdErr 95% CI
Ratio
IPD .6006 -.5098 .1370 [.4592, .7857]
AD .6013 -.5087 .1387 [.4581, .7891]
JSM 2008 23
24. AD vs IPD
Study-Level Patient-Level
Sample Size Always Small Relatively Large
Visual Forest Plot, Sensitivity Kaplan-Meier
Presentation Analysis
for TTE Data
Model Used Fixed/Random Effects Cox Regression Model
Model with Fixed Treatment
Effect
Heterogeneity No by Test of No by Model Assumption
Found Heterogeneity
JSM 2008 24
25. AD vs IPD (Cont’d)
Study-Level Patient-Level
Subgroup Subgroup of studies is Subgroups are able to be
Analysis available. Outcome for freely defined. The
subgroup of baseline power for each subgroup
may not be published. can be calculated.
Individual vs Overall may not be Should check poolability
Overall consistent with each between study and
individual study. study.
JSM 2008 25
26. Other Topics
• Extracting information from survival
curve
• Published with insufficient details
JSM 2008 26
27. References
• Riley RD, Lambert PC, Staessen JA, Wang J, Gueyffier F, Thijs L, Boutitie,
F. “Meta-Analysis of Continuous Outcomes Combining Individual Patient
Data and Aggregate Data”, Stat in Med. 2008; 27:1870-1893
• Simmonds MC, Higgins JPT, Stewart LA, Tierney JF, Clarke MJ,
Thompson SG. “Meta-Analysis of Individual Patient Data from
Randomized Trials: A Review of Methods Used in Practice”, Clin Tri.
2005; 2:209-217
• Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. “Introduction to
Meta-Analysis” (Draft), 2007
• Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F. “Methods for
Meta-Analysis in Medical Research” (Reprint with Correction), 2004
• Parmar MKB, Torri V, Stewart L. “Extracting Summary Statistics to
Perform Meta-Analyses to the Published Literature for Survival
Endpoints”, Stat in Med. 1998; 17:2815-2834
• Smith CT, Williamson PR, Marson AG. “Investigating Heterogeneity in an
Individual Patient Data Meta-Analysis of Time to Event Outcomes”, Stat
in Med. 2005; 24:1307-1319
• Software: “Comprehensive Meta-Analysis” (CMA), Version 2.2.030
• Software: “SAS”, Version 9.1
JSM 2008 27