Minimisation is an approach to allocating patients to treatment in clinical trials that forces a greater degree of balance than does randomisation. Here I explain why I dislike it.
2. (c) Stephen Senn 2008 2
The Value of Marketing
• I used to work in the
pharmaceutical industry
and know the value of
marketing
• In those days I would have
taken this opportunity to
promote my new book
• Now that I am an
academic, modesty forbids
me to do so
3. (c) Stephen Senn 2008 3
Outline
• Why you must condition on covariates
• The true value of balance
• What is minimisation
• How to balance if you must
• Variance matters
• Why I hate minimisation
4. (c) Stephen Senn 2008 4
Terminology
• Regression
– (Usually) that field of statistics that deals with estimating
parameters of linear equations using (ordinary) least squares
• Covariate – something you model and which will help
sharpen your inferences about the effect of treatment but is
not of direct interest
– Example age in a placebo-controlled trial of a beta-blocker in
hypertension
• Unbiased estimator
– A means of estimating a quantity that would on average over all
similar circumstances in which you used it equal the thing you
were estimating
5. (c) Stephen Senn 2008 5
Typical MRC Stuff
‘The central telephone randomisation system used a minimisation
algorithm to balance the treatment groups with respect to eligibility
criteria and other major prognostic factors.’ (p24)
‘All comparisons involved logrank analyses of the first occurrence
of particular events during the scheduled treatment period after
randomisation among all those allocated the vitamins versus all
those allocated matching placebo capsules (ie, they were “intention-
to treat” analyses).’ (p24)
1. (2002) MRC/BHF Heart Protection Study of cholesterol lowering with
simvastatin in 20,536 high-risk individuals: a randomised placebo-controlled trial.
Lancet 360:7-22
6. (c) Stephen Senn 2008 6
Three Games with Two Dice
• The object is to call the odds for getting a score of ten in
rolling two dice
– A red die and a black die
• The game is played three different ways
– Game 1. The two dice are rolled together you call the
odds before they are rolled
– Game 2. The red die is rolled you are shown the score
and then call the odds before the black die is rolled
– Game 3. You call the odds. The red die is rolled first
but you are not shown it and then the black one is
rolled
• How should you bet?
8. (c) Stephen Senn 2008 8
Game 2: Either the probability = 0 or the probability = 1/6
Game 3: The probability = ½ x 0 + ½ x 1/6= 1/12
9. (c) Stephen Senn 2008 9
The Morals
• You can’t treat game 2 like game 1.
– You must condition on the information you receive in order to be
wisely
– You must use the actual data from the red die
• You can treat game 3 like game 1.
– You can use the distribution in probability that the red die has
• You can’t ignore an observed prognostic covariate in
analysing a clinical trial just because you randomised
– That would be to treat game 2 like game 1
• You can ignore an unobserved covariate precisely because
you did randomise
– Because you are entitled to treat game 3 like game 1
10. (c) Stephen Senn 2008 10
Trialists continue to use their randomisation
as an excuse for ignoring prognostic
information, and they continue to worry
about the effect of factors they have not
measured. Neither practice is logical.
The Reality
11. (c) Stephen Senn 2008 11
The True Value of Balance
• It is generally held as being self evident that
a trial which is not balanced is not valid.
• Trials are examined at baseline to establish
their validity.
• In fact the matter is not so simple...........
12. (c) Stephen Senn 2008 12
A Tale of two Tables
Trial 1 Treatment
Sex Verum Placebo TOTAL TOTA
Male 34 26 60
Female 15 25 40
TOTAL 49 51 100
Trial 2 Treatment TOTA
Sex Verum Placebo TOTAL
Male 26 26 52
Female 15 15 30
TOTAL 41 41 82
13. (c) Stephen Senn 2008 13
Choices, Choices
Trial two is balanced whereas trial one is not.
One might think that trial two provides the more reliable
information.
However, the reverse is the case.
Trial one contains a comparable trial to trial two within it.
It is simply trial two with the addition of 8 further male patients
in the verum group and 10 further female patients in the placebo
group.
How could more information be worse than less?
14. (c) Stephen Senn 2008 14
Stratification
All we need to do is compare like with like.
If we compare males with males and females with females we
shall obtain two unbiased estimators of the treatment effects.
These can then be combined in some appropriate way. This
technique is called stratification.
A similar approach called analysis of covariance is available to
deal with continuous covariates such as height, age or a baseline
measurement.
15. (c) Stephen Senn 2008 15
Validity and Efficiency
• So if balance is not necessary to produce unbiased
estimates what is its value?
• The answer is that it leads to efficient estimates
• If you condition on covariates the estimate will be
more efficient the closer they are to being
balanced
• However if you don’t condition on observed
balanced covariates your inferences are invalid
– Example analysing a matched pairs design using the
two-sample t-test
16. (c) Stephen Senn 2008 16
A Definition
Minimization. A procedure for allocating patients
sequentially to clinical trials taking account of the
demographic characteristics of the given patient
and of those already entered on the trial in such a
way as to attempt to deliberately balance the
groups. In
practice this achieves a higher degree of balance
than for randomization, although the difference for
even moderately sized trials is small.
Where did I find this definition?
I read it in a book I rather like
In fact, I wrote it myself
17. (c) Stephen Senn 2008 17
What is minimisation?
• Approach to ‘balancing’ clinical trials
• Proposed by Taves (1974) and Pocock &
Simon (1975)
• Uses marginal balance
• Reflects the reality that patients in clinical
trials are treated when they present
• Avoids the ‘deep-freeze microwave’ fallacy
18. (c) Stephen Senn 2008 18
An Example
We can explain this with the help of an example of Pocock’s
(Clinical Trials A Practical Approach, Wiley, 1983, p85).
Suppose that we wish to balance patients in a clinical trial for
advanced breast cancer with respect to 4 factors: performance
status (ambulatory and non-ambulatory), age ( < 50, ≥ 50),
disease free interval (<2 years, ≥ 2 years), dominant metastatic
lesion (visceral, osseous, soft tissue).
Suppose that the next patient to be entered is ambulatory, age <
50, disease free interval ≥ 2 years and visceral metastasis and
suppose that 80 patients have been entered already so that the
position is as given in the table below.
19. (c) Stephen Senn 2008 19
Factor Level No. on each treatment
A B
Next
Patient
Performance
Status
Ambulatory
Non-Amb.
30
10
31
9
*
Age < 50
≥ 50
18
22
17
23
*
Disease-free
Interval
< 2 years
≥ 2 years
31
9
32
8 *
Dominant
met. lesion
Visceral
Osseous
Soft tissue
19
8
13
21
7
12
*
Using the levels of the factors for this particular patient we
see that the sum for A is 30+18+9+19=76 while for B it is
31+17+8+21 = 77. Therefore we assign the patient to A.
20. (c) Stephen Senn 2008 20
What you learn in your first regression
course
1 11 1 0 1
2 12 2 1 2
1
1
1 ...
1 ...
k
k
n n kn k n
Y X X
Y X X
Y X X
β ε
β ε
β ε
÷ ÷ ÷ ÷
÷ ÷ ÷ ÷= = = =
÷ ÷ ÷ ÷
÷ ÷ ÷ ÷
Xβ ε
Y = Xβ + ε
L
M M M O M M M
Y
( )ˆ ′ ′
-1
β = X X X Y ( )
1 2ˆ ˆ( ) , ( ) .E V σ
−
′= =β β β X X
Design matrixResponse vector
Parameter
vector
Disturbance
term vector
Estimator
Unbiased
Variance of the
disturbance termsVariance of the
estimates
21. (c) Stephen Senn 2008 21
1 2
11 12 1
12 22 2
1
22
ˆvar( ) ( )
2/
k
k kk
X X
a a a
a a
a a
a n
β σ
σ
−
′=
÷
÷=
÷
÷
≥
The value of σ2
depends
on the model.
For a given model, the
value of a22 depends on
the design and this only
achieves its lower bound
when covariates are
balanced.
The Value of Balance
Variance multiplier for the treatment effect
22. (c) Stephen Senn 2008 22
0 10 20 30 40 50 60
Patients in Trial
0.0
0.2
0.4
0.6
0.8
1.0
Efficiency
Efficiency of Randomised Trial Compared to Balanced One
23. (c) Stephen Senn 2008 23
6 8 10 12 14 16 18 20 22 24
Sample size
0.6
0.7
0.8
0.9
1.0
Efficiency
Efficiency of Randomised Design Compared to a Balanced One
10 x 1000 runs at each sample size
Balanced Numbers: One Covariate
run
mean
formula
24. (c) Stephen Senn 2008 24
Minimisation Doesn’t even
Minimise Well
• In fact minimisation achieves marginal and
partly irrelevant balance
– Adding together apples and pears
• The best solution as ‘enny fule kan kno’ is
to work with the design matrix
• Allocate so that the variance multiplier for
the treatment effect is as favourable as
possible
25. (c) Stephen Senn 2008 25
Atkinson’s Approach
11 12 1
21 22 2
0 1
k
k
k k kk
a a a
a a a
a a a
÷
÷=
÷
÷
A
L
L
M M O M
K
2ˆvar( )i iiaβ σ= Choose allocations such
that a22 is minimised
( )′
-1
X X = A
26. (c) Stephen Senn 2008 26
Non-Linear Models
• This is more complicated
• Variance depends not just on design matrix
but on response
• Nevertheless design matrix is important
• Furthermore the problems in not
conditioning are worse
– Gail et al, Robinson & Jewel, Ford et al
27. (c) Stephen Senn 2008 27
A Problem
• All such sequential balancing methods restrict the
randomisation strongly to a degree beyond that
necessary to balance by the factor by the end of
the trial
• This may lead to invalid variance estimates
– incompatible with Fisher philosophy of randomised
experiments
• see also Nelder general balance
• Student’s (and Taves’s) argument would be that it
leads to conservative inference
– and that this is good
28. (c) Stephen Senn 2008 28
Don’t Forget the Variance
Estimate
Full "Correct"
Model
Reduced
Randomised
Reduced
Minimised
Treatment Treatment Treatment
Covariate
Error Error Error
Total Total Total
29. (c) Stephen Senn 2008 29
A Red Herring
• The minimisers common defence is ‘conservative
inference’
• ‘So what if our reported standard errors are higher
than the true ones’
• ‘The result is conservative inference’
• If you like conservative inference why not just
multiply all your standard errors by two?
• The next slide shows the consequence of
conservative inference
30. (c) Stephen Senn 2008 30
Variance, Randomisation and Meta-
Analysis
Trial True
Variance
Weight Est
Variance
Weight
Randomised Higher Less Lower More
Balanced Lower More Higher Less
Consider the meta-analysis of two otherwise identical
trials: one randomised, one balanced.
Should be Will be
31. (c) Stephen Senn 2008 31
How to Eliminate The Effect of
Covariates by Allocation Alone
A B
Males 50 50
Females 50 50
A B
Males 100
Females 100
This eliminates the effect
of sex from the unadjusted
treatment difference
This eliminates the effect
of sex from the unadjusted
within-treatment variance
estimate
32. (c) Stephen Senn 2008 32
How to Eliminate the Effect of a Covariate from
Estimated Treatment Effect and Variance
A B
Male 47 53
Female 53 47
You do this by
conditioning on sex
(modelling) whether or
not sex and treatment
are balanced
33. (c) Stephen Senn 2008 33
What About Bayesians?
• Belief dictates the model
• The model dictates the analysis
• The design determines efficiency
– Design does not dictate analysis
• Randomised designs are (slightly) less efficient
• Why randomise?
– Randomisation prevents your having to factor in your beliefs about
how the trialists will behave
• Balance what you can and randomise what you can’t but
neither balance nor randomisation is an excuse for not
conditioning
34. (c) Stephen Senn 2008 34
Why I Hate Minimisation
Reasons
• It is not based on sound design theory
• Its contribution to improving efficiency is minimal
• It violates randomisation based analysis
• People who use it don’t even do good model-
based analysis (MRC, EORTC etc balance but
don’t condition)
– See the example of the MRC/BHF trial
• We should fit covariates not find elaborate
excuses to ignore them
35. (c) Stephen Senn 2008 35
In Short
I’d rather be hanged for a
sheep than a lamb
And I for one will not let
the minimisers pull the
wool over my eyes
36. (c) Stephen Senn 2008 36
My Philosophy of Clinical Trials
• Your (reasonable) beliefs dictate the model
• You should try measure what you think is important
• You should try fit what you have measured
– Caveat : random regressors and the Gauss-Markov theorem
• If you can balance what is important so much the better
– But fitting is more important than balancing
• Randomisation deals with unmeasured covariates
– You can use the distribution in probability of unmeasured covariates
– For measured covariates you must use the actual observed distribution
• Claiming to do ‘conservative inference’ is just a
convenient way of hiding bad practice
– Who thinks apart from the MRC that analysing a matched pairs t as a two
sample t is acceptable?
37. (c) Stephen Senn 2008 37
What’s out and What’s in
Out In
• Log-rank test
• T-test on change scores
• Chi-square tests on 2 x 2
tables
• Responder analysis and
dichotomies
• Balancing as an excuse for
not conditioning
• Proportional hazards
• Analysis of covariance
fitting baseline
• Logistic regression fitting
covariates
• Analysis of original values
• Modelling as a guide for
designs
38. (c) Stephen Senn 2008 38
"overpaid, oversexed and over here".
Tommy Trinder (1909-1989) on the subject of the GIs in WWII
“Over-hyped, overused and overdue for retirement”
Stephen Senn on minimisation
A plea to all right- thinking statisticians. Help me consign this
piece of garbage to the rubbish dump of history
39. (c) Stephen Senn 2008 39
Finally
• I leave you with this
thought
• Did you know that
there are only 260
shopping days until
Christmas
• May I make a small
suggestion?
Notas do Editor
&lt;number&gt;
From Senn, Statistical Issues in Drug Development, (Second edition due Jan 2008)