3. There are many interventions - in
schools, families and the community - which
claim to reduce drug misuse.
4. ? ?
How can we test whether they actually work?
? ?
5. Higher quality trials give more certainty
about results
Randomised
controlled trial
Best practice
Statistically matched
control group
Before and after data
with a control group
Before and after data
6. Measure outcomes before and after intervention
5%
Percent 4% 4.0%
3.5%
drinking
3%
alcohol in
past week 2%
1%
0%
Before After
Intervention
What does this tell us?
Very little without a control group.
7. With a control group...
5% 4.7% Control
Percent 4% 4.0% Smaller increase in
3.5%
drinking drinking in
3% 3.2%
alcohol in intervention group
past week 2%
1%
0%
Before After
Intervention
8. What do we know about the control group?
The following methods give increasing confidence that
the comparison is a fair one:
• Common sense – avoiding obvious pitfalls such as
those trying out the intervention tending to be in
more prosperous areas.
• Statistical ‘matching’ techniques using more
information about individuals in the groups.
• Random allocation between the intervention and
control groups: this is called a “Randomised
controlled trial”
10. Sample size matters
• This is because a behaviour (for example drinking in
the past week) will vary between individuals across
the wider population.
• If a sample of 1000 15 year olds in London are
randomly selected and 200 (20%) drank in the past
week, you can be confident that the true percentage
for all 15 year olds in London is fairly close to 20%.
• In contrast, if you select ten young people at random
and two of them drank in the past week, that doesn’t
tell you much at all.
11. Statistical analysis
• So, as well as looking at the size of the difference
between your intervention and control groups, the
size of your sample is also important.
• Larger samples give more confidence that the
difference between two groups is genuine and not
due to random variation.
• Once it is calculated as being sufficiently unlikely
(less than a 5% chance) that the result is due to
random variation, the finding is said to be
‘statistically significant’.
12. Find out more...
• The Centre for Analysis of Youth Transitions (CAYT) is
creating a repository of impact studies on young people’s
services and programmes -
http://www.ifs.org.uk/centres/caytRepository
• European Quality Standards have been developed for
drug prevention and contain useful information on
evaluating outcomes.
http://www.emcdda.europa.eu/publications/manuals/pr
evention-standards
• ‘Test, Learn, Adapt: Developing Public Policy with
Randomised Controlled Trials’ is a more general
discussion of RCTs in public policy
http://bit.ly/MHJ4aP
