Nick Axford's presentation from the March 7 Mentor-ADEPIS seminar 'Targeted prevention interventions and the case of PreVenture'

1. Outline
1. Programmes and standards of evidence
2. Where do targeted interventions fit?
3. A key challenge for targeted provision
4. A brave new world?

2. Evidence-based programmes (EBPs)
• A programme is a discrete, organised package of practices, spelled out in
guidance (sometimes called a manual) that explains what should be
delivered to whom, when, where and how.
• A programme is ‘evidence-based’ when it is ‘tested and effective’:
- Tested’ means that the programme has been put through its paces by a
high-quality impact evaluation
- ‘Effective’ means that there is strong evidence from that evaluation
that the programme makes life better for children or families

4. Standards of evidence
1. Intervention specificity – what is it?
2. Impact – does it work?
3. Evaluation quality – can we be confident in the results?
4. System readiness – can it be replicated?
www.investinginchildren.eu/standards-evidence

5. Continuum of prevention
1. Promotion – universal (promoting good)
2. Universal prevention – universal (preventing bad)
3. Selective prevention – elevated risk (group)
4. Indicated prevention – elevated risk (individual) / early problems
5. Treatment – established disorder
6. Maintenance – preventing relapse
O’Connell et al. (2009)

6. Spend on children in Northern Ireland
Executive Departments Levels 1-2 Levels 3-4 Levels 5-6 Not disaggregated
£m £m £m £m
Agriculture & Rural Development 1.34
Culture, Arts & Leisure 17.39 0.65 0.05
Education 1,193.99 212.41 0.97 225.24
Employment & Learning 48.95 54.41 0.03
Enterprise, Trade & Investment
Finance & Personnel
Health, Social Services, Safety 29.35 17.00 424.08 13.57
Environment 3.06 0.05
Justice 0.80 14.90 0.03
Regional Development 7.67
Social Development 7.71 0.77
First & deputy First Minister 0.25 1.57
Total 1,301.75 293.28 440.75 240.46
Percentage split 57% 13% 19% 11%

7. 4 circles
Children in need
Children in contact with specialist services
Children with informal support

8. Provide more services
4 circles

9. Re-focus services
4 circles

10. Reduce need
4 circles

12. KiVa – bullying prevention
• “Bullying”: verbal/psychological/physical behaviour, designed to
cause harm/distress, direct/indirect, repeated over time, power
differential
• Universal element:
- Classroom lessons (10 over year)
- Posters, playground monitors
- Parent engagement
• Targeted element:
- Structured process to deal with bullying incidents

13. Hard-to-reach families, or hard-to-access services?

14. What we learnt (and maybe should have known before…)
1. If we don’t engage the right parents, the programme won’t work
2. Engage providers first if you want them to engage parents
3. Have a clear recruitment process and train everyone involved
4. Invest in and incentivise recruitment and retention
5. Get out there! Go to parents; don’t expect them to come to you
6. Build relationships: visit, call, then visit and call again
7. Be practical: make it attractive and easy to come along
8. Be creative!
9. Recruitment is nothing without retention
10. Aim high but be realistic: life gets in the way of the best intentions

15. What’s up with evidence-based programmes?
1. Limited impact: no “breakthrough outcomes”, poor transportability
2. “Pseudo-science”: cherry-picking, developer bias
3. RCTs are not the gold standard
4. Fiddling trumps fidelity
5. Poor fit with systems (therefore not “scalable”)
6. Automatons replacing autonomy
7. Limit innovation
8. “Privatised” solutions to “private” problems
9. Technical not relational
10. ‘Real’ children and ‘My children’

16. Poor tansportability?
Programme Name Effects in the US Number of studies in the US Effects in Europe Number of studies in Europe
Big Brothers Big Sister Positive effect 4 No effect 1
Functional Family Therapy Positive effect 8 Positive effect 3
Good Behaviour Game Positive effect 5 Positive Effect 4
Incredible Years Parent Training Programme Positive effect 26 Positive effect 18
Incredible Years Child Training Programme Positive effect 5 No effect 3
Multisystemic Therapy Positive effect 12 Mixed effects 5
Nurse Family Partnership Positive effect 3 Mixed effects 4
Promoting Alternative Thinking Strategies (PATHS) Positive effect 8 Mixed effects 6
Multidimensional Treatment Foster Care (Oregon) Positive effect 8 Mixed effects 4
Strengthening Families 10-14 Positive effect 2 No effect 3
Programme Effect Size in the US Effect Size in Europe Countries Included
Functional Family Therapy -0.09 to -0.59 -0.96 Sweden
Good Behaviour Game -0.37 -0.35 The Netherlands
Incredible Years Parent Training Programme -0.02 to -1.18 -0.10 to -0.72 UK, Norway
Multisystemic Therapy -0.13 to -1.74 --0.40 UK
Multidimensional Treatment Foster Care (Oregon) -0.49 to -1.6 -0.92
UK (In Sweden, the effect on internalizing (ES = -0.39) and
externalizing (ES = -0.58) behaviour was measured.)
*Effect sizes based on effects on primary outcomes.
Some effect sizes have been calculated by the Washington State Institute of Public Policy

17. Developer bias?
Programme
Findings in developer-led
studies
Findings in independent
evaluations
Reconnecting Youth (Drug
prevention programme)
Increased GPA; increased self-
esteem; increased school
bonding; decreased hard drug
use; and decreased drug control
problems (Eggert et al. 1994)
Negative effects on most
outcome measures, no positive
effects. Negative effects the
stronger the better
implementation fidelity
(Sanchez et al. 2007)
Triple P Positive Parenting
Programme
Positive mean effect on child
problem behavior of d=0.35 in
33 trials (Nowak and Heinrichs
2008)
No positive effects on any
aspect of problem behavior
evaluated by teachers, parents,
or child self-reports (Eisner et al.
2007)
Olweus Bullying Prevention
Programme
Reductions of up to 50% in
bullying in the original study
(Olweus 1994)
No overall effects on either
attitudinal measures or
victimization (Bauer et al. 2007)
ALERT (Drug prevention
programme)
Reduction in cigarette,
marijuana and alcohol use by
19–39% (Ellickson et al. 2003)
No effects on mediators or
substance abuse itself (St Pierre
et al. 2006)
Source: Eisner, M. (2009). No effects in independent prevention trials: can we reject the cynical
view?. Journal of Experimental Criminology, 5(2), 163-183.

18. Towards EBP 2.0?
1. Do the same but better (e.g. progressive standards, implementation)
2. Re-invent EBPs (e.g. build on neuroscience, build in flex)
3. Develop and apply evidence-based “kernels” and “bundles”
4. Make “services as usual” more evidence-based (e.g. Lipsey, PDSA, rct)
5. Engage multiple systems (e.g. HPS, “collective impact”)
6. Explore common logic models / meta-theory
7. Segment, and then target better
8. Improve prevention training for practitioners
9. Empower civil society (e.g. community engagement, kernels)
10. Reform systems (e.g. de-commission, earmarking)

19. Evidence-based kernels
A “fundamental unit of behavioural influence”
that underlie effective prevention and treatment
and a small and simple entity that holds the
potential to transform into something much
larger.
They must:
• be inexpensive;
• show an immediate effect;
• be easily useable; and
• be flexible to simultaneously solving additional
problems that might arise in the course of
prevention or treatment
• be empirically found to be effective
• be indivisible (i.e. it would not work if altered).

20. Selected sources
Axford, N., Lehtonen, M., Tobin, K., Kaoukji, D. & Berry, V. (2012) ‘Engaging parents in parenting programs: lessons from research and
practice’, Children and Youth Services Review, 34 (10), 2061-2071.
Axford, N. and Morpeth, L. (2013) ‘Evidence-based programs in children’s services: a critical appraisal’, Children and Youth Services Review 35
(1), 268-277.
Davies, F. A., McDonald, L. & Axford, N. (2012) Technique is Not Enough: Making Evidence-based Programmes Socially Inclusive. Discussion
Paper for the British Psychological Society Professional Practice Board’s Social Inclusion Group. Leicester, BPS.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: a review of research on the influence of implementation on program outcomes
and the factors affecting implementation. American Journal of Community Psychology, 41, 327-350.
Eisner, M. (2009). No effects in independent prevention trials: can we reject the cynical view? Journal of Experimental Criminology, 5(2), 163-
183.
Embry, D. D., & Biglan, A. (2008). Evidence-based kernels: fundamental units of behavioral influence. Clinical Child and Family Psychology
Review, 11(3), 75-113.
Gottfredson, D. C., Cook, T. D., Gardner, F. E. M., Gorman-Smith, D., Howe, G. W., Sndler, I. W. & Zafft, K. M. (2015) Standards of evidence for
efficacy, effectiveness, and scale-up research in prevention science: next generation. Prevention Science 16 (7) 893-926.

21. Selected sources
Hanleybrown, F., Kania, J. and Kramer, M. (2012) Channelling Change: Making Collective Impact Work. Stanford Social Innovation Review.
http://ssir.org/articles/entry/channeling_change_making_collective_impact_work [Accessed 8th February 2016]
Kemp, F., Ohlson, C., Raja, A., Morpeth, L. & Axford, N. (2015) Fund-mapping: The Investment of Public Resources in the Wellbeing of Children
and Young People in Northern Ireland. Belfast: NICCY
Langford, R., Bonell, C.P., Jones, H.E., Pouliou, T., Murphy, S.M., Waters, E., Komro, K.A., Gibbs, L.F., Magnus, D., and Campbell, R. (2014), The
WHO Health Promoting School framework for improving the health and well-being of students and their academic achievement, Cochrane
Database of Systematic Reviews 2014, Issue 4.
Lipsey, M., Howell, J. C., Kelly, M. R., Chapman, G., & Carver, D. (2010). Improving the effective- ness of juvenile justice programs: A new
perspective on evidence-based practice. Center for Juvenile Justice Reform: Georgetown Public Policy Institute, Georgetown University.
O’Connell, M. E., Boat, T. and Warner, K. E. (Eds.) (2009) Preventing Mental, Emotional, and Behavioural Disorders Among Young People:
Progress and Possibilities. Washington DC: The National Academies Press.
Santucci, L. C., Thomassin, K., Petrovic, L. & Weisz, J. R. (2015) Building evidence-based interventions for the youth, providers, and contexts of
real-world mental-health care. Child Development Perspectives, 9 (2), 67-73.
Shonkoff, J. P. and Fisher, P. A. (2013) Rethinking evidence-based practice and two-generation programs to create the future of early
childhood policy. Development and Psychopathology, 25, 1635-1653.

22. Contact details
nick.axford@dartington.org.uk
www.dartington.org.uk
@nick_axford