This document summarizes the development and evaluation of the PCYC Catalyst Outdoor Adventure Youth Intervention Program. The program aims to provide a positive intervention for at-risk youth ages 13-16 using a 15-day outdoor adventure program. Research evaluated the program's effects on life effectiveness, mental health, and behavioral conduct. Results found moderate improvements in these areas from pre- to post-program and at 6-12 month follow up. Interviews revealed youth found the program challenging but fostered personal and social development. Recommendations focused on continued evaluation and ensuring program integrity.

1. Development and Evaluation
of the PCYC Catalyst
Outdoor Adventure
Youth Intervention Program
Arron Sullivan
PCYC Bornhoffen
James Neill
University of Canberra
NationalOutdoorEducation
Conference2016
19th National Outdoor Education Conference,
University of the Sunshine Coast, Sippy
Downs, Queensland, Australia, Wednesday
30 March, 2016

2. NationalOutdoorEducation
Conference2016

3. Project plan – 2 components
Research and
Evaluation
Program Delivery
• Work with 10 YP at risk
• 15 day program duration
• Partner with 6 education
and / youth centres
• Work regionally and
locally
• Collaborative practice
• Evidenced-based
practice
NationalOutdoorEducation
Conference2016

4. Key partners
NationalOutdoorEducation
Conference2016

5. Levels of at-riskness (Greenaway,1995:3)
1. Temporary - Commit minor crimes in the company of others
2. Difficult and disturbed - Temporary involvement
with crime, but whose offending is linked with wider problems like
home conflicts, school based difficulties
3. Persistent offenders – Often of low intelligence and
troublesome in school, with parents who exercise poor supervision
and may themselves be involved in crime
4. One-off serious offenders – Crimes are rare and
isolated, unexpected and not explained by social factors or
environment. For these offenders psycho-therapeutic or behaviour
modifying treatment in a secure setting is usually effective
5. Persistent and serious offenders - Offenders
from category three and four for whom effective intervention is
particularly difficult as there is a high likelihood of re-offending.
NationalOutdoorEducation
Conference2016

6. Purpose and intent
Project Purpose
To provide a positive intervention that facilitates change for young
people and their families using an interagency collaborative strategy
Program Purpose
Working with young people (aged 13 to 16 years) who are at risk of
adverse outcomes in their educational, vocational, and life-course
pathways.
Intent
To help young people to make positive life choices, experience a
meaningful life, make a positive contribution to their community, and
assist in the transition into young adulthood
NationalOutdoorEducation
Conference2016

7. NationalOutdoorEducation
Conference2016
Purpose and intent

8. Project outcomes
1. Encouraging positive change in each young person by
recognising and celebrating the observable outcomes
2. Partnering with Learning and Education centres that
already have a behavioural management or learning
engagement process in place
3. Engaging parents and other community
organisations/members to support the change process
for young people
4. Developing protective factors, positive self awareness
and resilience in young people
5. Leading an interagency collaborative strategy for the
development of community
6. Evaluating the short and long term outcomes of the
project to develop its effectiveness
NationalOutdoorEducation
Conference2016

9. Program specifics
1. Encouraging positive change in each young
person by recognising and celebrating the
observable outcomes
• By helping young people to reflect positively on
experience and actions
• Recognising and building on personal strengths
• Managing personal behaviour and asking for help
reflecting on areas for improvement in behaviour
• Encouraging participants and parents or guardians
to LEAD themselves first:-
Learn to reflect positively on experience
·Engage and manage their emotions
Adapt their thinking attitudes and
behaviour to their situations and
challenges
· Demonstrate a willingness to:
- work on yourself
- participate positively in a group
- contribute positively to the community
• Learning to set realistic and achievable personal
goals
• Working as part of a team and taking personal
responsibility for the outcome
• Raising awareness of all aspect of life resulting in a
positive and meaningful contribution to
community
NationalOutdoorEducation
Conference2016

10. Program model
Staff model
Leadership development
-Two facilitators
-Two teachers/school
councillors/youth worker
10 week program cycle
• Partner selection
• Participant selection
• Parent commitment
• 3-day Lead In
• 9-day Expedition
• 3-day Follow up
• Participant mentoring
NationalOutdoorEducation
Conference2016

11. Program delivery partners
PCYC branches
• Dalby
• Mt Isa
• Goondiwindi
• Zillmere
Qld Department of
Education State
High Schools
• Woodridge
• Brackenridge
• Helensvale
• Goondiwindi
• Dalby State High School
• Spinifex College (Mt Isa)
NationalOutdoorEducation
Conference2016

12. Background
• 2004 First Catalyst Program delivered
• 2008 International Consortium for Experiential
Learning
• 2009 Funding included seed funding for research
• 2011 produced our own in-house evaluation
• By 2012 we had worked with 24 partners and
invested $300,000 in programs
• Maintained long-term partnership with strategic
view (2008 – 2013)
NationalOutdoorEducation
Conference2016

13. Questions about the program
model ?
NationalOutdoorEducation
Conference2016

14. Research evaluation design
NationalOutdoorEducation
Conference2016
• Mixed methods
• Quantitative (questionnaires)
• Life effectiveness
• Mental health
• Behavioural conduct
• Qualitative (interviews)
• Multiple perspectives
• Youth self-report
• Observer-report
• Longitudinal
• Pre (1st day)
• Post (last day)
• Follow-up (6 to 12 months)

15. Adventure therapy outcomes
NationalOutdoorEducation
Conference2016
• Bowen & Neill (2013) synthesised the
outcomes of 197 adventure therapy
studies.
• The overall standardised mean effect size
was 0.47, a moderate effect.
• More specific benchmarks were
suggested for adolescents (0.44) and
specific types of outcomes.
• Useful for benchmarking evaluation of
specific programs.

16. Results: Life effectiveness
NationalOutdoorEducation
Conference2016
• Adapted Youth at Risk Program Evaluation Tool
(YARPET; Neill, 2007) , 30 items measuring 10 factors:
Emotional Resilience
Goal Setting
Healthy Risk-taking
Locus of Control
Self-Awareness
Self-Esteem
Self-Confidence
Communication Skills
Community Engagement
Cooperative Teamwork
• Short-term change: 0.16, N = 38;
60% of participants reported increases
• Longer-term change: 0.29, N = 20;
62% of participants reported increases

17. Results: Life effectiveness
NationalOutdoorEducation
Conference2016
0.03
0.03
0.10
0.11
0.26
0.33
0.44
0.44
0.51
0.72
Goal Setting
Community Engagement
Locus of Control
Self Awareness
Emotional Resilience
Cooperative Teamwork
Self Esteem
Self Confidence
Healthy Risk Taking
Communication Skills
Longer-term changes

18. Results: Mental health
NationalOutdoorEducation
Conference2016
• Adapted General Well-Being (GWB; Heubeck & Neill,
2000; Veit & Ware, 1983) : 10 items measuring 2 factors:
0.95
0.07
-0.1
-0.33
-0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2
Longer-term
Short-term
Distress Well-being
Distress
Well-being

19. Results: Behavioural conduct
NationalOutdoorEducation
Conference2016
• Adapted Adolescent Behavioural Conduct
(Mak, 1993): 8 behaviours over past 6 months
• Overall reduction (12.5 to 10.7 incidents/month):
0.38
0.17
0.14
0.04
-0.27
-0.34
-0.47
-0.68
Drug use
Cheating
Vehicles
Wagging
Vandalising
Stealing
Fighting
Harming

20. Results: Youth interviews
NationalOutdoorEducation
Conference2016
Youth interviews revealed common themes that:
• youths came from, but were seeking to overcome,
challenging backgrounds (personal, family, social)
• faced challenges during the program (physical, social,
existential)
• which fostered
• personal and social development
• motivation to work for change
• a more optimistic outlook on the future

21. Program recommendations
NationalOutdoorEducation
Conference2016
1. Adventure therapy vs. therapeutic
adventure
2. Screening/Profiling/Selection
3. Physical challenge of initial hiking
4. Group storming and norming
5. High adventure activities
6. Reflective activities

22. Key learnings
Project & program delivery
1. Multiple partners –
multiple masters
2. Program integrity
3. Evolving on the run
4. Staff turnover
Research evaluation
1. Program objectives
2. Research vs.
evaluation vs.
action-research
3. External vs.
internal
NationalOutdoorEducation
Conference2016

23. Contacts for further info
James Neill
Assistant Professor
Centre for Applied Psychology
Faculty of Health
Mob 0432 925 211
james.neill@canberra.edu.au
NationalOutdoorEducation
Conference2016
Arron Sullivan
Co Manager PCYC Bornhoffen
Leadership Development
Manager
Mob 0407 148 123
arron.sullivan@pcyc.org.au
www.bornhoffenpcyc.org.au

24. References
• Bowen, D. J. & Neill, J. T. (2013). A meta-analysis of adventure
therapy outcomes and moderators. The Open Psychology Journal,
6, 28-53. doi: 10.2174/1874350120130802001
• Bowen, D. J. & Neill, J. T. (2016). Effects of the PCYC Catalyst
outdoor adventure intervention program on youths' life skills,
mental health, and delinquent behaviour. International Journal of
Adolescence and Youth, 21(1). doi:
10.1080/02673843.2015.1027716.
• Neill, J. T. & Bowen, D. J. (2014). Research evaluation of PCYC
Bornhoffen Catalyst intervention programs for youth-at-risk [2012-
2013]. Canberra, Australia: University of Canberra.
• PCYC Bornhoffen Adventure Development (2010). Catalyst Project
2009-2010 Leading change: Literature review. Retrieved from
http://www.bornhoffenpcyc.org.au/CATALYST%20LIT%20REVIEW%
202010.pdf

25. References
• Neill, J. T. (2007). A measurement tool for assessing the effects of
adventure-based programs on outcomes for youth-at-risk
participants. Retrieved from
http://wilderdom.com/tools/leq/YouthDevelopmentLEQScalesPape
r.htm