Teen Counselling 2013 Conference

1. Designing and Implementing
Effective Services
Crosscare Teen Counselling Conference
Nuala Doherty - Director, Centre for Effective Services
Sarah Rochford - Intern
27th September 2013
© The Centre for Effective Services 2013

2. Overview
1. Designing effective
services
2. Implementing
effective services
What is evidence?
The Implementation Gap
Sourcing and Using evidence:
Challenges and Benefits
Implementation Drivers
Putting Evidence in Context
Outcomes
Logic Models
How CES Supports evidence
into practice

3. Introduction to CES
A small organisation
with charity status
Established in 2008
(5 year grant)
A multi-disciplinary
staff team
Jointly funded by
All Island organisation
Government
Belfast
• Dept. Environment,
Community and
Local Government
• Dept. Children and
Youth Affairs
Dublin
Atlantic Philanthropies
3

4. CES Mission
Better Outcomes for Children, Young
People, Families and Communities
Effective
Efficient
Policy
Good Practice
Systems
Evidence to
Implementation
All-Island
Partnerships & Collaborations
The Centre for Effective Services works with others to connect and support the
implementation of effective policy, efficient systems and good practice, using the best
available evidence, so that children, families and communities thrive.

5. 1. What do we mean by Evidence?
Evidence-based
A programme, service or intervention that has consistently
been shown to produce positive results by independent
research studies that have been conducted to a particular
degree of scientific quality.
Evidence-informed
Practice based on the integration of experience, judgement and
expertise with the best available external evidence from
systematic research.
5

6. Types of evidence
Consultation
Research
Practice
Wisdom
Policy

7. What Works?
•
•
•
•
•
For whom
In what circumstances
At what point in time
Using which approach
Under what conditions

8. Developing a shared understanding of the evidence
journey
• Good Practice - A community of judgement of what works
• Promising Approaches - Some positive findings but findings are not
consistent or rigorous enough to be sure
• Research Based - Based on sound theory informed by a growing body
of empirical research
• Evidence Based - rigorously evaluated with consistent results
Evidence must be fit for purpose and take account of the complex nature
of service delivery and social needs. It helps us to assess and make
decisions on interventions with an ‘evidence lens’

9. What are the Benefits of Using Evidence?
It can
improve
outcomes for
children,
families and
communities
It is Ethical –
do no harm.
It can help us
to understand
how and why
it worked and
for whom, so
it we can
develop
service
delivery for
the future
It can
increase
chances of
future
funding &
sustainability
for
organisations
Relating
costs to
benefits can
inform the
debate
about policy
priorities
and
resource
allocation
Establishing
what is not
known helps
to identify
risks,
limitations
and grey
areas for
new
services.

10. What are the Challenges of Using Evidence?
• Be able to select, weigh and interpret evidence
• Contested definitions of evidence
• Limits to the evidence - Findings from different studies can
sometimes be contradictory
• Dearth of research in some areas and can be incomplete
• Take into account the complexity and systemic nature of social
problems and their underpinning issues
• The best evidence in the world will not result in better outcomes if
poorly implemented

11. Evidence Needs a Context to be Useful
•
Assess how ‘workable’ this approach will be for your area
•
Understand the qualitative local experiences, needs and strengths in your
area
•
Look carefully at the service delivery mechanism most likely to work for your
client group
•
Attempts to distil out the ‘active ingredients’ of effective programmes to
identify effective practice and mainstream.
•
Resources are important - think about sustainability from the start
•
This will let you make an informed choice

12. Outcomes
•
Outcomes are the changes for service users or other targets of change that happen as a result
of an intervention or service being provided.
•
There are levels of outcomes, i.e. individuals, families, specific groups, communities or
organisations themselves.
•
Outcomes can be conceptualised in various ways:
•
•
•
soft outcomes - changes in feelings, thinking and perceptions.
Hard outcomes - changes in behaviours, attainment or status which can be measured.
Short-term outcomes, medium or long-term.
•
Being outcomes as opposed to outputs focused allows for the development of indicators of
success and provides a common end for stakeholders.
•
It provides a more direct, focused approach to service provision.

13. Logic Models
A Logic model is key in the design phase in ensuring your
services or intervention is outcomes focused, and helps us
to be accountable.
It is a depiction of a programme/intervention showing
what it will do and what it is to accomplish.
A series of “if-then” relationships that, if implemented as
intended, lead to the desired outcomes.
Research shows logic models can improve
communication, clarify programme purpose, improve
service delivery, provide coherence across complex tasks
and diverse environments, and help us do better
evaluation (what variables to measure).

14. More specifically….
Monitoring and evaluation
INPUTS
Program
commitments
What we
invest
OUTPUTS
OUTCOMES
Activities
Participation
What
we do
Who we
reach
Short
Medium
Longterm
What results
Evidence
14
© Centre for Effective Services 2013

15. 2. Implementing Effective Services
The “Implementation Gap”
15

16. Understanding Implementation
At its simplest, implementation can be described as the carrying
out of a plan for doing something. It focuses on operationalising
the plan – the How , rather than the What
Implementation involves
— a specified set of purposeful activities
— at the practice, programme and systems levels
— putting into practice a programme, intervention or service with established
and recognisable processes that are necessary to achieve desired outcomes.
16

17. Bridging the Gap
Science to Service
Best Evidence
Gap
Service
IMPLEMENTATION
Source: Implementation Masterclass, Dublin, May 2011, K. Blase and D. Fixsen
17

18. What Works?
Implementing properly matters!
Fidelity is important.
Fidelity is the degree to which the activities undertaken in a
programme are true to the design of the original programme
on which it is based.
Higher Fidelity is correlated with better outcomes across a
wide range of programmes and practices

19. Implementation Drivers
Improved Fidelity and Outcomes
Staff capacity to
support clients
with the selected
practices
Organisational
capacity to support
staff in implementing
practices with fidelity
Leadership
Capacity to provide direction, vision, & the
“right” leadership approach
Source: Adapted from Implementation
Masterclass, Dublin, May 2011, K. Blase and D.
19
Fixsen

20. Implementation Drivers: Improved Fidelity &
Outcomes
Selection
Training
Coaching
Staff capacity to
support clients with the
selected practices
20

21. Implementation Drivers: Organisational support
Improved Fidelity and Outcomes
Systems Intervention
Facilitative Administration
Organisational
capacity to support
staff in implementing
practices with fidelity
Decision Support Data System
21

22. Implementation Drivers: Leadership
Improved Fidelity and Outcomes
• Adaptive challenges
cannot be solved by
existing systems.
• Adaptive leadership
involves finding
solutions that are new
and break away from
normal approaches.
This involves convening
staff, developing plans
& reaching a consensus.
Adaptive
Capacity to provide
direction, vision, & the
“right” leadership
approach
Leadership
• Technical challenges
are those which can be
solved by existing
organisational systems.
• Technical leadership
involves managing
these challenges by
knowing what needs to
be done and doing it.
Technical
22

23. Practitioners and Implementation
Need support
Need to feel
their work
and
contributions
are valued
Need to feel
involved and
that their
opinions are
heard
Need to be
empowered
to use their
skills and
wisdom

24. How CES Promotes Evidence in Practice
Prevention and
Early Intervention
Projects
Practice Seminars
e.g. Sourcing
Useable Evidence
Strategic Advice
and Support
The What Works
Process
CES Library
Online Resource
for Youth Workers

25. Evidence Route Map for Youth Work

26. To Conclude…..
•
In the Design phase, factors to consider are:
o The Evidence
o Local context and need
o Logic Models to focus outcomes and accompanying indicators
•
During Implementation, key ingredients are:
o Building staff competency
o Organisational Readiness
o Leadership to match the challenge and Championing of the new
approach

27. Evidence + Implementation = Better Outcomes for Children
Effective
Interventions
The “WHAT”
Designing
Effective
Implementation
The “HOW”
Implementing
Positive
Outcomes for
Children
Effective Services
Source: Implementation Masterclass, Dublin, May 2011, K. Blase and D. Fixsen

28. If you are interested in ‘evidence based/informed practice’
•
•
•
•
•
www.effectiveservices.org
www.iriss.org.uk
www.ceni.org
www.scie.org
www.cssp.org

29. Nuala Doherty
Director, Centre for Effective Services
ndoherty@effectiveservices.org
www.effectiveservices.org