Dr. Margaret Handley (UCSF) provides the learning goals for this webinar, which are the following: 1) Understand Background ideas that informs the UCSF Implementation Science Training Program, 2) identify components of the conceptual model for Implementation science have been applied to course development, and 3) understand variations of learner experience, ranging from curriculum and examples of completed work. For more information and to see other dissemination and implementation content, please visit: http://ctsi.ucla.edu/patients-community/pages/dissemination_implementation_improvement

1. UCSF Training Program in Implementation
Science (ImS)
Activities and Impacts
Margaret Handley PhD MPH
Professor, Dept. Epidemiology and Biostatistics and
Department of Medicine, University of California San Francisco
Center for Vulnerable Populations, ZSFGH
Co-Director UCSF Training Program in Implementation Science

2. Learning Objectives
1. Understand Background ideas that informs the UCSF
Implementation Science Training Program
2. Identify components of the conceptual model for ImS have
been applied to course development
3. Understand variations of learner experience, ranging from
curriculum and examples of completed work

3. I. Background

4. What is implementation science?
Implementation research is the scientific
study of methods to promote the
systematic uptake of research findings
and other evidence-based practices into
routine practice, and hence, to improve
the quality and effectiveness of health
services and care.
-Eccles MP, Mittman BS. Implementation Science 2006

5. What is the ‘science’ part?
Implementation science research is the scientific study of
methods to promote the systematic uptake of research
findings and other evidence-based practices into routine
practice, and, hence, to improve the quality and
effectiveness of health services and care.
Eccles MP, Mittman BS. Implementation Science 2006

6. Implementation Challenge 1:
Often, we don’t know what to do with our ‘evidence’

7. …and we realize there are persistent Evidence Gaps
“Many evidence-based
innovations fail to produce
results when transferred to
communities in the global south,
largely because their
implementation is untested,
unsuitable or incomplete”
“Scientists have been slow to
view implementation as a
dynamic, adaptive, multi-scale
phenomenon that can be
addressed through a research
agenda”
T. Madon et al .2007

8. “We had this great tool here called
the grinder, which we knew could
grind peanuts into peanut butter very,
very well. But when we got over to
those countries, we found out they
didn’t make a lot of peanut butter.”
“What they needed was a peanut
sheller.”
http://www.pbs.org/newshour/bb/inventing-new-efficient-kitchen-tools-developing-world/
Implementation Challenge 2:
What constitutes the ‘best’ research evidence?
???

9. “We had this great tool here called
the grinder, which we knew could
grind peanuts into peanut butter very,
very well. But when we got over to
those countries, we found out they
didn’t make a lot of peanut butter.”
“What they needed was a peanut
sheller.”
http://www.pbs.org/newshour/bb/inventing-new-efficient-kitchen-tools-developing-world/
Implementation Challenge 2:
What constitutes the ‘best’ research evidence?
? What about evidence
related to local context
and knowledge?
???

10. A goal of Implementation Science (ImS) is to ensure that
the knowledge and materials produced by health
research actually lead to improved population health by:
1) Reaching the people for whom they are intended
2) Being implemented safely, efficiently, equitably,
and in a patient-centered manner (IOM)
3) Being co-developed and co-implemented through
partnerships that enhance relevance and sustainability
Goals of Implementation-Informed Research in
Community Settings

11. Types of implementation science research
• Identify barriers and facilitators to behavior change through
community engagement with stakeholders
• Tailor outreach efforts to encourage use of effective health services or
reduce access to unhealthy products
• Identify strategies to promote the integration of evidence into policy
and program decisions (study the process and outcomes)
• Adapt interventions according to population and setting
• Design an impact evaluation for a population-based intervention
• Evaluate different implementation strategies to improve clinician
screening behaviors, targeting across staffing groups and leadership

12. SOCIAL
SCIENCES
Anthropology
Economics
Psychology
Public Policy
Sociology
Social Work
Education
Who does implementation science? Just a start!
POPULATION
SCIENCES
Biostatistics
Epidemiology
Health Policy
ENGINEERING
SCIENCES
Computer Science
Logistics
Operations
Management
Implementation
Science
CLINICAL
SCIENCES
Dentistry
Medicine
Nursing
Pharmacy

13. II. Conceptual Framework for How
“New Evidence” is Translated to
Improve Health:
UCSF Program in ImS

14. Principle 1: Engagement with stakeholders is
essential at all stages
Principle 2: Behavior change is inherent to
translation of evidence into practice and policy
Principle 3: The process of behavior change is
iterative, favoring cycles and bidirectional
relationships, and a socio-ecological approach.
Key Principles: UCSF ImS Program
Gonzales, Handley, Ackerman, O-Sullivan 2012.
Handley, Gorukanti, Cattamanchi 2016.

15. Individual or a Socio-Ecological Perspective?

16. PRACTICE
HEALTH
A Socio-Ecological View of Improving Practice
EVIDENCE
• Government
Stakeholders • Payors/Insure
rs
• Societies
Delivery • Hospitals
Systems • Clinic/Practice
s
• Health Depts
• Providers
Individuals • Patients
• Public

17. Key elements: UCSF ImS Training Programs
Systematically tackles real-world knowledge-practice gaps
Seeks to use and create generalizable knowledge
Requires interdisciplinary collaborations and methods
Engages stakeholders at multiple levels
Uses theory to explain mechanisms & design interventions
Includes rigorous multi-level evaluation

18. • Epi 245. Introduction to Implementation Science Theory and Design
• Epi 246. Designing Individual-Level Implementation Strategies
• Epi 247. Designing Organizational-Level Implementation Strategies
• Epi 248. Community-Engaged Research
• Epi 249. Translating Evidence into Policy
• Epi 252. Program Evaluation in Clinical/ Public Health Settings
UCSF Program in Implementation
Science
Core Courses

19. UCSF Program in Implementation
Science
Core Courses
Epi 245. Introduction to Translating Evidence into Practice: E.
Geng Course Director
http://ticr.ucsf.edu/courses/schedule/translating.html
Epi 246. Designing Individual-Level Implementation Strategies:
M Handley Course Director
http://ticr.ucsf.edu/courses/schedule/translatingii.html

20. UCSF Program in Implementation
Science
Core Courses
Epi 247. Designing Organizational-Level Implementation Strategies:
L. Schmidt Course Director
http://ticr.ucsf.edu/courses/schedule/translatingiii.html
Epi 248. Community-Engaged Research: K. Grumbach Course Dir.
http://ticr.ucsf.edu/courses/schedule/translating_practice.html

21. UCSF Program in Implementation
Science
Core Courses
Epi 249. Translating Evidence into Policy
http://ticr.ucsf.edu/courses/schedule/translating_policy.html
B. Hollister Course Director
Epi 252. Program Evaluation in Clinical/ Public Health Settings
J. Meyers Course Director
http://ticr.ucsf.edu/courses/schedule/program_evaluation.html

22. 1. Design and Implement More Effective Interventions
to translate evidence into practice.
This will be achieved by identifying community, patient,
clinician and organizational factors that serve as barriers
and facilitators to translating research results into
everyday practice, policy and public health.
Learning Objectives for Intensive Course
Completion
Upon completion of the program, scholars
will be able to:

23. 2. Design More Comprehensive Evaluations of
Interventions that translate evidence into practice.
This will be achieved by applying the basics of process and
outcome evaluation, and by identifying the appropriate
qualitative and quantitative measures of effect.
Upon completion of the program, scholars
will be able to:

24. 3. Develop Better Grant Proposals.
This will be achieved by making a compelling case for
translational research and integrating conceptual
frameworks into the intervention design and/or evaluation
of your program.
Upon completion of the program, scholars
will be able to:

25. PRACTICE
HEALTH
An Ecological View of Improving
Practice: Intervention Design and EvaluationEVIDENCE
Epi 246:
Designing
Individual-Centered
Implementation
Strategies
Ex. Design a community education campaign
to promote colon cancer screening among
adults age > 50 years.
• Providers
Individuals • Patients
• Public
Epi 252:
Evaluation Strategies

26. PRACTICE
HEALTH
An Ecological View of Improving
Practice: Intervention Design and EvaluationEVIDENCE
Epi 247:
Designing System
Implementation
Strategies
Ex. Promote public report cards of health plans
for cancer screening rates
Delivery • Hospitals
• Clinic/Practices
Systems • Health Depts
• Providers
Individuals • Patients
• Public
Epi 252:
Evaluation Strategies

27. PRACTICE
HEALTH
An Ecological View of Improving
Practice: Community and PolicyEVIDENCE
Epi 249:
Advocacy &
Policy Strategies
Epi 248:
Community-Engaged
Strategies
Ex. Community Advisors co-create intervention with research
team and also embed structural targets community has prioritized
(e.g. social marketing to venues that promote teen smoking)
• Government
Stakeholders • Payors/Insurers
• Societies
Delivery • Hospitals
Systems • Clinic/Practices
• Health Depts
• Providers
Individuals • Patients
• Public
Ex. Create legislation to make cancer screening free
for Medicare beneficiaries

28. Available formats for the UCSF Training Programs
UCSF-Based Training- In-Person Options
(credit towards degrees available)
N Classes
Offered
N people
Completed to
date (since 2010)
1) ImS Track Within Existing Master’s Program
Training in Clinical Research (TICR)
6* 31
2) Certificate Program in Implementation Science 6 39
3) Electives within other programs in Department
of Epidemiology and Biostatistics or as elective
w/o program affiliation
6 156
Target Audience:
Post-doctoral and junior faculty clinicians in research fellowships, public health/public
policy post-doctoral trainees, senior faculty, doctoral students, practitioners and quality
improvement officers or other staff engaged in the development, implementation or
evaluation of strategies to promote health and health care quality.
* A series of ImS-specific courses, complemented by a series of related methods
courses from several departments.

29. • Health Disparities Research Methods
• Clinical Performance and Health Outcomes Measurement
• Qualitative Research Methods
• Decision and Cost Effectiveness Analysis
• Medical Informatics
Examples of ImS Track Electives

30. UCSF Training Programs in ImS
UCSF-Based Training – Other Formats
(not credit bearing)
Format N people to date
(Fall,2016)
1) On-Line: Certificate Program in
Implementation Science or a la carte
( on-line courses also offered individually)
6 classes on line,
two per quarter
Starting this Fall
!
2) Summer Training Institute: RISE (Research
in Implementation Science for Equity)*
2 week intensive/+
small groups/WIP
22
3) International ImS Training Institutes** 1 week
intensive/+small
groups
60
*MPI: Bibbins-Domingo, Fernandez, Handley (09/15/2014 – 05/31/2018 ). The UCSF Research in
Implementation Science for Equity (RISE) two-week program is part of NIH-NHLBI Program to Increase
Diversity among Individuals Engaged in Health-Related Research (PRIDE) and recruits junior faculty. The
UCSF RISE program consists of a Summer Institute on both Implementation Science (ImS) and Careers-
In-Progress, distance mentoring and works in progress, and return immersion/sharing with new group.
**Fogerty Funded Programs in Uganda (Davis, Reingold, Cattamanchi, Kamya), and others with WHO
collaborators in Region 9 (E. Europe/N. Africa) involving modified versions of the 2 week programs
(Rutherford, Handley, Bozicevic).

31. UCSF On-LineProgram in Implementation
and Dissemination Science
Fall 2016 Winter 2017 Spring 2017
Translating Evidence
into Practice
Organization-
Centered
Implementation
Strategies
Translating Evidence
into Policy
: Individual-
Centered
Implementation
Strategies
Program
Evaluation
Community-Engagement
Overview Six online courses (2 each in Fall, Winter and Spring) designed to train
health professionals to 1) design, adapt, and evaluate more effective strategies to
facilitate implementation of health interventions; and 2) produce high-quality funding
proposals. All six courses are required for the Certificate Program.

32. Examples of research from UCSF ImS
Controlled Intervention trials
• Can community-based TB screening vans increase TB case finding?
(Davis)
• Does adapted Diabetes Prevention Program (DPP) reduce diabetes risk
factors in high risk pops? (Handley)
Implementation trials
• What strategies are most effective in linking TB patients to care and
retaining them to the end of treatment? (Cattamanchi)
• Which medical home referral program best engages psychiatrists in
managing cardiometabolic outcomes in severely mentally ill pts?
(Mangurian)
• What is the effectiveness of referral to home- vs. center- based cardiac
rehab after hospitalization for myocardial infarction, coronary bypass
surgery, or percutaneous revascularization? (Whooley)

33. Alumni Survey- 2016
For those graduating 2010-2016
• Training Survey Focused on ImS-related Productivity and Current Positions
• Competency Survey Focused on ImS-related Competencies
Surveys targeted alumni of the two more intensive programs – the ImS track in Masters
Degree in Clinical Research and ImS Certificate Program (completed 4-6 ImS intensive
courses)
• Surveys sent via SurveyMonkey in Winter-Spring 2015-2016
• Surveys were not not anonymous but were de-identified

34. Training Alumni Survey Results (preliminary)
Overall
Participation
Rate
54/70 (77%).
Not different
for the two
groups

35. Alumni Survey Results (preliminary)

36. Alumni Survey Results (preliminary)

37. 1. See us on-line
https://accelerate.ucsf.edu/training/implementation-science-certificate
http://ticr.ucsf.edu/courses/implementation_research.html
2. Contact us!
Margaret Handley, PhD MPH – ImS Program Co- Director
margaret.handley@ucsf.edu
Adithya Cattamanchi, MD MAS – ImS Program Co- Director
adithya.cattamanchi@ucsf.edu
For More Information