IPSSW Pre-Conference Workshop 7 (2016, Glasgow)

Simulation Research:
Improving Rigor, Impact, and
Methodology
May 8, 2016
Glasgow, UK
IPSSW Pre-Conference Workshop

International Network for Simulation-based Pediatric Innovation, Research and Education
Introduction / Faculty
• Ralph MacKinnon
• Todd Chang
• Vinay Nadkarni
• David Kessler

Disclosures
• No relevant financial conflicts of interest
• INSPIRE receives funding/support from

Mission
We aim to improve the delivery of medical care to acutely ill children
by answering important research questions pertaining to resuscitation,
technical skills, behavioral skills, debriefing and simulation-based
education

Who are you?
• Where do you work with Simulation?
• What is your simulation experience?
• What is your research experience?

Learning Objectives
• Define the 2 types of simulation-based research
• Evaluate characteristics of a simulation-based research
question or protocol that are conducive to quality
research and output
• Identify 6 specific challenges in simulation-based
research that threaten rigor and quality of research
output

Objectives
1. Develop a study design using simulation as the
subject of research
2. Develop a study design using simulation as an
investigative methodology
3. Use Reporting Guidelines to improve study
methodology

Schedule
Didactic Discussion Small group work Break
Closing Discussion & Networking

2 Types of Simulation Research
Research subject
(e.g. Team training)
Research method
(e.g. Test new defibrillator)

1: Simulation as the research subject
• Is simulation an effective training methodology?
• How do we maximize effects on patient outcomes?
2: Simulation as the research method
• Leverages standardization of simulation
• Questions that might not be safe, feasible, ethical, timely in traditional
research

Simulation as the subject of
research

INSPIRE Research Themes
Simulation as the subject
Debriefing
Develop/assess/implement effective techniques for debriefing
real/sim events
IPE, Teamwork,
Communication
Develop/assess/implement effective techniques for team
training
Procedural,
Psychomotor Skills
Develop/assess/implement effective techniques for skills
development retention

• Research examining whether or not specific
features of simulation experiences are effective

#1 start with strong conceptual model

What we know…
• 609 studies
• 23% RCT’s
• 12% multicenter studies
• 5% reported patient and/or
healthcare outcomes
• Sim is an effective educational
intervention compared to no
intervention

Pediatric Research
• 57 studies
• 1/3 RCT
• compare sim to no
intervention
• Large pooled effective size
• Knowledge
• Skills in simulation
• Time to completion
• < 10% on patient outcome

2 Lessons
1. Avoid the null comparison
• (Don’t compare to no intervention)

Varying Instructional Design
1. Feedback ***
2. Repetitive Practice ***
3. Distributed Practice
4. Curriculum Integration
5. Clinical variation
6. Range of difficulty
7. Individualized learning
8. Multiple learning strategies
9. Defined Outcomes
10. Valid simulator

What examples of Instructional
Design Features have you
investigated?
Discussion 1

2 Lessons
2. Measure patient outcomes
• (Or at least provider behavior)

Knowledge/skills
Attitudes/perceptions
Learners’ satisfaction/reaction
Yadley 2011, BEME 2005, Miller 1990, Kirkpatrick 1967

Δ practice
Δ behavior
Knowledge/skills

Popul-
ation
Patient
Δ practice
Δ behavior
Knowledge/skills

Framing your Research Outcomes
• Goal of medical education
• To transmit knowledge, impart skills, and inculcate
the values of medicine
that will improve health outcomes
Cooke, Irby, O’Brien. Carnegie Foundation Report: Educating Physicians: A call for reform of
medical school and residency. 2010

Selecting outcomes
• Important to patient
• Measurable
• Dependent on provider skills/behaviors
• Quality
• Satisfaction, comfort
• Procedure related **
• Success/failure
• Complications (Erb’s Palsy, Line Infections)
• Survival
• Cardiac resuscitation, obstetrics, NICU

INSPIRE Examples

Infant Lumbar Punctures
• Essential pediatric procedure
• Decreasing experience
• Novice success rate 35%
• Bench top simulator
• Low cost
• Easy to use
Auerbach et al Pediatric Emergency Care 2013, Gaies Pediatrics 2007

Infant Lumbar Punctures
• Instructional Design principles:
• Distributed Practice (JITT)
• Repeated Practice
• Defined Outcomes
• Outcomes selected:
• Clinical Lumbar Puncture success

Research Question
• Population
• Intervention
• Comparison
• Outcome

Research Question
• P – 1st year residents (‘interns’)
• I – Mastery Learning LP Training + Just-in-Time
Training on LP Simulator just prior to clinical LP
• C – Mastery Learning LP Training only
• O – Clinical LP success rate

Auerbach et al, Pediatric Emergency Care 2013
Overall Performance Score

Let’s design a Research Project:
Small Group 1

Simulation as an investigative
methodology

INSPIRE Research Themes
Simulation as the method
Technology
Develop/assess/implement novel technologies designed to
improve processes of care and pediatric patient outcomes
Acute Care and
Resuscitation
Develop/assess/implement novel techniques for improving care
of pediatric patients
Human Factors
Assess the role of human factors when providing care to
pediatric patients
Patient Safety
Explore the key variables that influence patient safety and
assess strategies to mitigate

• The simulated environment is used as an
experimental model to study factors affecting
human and systems performance in healthcare.

Why use simulated instead of real scenarios
• Standardization of “noise”
• Patient- age, clinical status, parent actions
• Individuals- experience, stress, fatigue
• Team- structure, numbers
• Environment- structure, distractions, noise
• Technology- equipment availability, size, safety
• Ask questions that might not be safe, feasible, ethical,
timely in traditional research

Threats to internal validity

Threats to validity
• Educator training
• Simulator Selection
• Scenario Design
• Confederates
• Realism (physical, conceptual and emotional)
• Debriefing
• Video Capture and Review
• Study Outcomes
Cheng et al, Pediatrics, 2014

• ADD SLIDES ON EDUCATIONAL DESIGNS- e.g.
solomon 4, cluster randomized etc…(martin’s
paper, selin workshop)
• Ethics- rct not always possible. (perfume
example)

What examples of Threats to
Validity have you encountered?
Discussion 2

Technology
• A QCPR simulator was
used to show that
Feedback Devices
improved CPR
performance

Systems/Human Factors
• Individuals
• Teams
• Environments
• Technological Factors
• Systems Factors
• Patient Factors

SEIPS 2.0. Ergonomics . 2013 November ; 56(11) Holden, Carayon et al

New Environments

Safety / Quality
• In-situ simulation
• Mobile unit brought into work environment
• Use real life equipment in “context”
• Work with real life team
• Identify latent safety threats
• Provider level - insufficient staffing/training
• Systems level - missing/broken equipment

Direct comparison: same patient at different
hospital

Crash Testing, not Normal Testing
• Identify and remediate threats to safety
• Active
• Latent: unrecognized errors in design or organization

Let’s design a Research Project:
Small Group 2

Reporting Guidelines
• Standardized reporting will enhance quality of studies and assist journal
reviewer/editors
• Explanation and elaboration document will support implementation of
reporting guidelines
• Reporting guidelines currently under review
• Eventual adoption for author guidelines

Reporting Guidelines
• Extensions to prospective trial design:
• CONSORT (randomized-control)
• STROBE (observation cohort)

How do we use Reporting Guidelines?

Let’s review our Research
Project: Using Reporting
Guidelines
Small Group 3

What changes will you be making?

Questions?

www.inspireSim.com

IPSSW Pre-Conference Workshop 7 (2016, Glasgow)

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (7)

Destaque

Destaque (20)

Semelhante a IPSSW Pre-Conference Workshop 7 (2016, Glasgow)

Semelhante a IPSSW Pre-Conference Workshop 7 (2016, Glasgow) (20)

Mais de INSPIRE_Network

Mais de INSPIRE_Network (20)

Último

Último (20)

IPSSW Pre-Conference Workshop 7 (2016, Glasgow)