Zabar final report cg

Picker Institute/ACGME Challenge Grants

Project Name:
Emergency Medicine Resident Training in Inter-professionalism Skills
Evaluating a Needs-Based Curriculum

FINAL REPORT
(February 29, 2007 – April 15, 2008)

Date of Report: April 15, 2008

Grant Number: 16

Grantee Institution: New York University School of Medicine

Principal Investigator Information: Sondra Zabar, MD
Associate Professor of Medicine
New York University School of Medicine
550 First Avenue, OBV D401
New York, NY 10016
(212) 263-1138
szabar@breitezabar.com

Co-Investigator Information: Linda Regan, MD
Assistant Professor of Emergency Medicine
New York University School of Medicine
lregan@jhmi.edu

TABLE OF CONTENTS

A. EXECUTIVE SUMMARY (ABSTRACT)........................................................................................................2
B. INTRODUCTION (BACKGROUND)............................................................................................................3
C. METHODS (PROJECT IMPLEMENTATION AND ADMINISTRATION) ...........................................4
D. RESULTS............................................................................................................................................................9
E. DISCUSSION...................................................................................................................................................14
F. DISSEMINATION ..........................................................................................................................................16
G. FINANCIAL REPORT ...................................................................................................................................16
H. ATTACHMENTS ............................................................................................................................................17
ATTACHMENT – SAMPLE CASE AND CHECKLIST (MEDICAL ERROR).........................................................................18
ATTACHMENT – SAMPLE REPORT CARD ..................................................................................................................28
ATTACHMENT – SESSION OBJECTIVES .....................................................................................................................34
ATTACHMENT – SAMPLE POCKET CARD ..................................................................................................................35
ATTACHMENT – GOLD FOUNDATION ABSTRACT .....................................................................................................36

A. EXECUTIVE SUMMARY (ABSTRACT)

Since the 1960’s, Emergency Medicine (EM) researchers’ efforts have worked to
demonstrate the importance of patient-centered doctor-patient communication, only acknowledging
decades later that advancing such patient-centered care will require increased and effective provider
education. Having had experience with the development and implementation of a controlled study
on the impact of comprehensive, integrated clinical communication skills curriculum on student
patient-centered skills, the Section of Primary Care faculty at New York University School of
Medicine’s were prepared and eager to partner with Emergency Medicine faculty on this very
important topic. With the commitment of NYUSOM-Bellevue Emergency Medicine Residency
leadership, we created the Emergency Medicine Professionalism and Communication Training
(EMPACT) Project.
EMPACT aimed to improve EM resident competency in communication and
professionalism through the development, implementation, and evaluation of new curriculum and
assessment measures. Our objectives were to: 1) design, implement and evaluate patient-centered
healthcare curriculum for all 60 EM residents; 2) evaluate predictive validity of Objective Structured
Clinical Examinations (OSCEs) by assessing correlation of OSCE performance with actual resident
performance in emergent care setting for cohort of PGY2 residents (n=15); and 3) disseminate this
Patient-Centered Care educational program to EM programs nationally. We conducted EMPACT in
four phases: Phase I) established baseline competency of EM interns using a 5 station OSCE; Phase
II) integrated an interactive skills-based series of five workshops focusing on interpersonal and
professionalism skills—into monthly required EM seminar series; Phase III) conducted post-
curriculum OSCE to evaluate impact of curriculum; and Phase IV) developed and implemented two
“Unannounced” Standardized Patient (USP) cases.
In completing all four phases of the EMPACT Project, we learned a lot about our residents,
how to improve our OSCEs, and how to implement another USP project in the future. Residents
agreed that the curriculum helped them to improve on the strengths and weaknesses identified by
the OSCE. Our comparison of the residents’ pre- and post-OSCE performances has shown
significant improvement in overall Communication, Relationship Development, and Patient
Education Skills. Also, through our USP pilot, we learned that we will need a better understanding
of the system in which we practice before embarking on such an endeavor and more USP cases to
better gauge how residents perform in reality.
Even having taught communication skills in other disciplines, teaching the same skills in EM
provided rich learning opportunities for us as curriculum innovators, evaluators, and administrators.
It is clear that learners need and appreciate curricula that are interactive and role model key patient
centered skills. Performance based assessment, OSCE and Unannounced Patients though time
intensive are meaningful assessment tools for both learners and programs.

EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 2
PI: Sondra Zabar, MD
NYU School of Medicine

B. INTRODUCTION (BACKGROUND)

Since the 1960’s, Emergency Medicine (EM) researchers’ efforts have worked to
demonstrate the importance of patient-centered doctor-patient communication, only acknowledging
decades later that advancing such patient-centered care will require increased and effective provider
education. 12 Having completed the Macy Initiative in Health Communication, a controlled study of
the impact of comprehensive, integrated clinical communication skills curriculum on student
patient-centered skills,3 the Section of Primary Care (PC) faculty at New York University School of
Medicine’s (NYUSOM) were prepared and eager to continue such work with the EM faculty on this
very important topic. Drs. Linda Regan, Jeffrey Manko, and Eric Legome, directors of the
NYUSOM-Bellevue Residency in EM, an integrated four-year residency dedicated to training highly
competent emergency physicians, shared this enthusiasm and began to plan for such an initiative.
Our program, entitled Emergency Medicine Professionalism and Communication Training
(EMPACT), expands on previous work by assessing and improving EM resident competency in
communication and professionalism through the development, implementation, and evaluation of
new curriculum and assessment measures. To ensure clinical competency of EM graduates in
delivering patient-centered care, we incorporated both ACGME core competency requirements and
several of the Picker Institute’s Dimensions of Patient-Centered Care into our program/research
design. Our objectives were to:
1. Design, implement and evaluate patient-centered healthcare curriculum for all 60 EM
residents;
2. Evaluate predictive validity of Objective Structured Clinical Examinations (OSCEs) by
assessing correlation of OSCE performance with actual resident performance in
emergent care setting for a cohort of PGY2 residents (n=15); and
3. Disseminate this Patient-Centered Care educational program to EM programs nationally.

1
Korsch BM, Negrete VF. Doctor-patient communication. Sci Am. 1972 Aug; 227(2):66-74.
2
Rhodes KV, Vieth T, He T, Miller A, Howes DS, Bailey O, Walter J, Frankel R, Levinson W. Resuscitating the
physician-patient relationship: emergency department communication in an academic medical center. Ann Emerg
Med. 2004 Sep; 44(3):262-7.
3
Kalet A, Pugnaire MP, Cole-Kelly K, Janicik R, Ferrara E, Schwartz MD, Lipkin M Jr., Lazare A. Teaching
communication in clinical clerkships: a model from the Macy Initiative in Health Communications. Acad Med.
2004; 79(6):511-20.

C. METHODS (PROJECT IMPLEMENTATION AND ADMINISTRATION)

To achieve our objectives, we conducted EMPACT in four phases. (See Figure 1. Project
Timeline) In Phase I, we established a baseline competency of EM interns using a 5-station OSCE.
Phase II, we developed an interactive skills-based series of five workshops focusing on interpersonal
and professionalism skills and integrated them into required monthly EM seminar series. In Phase
III, we conducted a post-curriculum OSCE to evaluate impact of curriculum. In Phase IV, we
developed and implemented two cases for the “unannounced” standardized patient (USP) project.4
Figure 1. Project Timeline
3/2007 4/2007 5/2007 6/2007 7/2007 8/2007 9/2007 10/2007 11/2007 12/2008 1/2008 2/2008

Curriculum Curriculum Development Curriculum Implementation Curriculum Packaging

OSCE Development (Case Individual
Pre- Post-
Evaluation development, SP Data Analysis Report Card Generation Remediation of
OSCE OSCE
Recruitment & Training) Poor Performers
Generation of Program
“Unannounced” Program Development (Logistics of Data
Case Development “Patient” in Implementation in
SP Program Implementation) Analysis
computer record ER
Mid-
Project Production of manuscripts, abstract
year
Dissemination submissions, final summary reports, etc.
Report

Phase I - Establish baseline competency of EM interns using a 5-station OSCE
In order to determine effectiveness of our curriculum, we chose to evaluate a subset of
resident performance in a pre- and post-OSCE. We wrote five cases and developed checklists that
assessed communication skills in scenarios commonly encountered by EM residents (See Table 1.
OSCE Cases). The checklists used to evaluate residents’ performance included items that assessed
overall communication skills (information gathering, relationship development, and patient
education), case-specific skills, and whether patients would recommend seeing the resident as their
physician.
Table 1. OSCE Cases
OSCE Case Picker Dimension Communication Skills
Informed Consent Access; Respect for patient’s values, preferences, and Obtaining Informed Consent;
Via an Interpreter expressed needs; Information, communication and Patient Education; Dealing with
education Challenging Patient
Disclosing a Medical Respect for patient’s values, preferences, and expressed Rapport Building; Emotion
Error needs; Emotional support and alleviation of fear and Handling
anxiety
Delivering Emotional support and alleviation of fear and anxiety; Emotion Handling; Patient
Unexpected Bad Information, communication and education Education
News
Transferring Care to Coordination and integration of care; Transition and Interdisciplinary Communication;
Another Service continuity Telephone Skills
Using the Emergency Access; Respect for patient’s values, preferences, and Dealing with Challenging Patient;
Room for Primary expressed needs; Emotional support and alleviation of Emotion Handling; Patient
Care fear and anxiety; Information, communication and Education
education

4
Kravitz RL, Epstein RM, Feldman MD, Franz CE, Azari R, Wilkes MS, Hinton L, Franks P. Influence of Patients’
Requests for Direct-to-Consumer Advertised Antidepressants: A randomized controlled trial. JAMA 2005;293:1995-
2002.

The preparation for the pre-OSCE included multiple preparatory steps. We trained five
standardized patients (SPs) to reliably and repeatedly portray their roles for the OSCE. SP training
sessions allowed the SPs to ask questions about their character, develop the improvisational range
that should be portrayed in their role, and practice how to consistently respond to participant
reactions. Prior to the pre-OSCE, we piloted the five cases and videotaped them to fine tune the
content of the cases and the checklists. Five EM chief residents, junior faculty, and medical students
were assessed as the participants. After reviewing the videos of their performances, examining the
data from checklists completed by the SPs, and hearing feedback from the participants in a
debriefing session, we adjusted the OSCE and checklist for clarity, timing, and realism. After making
the appropriate adjustments to the five cases, we were ready to launch the OSCE.
We conducted the pre-OSCE in three sessions. At each session, five residents went through
all five stations. All 15 PGY2 EM residents completed the OSCE. We chose to test the PGY2
because we believe, developmentally, the intervention will have the most impact at this stage of
learner. 90% of the OSCEs were audio and videotaped for the purposes of assessing inter-rater
reliability afterwards.
Colleen Gillespie, PhD, our evaluation researcher, compiled the feedback from faculty
observers and checklist data from SPs and summarized them as both a presentation for EM faculty
and report cards for each individual resident (See Attachments – Sample Report Card). The report
card noted each resident’s performance in five core areas: 1) communication, 2) overall
recommendation, 3) ratings of ability to apply expertise, 4) specific skills across cases, and 5) overall
case-specific skill scores. One case was not reliably scored (Delivering Bad News) and so scores
associated with that case should be interpreted with caution (details of how these scores were
calculated are included in the sample report card provided in the Attachments).
Overall, we noted there was room for improvement for all the residents in their Data
Gathering, Relationship Building, and Patient Education Skills. Residents performed best at Data
Gathering, less well at Relationship Building, and worst at Patient Education. As a group they also
scored low on Emotion Handling. Such information was also included in the report cards, which
demonstrated how the individual performed in comparison to the rest of the participants. This data
guided us in our focus and approach to key topics covered in the curriculum. Residents told their
program director that they found the OSCEs enjoyable and educational.

Phase II - Integrate an interactive skills-based series of five workshops —focusing on
interpersonal and professionalism skills—into monthly EM seminar series
We developed curricula based on the Macy model and other literature that taught five key
patient-care tasks, including: 1) relationship development and maintenance, 2) patient assessment, 3)
education and counseling, 4) negotiation and shared decision making, and 5) organization and time
management of EM. Our curriculum was composed of five one-hour interactive sessions that
addressed each of the core skills during the OSCE using different teaching modalities. (See Table 2.
EMPACT Course Schedule) We clearly delineated cognitive, skills, and affective objectives for each
session and highlighted them at the beginning of each session. We also created pocket cards that
included take-home points and a bibliography of relevant literature for each session. (See
Attachment X for the Session Objectives) Approximately 40 residents attended each of the session,
with ~10 PGY2 residents at each.


Table 2. EMPACT Course Schedule
Session Title Date Picker Dimension Communication Teaching Method
Skills
1. Making Every Session 08/01/2007 Respect for patient’s Patient Education, Videotape
Count: Effective values, preferences, and Rapport Building Reenactment and
Communication Skills in expressed needs; Debriefing, Mini
the Emergency Room Information, Lecture
communication and
education
2. Interdisciplinary 09/12/2007 Coordination and Conflict Negotiation; Audiotape Trigger,
Communication and integration of care; Telephone Skills Role Play
Respect Transition and continuity
3. Delivering Bad News 10/03/2007 Emotional support and Emotion Handling Videotape Trigger
in the Emergency alleviation of fear and from Medical TV
Department anxiety; Information, Show, Rolling Role
communication and Play between
education Attending and SP
4. Dealing with 11/07/2007 Access; Respect for Effective use of an Rolling Role Play
Culturally Diverse patient’s values, interpreter, Elements of between Residents
Populations in the preferences, and expressed informed consent and SP, Mini Lecture
Emergency Department needs; Information,
communication and
education
5. Discussing Medical 12/05/2007 Respect for patient’s Emotion Handling; Videotape Trigger
Errors in the values, preferences, and Patient Education; from Medical TV
Emergency Department expressed needs; Dealing with Show, Role Play with
Emotional support and Challenging Patient Small Groups
alleviation of fear and
anxiety

The first session, entitled “Making Every Session Count: Effective Communication Skills in
the Emergency Room,” aimed to provide residents with tools to maximize the effectiveness of their
communication with patients and their families. The session began with a videotaped reenactment of
OSCE case as a trigger for discussion. The session also included a PowerPoint presentation of how
residents performed in the OSCE overall and how they can improve their professionalism skills.
Residents’ feedback on this first session was very positive. They noted, “I feel the hurried
atmosphere of the ER causes the communication skills to atrophy. I think this was a useful
reminder of that and an effective tool relevant to ER situations.”
Our second session, entitled “Interdisciplinary Communication and Respect,” aimed to teach
residents to effectively work with the professionals around them to optimize patient care. This
session proceeded with a general discussion of how interdisciplinary communication can be both
positive and negative. Then, we played a re-enacted audiotape of the “Transferring Care to Another
Service” case they experienced in the OSCE, which we used as the trigger for discussion on how
interdisciplinary communication can be made better. A short lecture outlined the key steps and skills
to successful conflict negotiation and effective phone skills. Residents then participated in a role play
to practice these skills. We debriefed the role play as a large group to help residents identify what
personal traits or attitudes are barriers for successful interdisciplinary communication. We handed
out a pocket card summarizing an approach to conflict negotiation and telephone skills. A number
of residents stated that this was the first time these issues were ever addressed as part of their
curriculum. In particular, they said, “Good suggestions on how to approach multidisciplinary
communication. Short handout with key points helpful. Tape [was] very pertinent and important.”


The third session, entitled “Breaking Bad News in the Emergency Department,” aimed to
improve residents’ effectiveness in their delivery of bad news and provide residents with facts about
post-death procedures. The session began with the viewing of a trigger video clip from the Fox
television series, “House,” where a patient is abruptly given an AIDS diagnosis by the maverick, Dr.
Gregory House. This led to a conversation about what contributes to the sensitivities and difficulties
of delivering bad news, regardless of how the residents may perceive the severity of the news to be
(e.g. broken limb, new diagnosis of disease, or death of a loved one). Then, the residents directed a
rolling role play between an SP and Dr. Regan, who had to break the news of a positive HIV
diagnosis. The roll play was stopped a few times midstream to allow for a discussion of possible
strategies to better manage the situation. The session concluded with the key take-home points,
including protocol on how to follow-up on death notification, which residents took with them on
pocket cards. The residents notes that this topic "...can be fairly dry, has been done so much in med
school, BUT this was a very strong revisiting of this hard issue.” In particular, they said the session
was “excellent because it was DYNAMIC… well prepared, very interactive. The role play was very
well done."
The fourth session, entitled “Dealing with Culturally Diverse Populations in the Emergency
Department,” aimed to improve residents interactions with culturally diverse patients and
understand appropriate use of interpreters in the ED. The session began with a discussion of the
challenges of providing cross-cultural care, including how different health beliefs affect patient and
provider behavior and how language can act as the most apparent barrier. The conversation turned
to the challenge of working with various kinds of interpreters and strategies to overcome common
errors. During this session, a pair of Bengali-speaking SPs participated in a role play with Dr. Regan,
who demonstrated a bad version. Residents were asked to strategize on how to improve the
interaction and asked to come up and interact with the sp in front of the group. We used a Rolling
Role Play as the educational strategy for this session. We concluded the session with a summary on
how to use interpreters better. Residents again took home pocket cards that reviewed the key skills.
They enjoyed the use of small group role play and said it was "a refreshing approach to this topic."
The fifth session, entitled “Medical Errors in the Emergency Department,” aimed to improve
resident’s effectiveness in their disclosure of medical errors. This session began with a viewing of a
videoclip from the NBC television series, “Scrubs,” where a resident debates whether or not to
expose a potential medical error he believes was committed by his friend and colleague. While
comical, this clip helped the residents to begin broaching the difficult topic. Then, the session
continued with a discussion of frequent barriers to the disclosure of medical errors in general, as well
as specific to the ED. Residents were then given a checklist of items to follow which represented
common good practice for this sensitive topic. After explicitly discussing the 5Ws (Who, What,
Where Why, and When), the session proceeded with a skills practice. Each group of three to five
residents were given a scenario where one resident played the patient and another played the
resident who had to deliver the news about one of three medical error scenarios. Each group was
facilitated by a faculty member. The rest of the group observed and scored the scenario with a
checklist, similar to that which the SP's would use during the OSCE. Each small group reported
larger group the key learning points from their scenario. The session ended with the viewing of a
final clip from “Scrubs,” where everyone is relieved to find out an error did not occur and a re-
emphasis on the take-home points for the session.

Phase III - Conduct a post-curriculum OSCE to evaluate impact of curriculum.
Two months following the final EMPACT session, we held the post-OSCE. For comparison
purposes, we used the same five cases as the pre-OSCE. Due to the availability of the SPs, however,
we needed to train new SPs for four of the five cases. However, we purposefully chose SPs whom

we have worked with in the past and found to be reliable raters. Consequently, we believe the overall
integrity of the OSCE remains the same.
The post-OSCE was held in three sessions, with approximately five residents attending each
session. All 15 EM PGY2s participated in the post-OSCE and completed all five stations. Again, for
interrater reliability purposes, each station was videotaped, with the exception of the Transfer Case,
which was audio taped.
Colleen Gillespie and Tavinder Ark, MSc, our research associate, collected feedback from
faculty observers, checklist data from SPs, and resident satisfaction data relating to both the
EMPACT OSCE and curriculum. They summarized all data into report cards for each individual
resident, this time with a comparison of how their performance differed in the two OSCEs. The
report card reported each resident’s comparative performance in five core areas: 1) communication,
2) overall recommendation, 3) ratings of ability to apply expertise, 4) specific skills across cases, and
5) overall case-specific skill scores. The comparative data of the pre- and post-OSCE are described
later in the Results section.

Phase IV - Develop and implement two cases for the “unannounced” standardized
patient (USP) project.
The USP portion of EMPACT, was both exciting and educational. To our knowledge, based
on an extensive literature search in PubMed and Medline, the use of USPs in emergency clinical
settings had not been done prior to our attempt. Despite posing us with many labor-intensive
challenges, with full prior consent of residents, support of department and hospital leadership, and
approval from our IRB, we launched the USP program in December 2007 and assessed 12 residents
through 17 successful USP encounters in the ER.
For comparison purposes and to protect our SPs, we chose to use the Medical Error and
Repeat Visitor cases for the USP visits, as they required non-invasive interventions by the residents.
Having obtained verbal confirmation from Medical Records, Registration, EM Nurses, EM
Attendings, and the radiologists, we were poised to begin this aspect of the project. As the USPs in
both the cases were supposed to have visited the Bellevue ER before, both cases required the entry
of previous medical notes, x-rays, MRIs, and labs in the medical record system. We obtained
specified Medical Record Numbers for the USPs. However, the challenges of this effort soon
became apparent.
The rate limiting step in setting up the Medical Error case was the time frame allowed by
MISYS, the medical records system, to enter prior visits into the record history. Because the USP
was supposed to have visited the ER two days prior to the actual USP visit, we needed a visit to be
opened two days prior in real time. The system would not allow us to enter future visits. This meant
that the Bellevue Hospital EM Admitting needed to be ready to open the visit when we asked two
days prior to the actual USP visit. This also meant that the PACS team, the group that handled all
radiology related issues, had to be ready to upload the X-ray images and reports onto the system
once the prior visit was opened. Because this was a voluntary effort on the part of the Admitting
and PACS, it took a few tries to come up with an efficient system for getting all the required
information adequately noted in the USPs fictitious medical records prior to the actual USP visit.
The main challenge of the Repeat Visitor case was the manipulation of the MRI images.
Based on the original version of our case, the USP was supposed to have visited the Bellevue ER
twice in the past and have taken MRI images here. In order to have the MRI images reflect the case
details of each visit (e.g. dates, patient name, etc.), we needed to edit more than 50 images per visit.
We consulted Sectra, the company that services our PACS system, who offered to write us a
program that would quickly do so for $12,000. Since this was not possible given our financial
situation, we ended up editing the USP case. In the new version, the USP visited another ER in New

York City two times and got an MRI at another location. The USP then brought the MRI report to
the actual USP visit at Bellevue.
The third most prominent challenge of the USP project resulted from our need to limit the
number of informed people in the ER, the unpredictability of the ER, and the assignment of the
residents to the USP case on a given day. We tried to limit the number of people in the ER who
knew that a USP was present to avoid detection. Although we tried our best to have the USP triaged
exactly to where the targeted resident was supposed to be working on the given day, our efforts were
often thwarted by eager medical students, rotating orthopedic residents, or unexpected schedule
changes. During a few of our scheduled visits, the USPs were mistakenly examined by another care
provider while the target resident was called away to see a more acutely ill patient. The attending
may have known about the USP, but at times was engaged in the care of another patient when non-
targeted personnel elected to see the USP.
After 29 attempts, we successfully evaluated 17 of the 30 planned visits (five residents were
visited by both types of USPs, which accounted for ten of the visits). We audio taped ~71% of the
encounters (12/17), which we will use to establish intra- and inter-rater reliability. Following each
visit, we videotaped the USPs as they debriefed the entire experience and completed the checklists.
As the last USP visit was just completed on April 8, 2008, a comprehensive comparison of the USP
and OSCE performances is still pending.

D. RESULTS

The OSCEs assess residents’ clinical skills in two major areas: 1) Communication Skills and
2) Case-Specific Skills. The Communication Skills describe residents’ ability in information
gathering, relationship development and patient education skills. The Case-Specific Skills describe
the residents’ ability to perform skills specific to each case. They are divided into five broad
categories: 1) managing a difficult case, 2) accountability, 3) delivering bad news, 4) patient education
and 5) treatment plan and management.
For the EMPACT OSCE and USP visits, Communication and Case-Specific Skills questions
are rated by the SP on a 3-point scale of “not done” (resident did not perform the task at all),
“partially done” (the resident attempted the task, but did not do it entirely correctly), or “well done”
(the resident performed the task and did it correctly). In addition, residents’ were rated by the SPs
on the degree to which they would recommend this doctor to a friend based on their interpersonal
skills and expertise on a 4-point scale (1= Not recommend and 4= Highly Recommend). Residents’
Communication and Case-Specific Skills are calculated as the percent of items rated as “well done”
across all cases. The overall recommendation rating was based on interpersonal skills and expertise
was calculated across all cases as a mean average on a 4-point scale. These score was calculated
across all 5 cases. A pre and post comparison was conducted. For the USP visits, this score was
computed only across the repeat visitor case and broken wrist (medical error) and compared to the
pre and post of only these two cases.

D1. Resident Experience of EMPACT
Data on residents’ exposure to actual clinical situations similar to the OSCE cases highlight
the importance of having an opportunity to practice low frequency clinical situations: only 29%
reported encountering a situation involving giving bad news since the pre-curriculum OSCE and
slightly less than half (43%) reported exposure to a clinical situation involving a medical mistake.
Despite evidence reported below that residents made substantial improvements from pre- to post-
curriculum in some core clinical areas, from more than a third to close to half of residents reported

that their performance on the post OSCE was “about the same” as their performance on the pre
OSCE (depending on the case, % ranged from 36% to 50%). Most agreed that the OSCE helped
them identify their strengths and weaknesses (60%) and provided a good cross-section of cases
(74%). However, some skepticism of the value of OSCEs was also apparent as just over half did
not think that the OSCEs taught them something new (54%) or was a fair evaluation of their skills
(60%). When asked in an open-ended manner to describe what was most helpful about EMPACT
most focused on the OSCE (perhaps reinforced by having just completed the post OSCE!), focusing
on practice (“repeated exposure to clinical scenarios”) and on being able to assess and reflect on
one’s skills (“recognizing my triggers for what is a problem for me;” “self reflection about my
weaknesses,” “the situations are a good reflection of what we see in the ED and they highlight some
of the weaknesses we have in dealing with difficult situations. I know I tend to make the same
mistakes over and over again.”). Several residents simply said that the EMPACT “curriculum” was
the most helpful aspect of EMPACT overall.

D2. Impact of the Curriculum: Pre- vs. Post-Curriculum OSCE Results
Comparison of the pre- and post-curriculum OSCEs showed significant improvement in
residents’ overall Communication Skills (pre=53.4% SD 14.9% vs. post=65.5% SD 11.5%;
p=0.003). In particular, they improved on overall Relationship Development skills (pre=49.2% SD
21.5% vs. post=59.8% SD 17.8%; p=0.025) and especially in their overall Patient Education skills
(pre=31.6% SD 15.1% vs. post=57.0% SD 15.2%, p<.001).
In terms of residents’ case-specific skills, significant improvement from pre- to post-
curriculum was seen in the Repeat Visitor case (pre=38.7% SD 18.1% vs. post=73.3% SD 16.7%,
p<.001) and close to significant improvement in the Bad News case (pre=54.0% SD 15.5% vs.
post=66.9% SD 22.1%; p=.066).
SPs rated residents more highly in terms of the degree to which they would recommend
them (using a 4-point scale) for their interpersonal skills (pre=2.84 SD .58 vs. post=3.09 SD .41;
p=.066) and for their medical expertise (pre=2.90 SD .48 vs. post=3.19 SD .29; p=.014).


Impact of EMPACT:
Pre-Curriculum vs. Post-Curriculum OSCE Communication Scores (n=15)
80%
74%
Pre Post
70%
70% p<.01
65%
p<.05
60% p<.001
60% 57%
53%
49%
50%
% Well Done

40%

32%
30%

20%

10%

0%
OVERALL Information Gathering Relationship Patient Education
COMMUNICATION Development


Impact of EMPACT:
Pre-Curriculum vs. Post-Curriculum OSCE Case Specific Scores (n=15)
80%
p<.001 73% Pre Post
p<.10
70% 67%

59%
60%
54% 54% 53% 54%
53%
50%
% Well Done

44%

40% 39%

30%

20%

10%

0%
Bad News Interpreter Broken Wrist Repeat Visitor Transfer
(Medical Error)


Impact of EMPACT:
Pre-Curriculum vs. Post-Curriculum Recommendation Ratings (n=15)
Highly 4
Recommend
Pre Post

p<.10 p<.01
3.19
3.09
3 2.90
Recommend 2.84

Recommend
w 2
Reservations

Not 1
Recommend
Recommendation - Interpersonal Skills Recommendation - Applic of Expertise

D2. Comparison of OSCE and USP Scores
A major goal of this project was to begin to explore how residents’ performance in an OSCE
relates to their actual clinical performance, at least as assessed by an USP. Given that the pre-OSCE
took place in July, the post in March, and the USP visits anytime between mid-January and early
April, scores generated from the USP visits were compared with both pre- and post-curriculum
OSCE scores. Although, we expected USP scores to be more highly correlated with post-OSCE
scores since they generally occurred closer in time. Twelve residents had at least one USP visit and 5
residents were visited by both USPs (Repeat Visitor and Medical Error). We report correlations for
both sets of data in order to maximize our sample size (including all 12 residents by reporting
whatever USP data is available for each resident be it one or two visits) and maximize our sample of
actual clinical performance (including only those 5 residents from whom we have two samples of
performance data, i.e., two USP visits).

Correlations between OSCE and USP Scores
At least 1 USP Visit (n=12) 2 USP Visits (n=5)
USP Scores Pre OSCE Post OSCE Pre OSCE Post OSCE
Overall .70 .17 .83 .53
Communication (p=.011) (p=.600) (p=.088) (p=.379)
Skills
Overall Case .63 .17 .64 .85
Specific Skills (p=.029) (p=.598) (p=.249) (p=.066)


Results suggest that the USP scores are strongly correlated with the pre-OSCE scores for
both residents with one or more visits and for only those residents with an adequate sample of
clinical performance (both Repeat Visitor and Medical Error USP visits). However, it is only among
those with both USP visits that we see strong correlations with post OSCE scores. It may be that
residents’ performance on the pre-curriculum OSCE best represents how they are in actual clinical
practice while their performance on the post-curriculum OSCE was more reflective of how they
perform when being evaluated on the basis of clear criteria (as shared through the 5-session
curriculum). These exploratory results also demonstrate the importance of including multiple
samples of performance – one USP visit is probably not sufficient to obtain a true and accurate
picture of physician skills.
We assessed two additional dimensions of clinical performance: patient-centeredness (e.g.,
fully explored my experience of the problem, took a personal interest in me, earned my trust,
acknowledge impact of situation on my life) and the degree to which the resident “activated the
patient” (e.g., helped me to understand the nature and causes of my condition, helped me find out
about the different medical treatment options available, made me feel confident I can figure out new
solutions if my situation changes) (Hibbard ref). There is increasing evidence that these skills, along
with core communication and case-specific skills, are associated with important patient outcomes.
Therefore, we examined correlations between average scores residents received from USPs on these
items and their OSCE scores and found, as above, that both pre and post OSCE communication
and case-specific skills were strongly (albeit not significantly) and positively correlated with patient
centeredness and patient activation.

2 USP Visits (n=5)
Overall Communication Skills Overall Case Specific Skills
USP Scores
Pre OSCE Post OSCE Pre OSCE Post OSCE
Patient .56 .78 .79 .84
Centeredness (p=.326) (p=.120) (p=.112) (p=.078)
Patient .68 .60 .85 .84
Activation (p=.202) (p=.282) (p=.070) (p=.078)

E. DISCUSSION

There are many things we can learn from the development and implementation of a new
curriculum designed to help residents with their communication skills. Even having taught
communication skills in other disciplines, teaching the same skills in EM provided rich learning
opportunities for us as curriculum innovators, evaluators, and administrators
First, residents portray an outward confidence about their communication skills, which
lacked grounding in their assessment levels. Despite their relaxed attitude about the OSCE cases,
the data showed that they had difficulty with some of the scenarios. This came as a great surprise to
some, though the majority already knew there was some deficiency when questioned. Resident
reported they learned that: 1) without listening to what patients have to say about their condition, it
is difficult to hear what the patient is actually trying to convey, without appropriately providing
patient education, quality of care may be compromised, 2) without communicating effectively with
other disciplines, it will be difficulty to coordinate care, and 3) without demonstrating empathy,
kindness, patient satisfaction is hard to achieve. Having the opportunity to step back from the flurry

of activities in the EM, residents were able to acknowledge their respective shortcomings in
communication skills and commit to improving them for their patients.
Second, residents received their feedback in a much more affirmative manner than we had
hoped. We are struck by their positive feedback for the “much needed” education on “basic skills”
that are essential for success as EM physicians. Their enthusiasm for this education is surprising and
gladly received. They have been instructive in helping us to design our curriculum so that they can
get the most out of the experience for their practical day-to-day use.
Third, as measured by a reliable and valid OSCE, the EMPACT project shows that a focused
curriculum, with five one-hour group interactive sessions on communications and professional
curriculum, can significantly improve residents’ rapport building and patient education skills. These
skills were tested months after the curriculum. Our curriculum is unique, not for its topics, but
because of the variety of educational methods we incorporated (i.e. role play, modeling with
standardized patients, discussion triggered by “TV medical clip” and reenactments of real residents’
performances). This approach is highly acceptable and engaging to residents, as evidenced by their
feedback.
Fourth, through the USP aspect of this project, a novel endeavor, we have shown that this
methodology is feasible and acceptable to residents, program directors, and faculty and hospital
administrations. As noted by the program director, this project has already brought added value to
the resident learning and patient care. By informing the residents that USPs would be visiting them
in the ED, the residents seemed to perform at a higher level, not knowing which patients might be
evaluating their performance and what measures were being evaluated. One resident commented
that when he thought a patient was a USP, he washed his hands more frequently, thinking that hand
washing was the metric we were evaluating. A faculty member noted that when one resident thought
he had identified a USP, he seemed more empathic and professional when discussing the discharge
plan and follow-up care. Clearly, the patients also benefited from the study, as higher professional
standards, including stricter adherence to Joint Commission Safety Initiatives were being executed
by the residents to more patients, not only the USPs.
We must further analyze our USP results, debriefing tapes, and audio tapes to understand
what additional information we can learn about our residents’ skills using this innovative
methodology. The fact that our post-OSCE results did not fully match the residents’ USP
encounters further supports the need to perform larger USP studies with multiple cases in order to
better understand the degree to which OSCEs reflect real world skills. It is our hope that we can in
what ways OSCEs can predict real life performance in order to enable us as educators to use them
as efficient and effective tools to help learners become expert physicians.
With the ACGME recently placing greater importance on evaluation of patient outcomes
and its linkage to medical education, we believe that our project is representative of a new way to
assess real-time resident physician performance. As program evaluators working toward
enhancement of curricula that better meet patient needs, this project has contributed much to our
larger efforts. The data collected from these OSCEs have been incorporated into Database for
Research on Education Academic Medicine (DREAM), an initiative of our Research on Medical
Education Outcomes Unit (ROMEO), which enables long-term, longitudinal assessments of
participant performance both in residency and beyond. Further comparison of OSCE evaluations
with USP encounters will enable educators to determine whether or not these commonly used
evaluation tools actually mimic real practice. The current OSCE data will be assessed in conjunction
with future evaluations and patient outcomes. We eagerly await results of a larger trial.
Lastly, this collaboration between NYUSOM Primary Care and Emergency Medicine has
enabled us to further heighten the overall abilities of NYUSOM faculty to teach and communicate
with each other and to our residents. Additionally, we believe this curriculum also provided an added

value as a faculty development opportunity. Faculty members in the Emergency Department have
gained a standardized approach to teaching and assessing communications skills after participating
or playing facilitative roles in the curriculum.

F. DISSEMINATION

We have already begun to share our methods with other departments and institutions.
Owing to the success of the EMPACT OSCE, the Gastroenterology fellowship used our cases for
their OSCE held on October 6, 2007. Their use of our communication skills checklist will enable us
to compare performance across disciplines and levels of training. They are planning a second OSCE
for additional fellows in May 2008. Additionally, current plans are under way within the Department
of Emergency Medicine at Johns Hopkins to apply for funding to support the use of USPs in
evaluation of curriculum focusing on disaster education.
In terms of publication, the Arnold P. Gold Foundation, which promotes and affirms more
compassionate medical care and caregivers, accepted our abstract (“A Curriculum in Patient-
Centeredness for Surgery and Emergency Medicine Residents: Establishing the Baseline.” M.
Hochberg, S. Zabar, L. Regan, R. Laponis, R. Richter, A.L. Kalet), for presentation at the Gold
Foundation Symposium, How Are We Teaching Humanism in Medicine and What is Working?,
which was held on September 27-29, 2007, Chicago, IL. Future plans include submission to
Academic Emergency Medicine, the journal of the Society of Academic Emergency Medicine as well
as to the national Council of Residency Directors (CORD) meeting for Emergency Medicine which
is held annually.

G. FINANCIAL REPORT

The Financial Report will be provided by the NYUSOM Sponsored Programs
Administration under separate cover.


H. ATTACHMENTS
a. Sample Case and Checklist
b. Sample Report Card
c. Session Objectives
d. Sample Pocket Card
e. Sample Feedback
f. Dissemination
i. Gold Foundation Abstract


Attachment – Sample Case and Checklist (Medical Error)

STATION OVERVIEW

OBJECTIVES To test the resident’s ability to:
1. Admit an error has been made
2. Be empathic
3. Address patient concerns surrounding an error

LOGISTICS Personnel: Standardized patient, male,
32 y.o., dressed in regular
clothing, sitting in chair.

Station Materials: • Resident instructions
• SP Instructions
• SP evaluation forms
• Faculty evaluation forms

Room Arrangement: • Station signs
• Chair (2)
• Exam table

EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 18

RESIDENT INSTRUCTIONS

PATIENT Name: John McCoy
INFORMATION Age: 32

REASON FOR
ENCOUNTER • John McCoy came to the ER 2 days ago complaining of
right wrist pain after falling while rollerblading near
Washington Square Park.

• At that time, his hand x-ray was MISREAD by a
resident as normal and he was sent home with an Ace
bandage and some ibuprofen.

• The Radiology Attending re-read the x-ray and found a
non-displaced, non-intra-articular right distal radius
fracture.

• He presents today to the ER after having been called
back.

YOUR ROLE ER Resident

YOUR TASKS 1) See the patient, explain what has occurred, and
develop a plan.


STANDARDIZED PATIENT INSTRUCTIONS

THE SCENARIO Your name is John McCoy and you are 32 years old. 2 days ago you
were rollerblading in Washington Square Park prior to when your
shift started for work at a restaurant (you work as a waiter at the
Union Square Cafe). You fell and hit your outstretched right hand on
the pavement. Your right wrist hurt a lot and you were afraid that it
might have been broken. This was particularly concerning as you
work as a jazz pianist occasionally. You went to the Emergency
Room and after waiting for 4 hours, finally saw a doctor. They took
some x-rays and told you it was just a sprain. You got some pain
drugs (ibuprofen) and a bandage to wrap your wrist. You were told to
rest your wrist, use ice, and keep it wrapped and raised as much as
possible. Because of the wait at the ER, you had to have someone
cover for you at work.

Because you don’t get sick pay, you decided to work yesterday even
though you were in pain. This morning, you got a call from a nurse
instructing you to return to the ER as the doctors had some
information about your wrist. You again got someone to cover for
you (although you still won’t get paid) in order to go back to the ER
today. Today, the pain in your right wrist is about 5/10 (10 being the
worst pain in your life) and it only gets worse when you bend it back
or press on it. The swelling has gone down from 2 days ago and it
seems like it is slowly getting better, despite having used it yesterday
at work.

CHARACTER Objective: • To understand what has occurred and know when
DESCIRPTION you can return to work
Obstacles: • You are upset about missing work as you are
having a tough time making ends meet.
Tactics: You are initially somewhat agitated as you are
missing work again
When you hear the news of the mistake you become
further agitated
If the resident is empathic, apologizes, and is
helpful, you calm down a little.
If, however, the resident is at all defensive,
argumentative or unhelpful, then your agitation
continues to increase.


SINCE YOU Since you left the ER 2 days ago, you have been trying to do what the
LEFT THE ER doctor told you to do: rest it, use ice, compress it with the bandage
and keep it elevated. You did, however, go to work yesterday after
taking a few ibuprofen (Advil) tablets and a strong gin and tonic in
order to minimize the pain. You got thru your shift without too much
trouble and were able to compensate using your left hand more often
than usual. Today, you still have some pain, but the ibuprofen is
helping.

PERSONALITY You tend to be a little dramatic. When you are happy, you border on
gushy and when you are upset, you can get angry. This is partly due
to the fact that your financial situation is slightly unstable and it can
put you on edge at times.

CURRENT LIFE You live with a roommate in the East Village.
SITUATION You have no children.
You work as a waiter at the Union Square Cafe and play jazz piano
intermittently with various local groups. You are still hoping to make
it as a pianist, but it hasn’t worked out that well so far.

PAST MEDICAL None. You are otherwise very healthy and active.
AND SURGICAL
HISTORY

FAMILY Your mother and father are both living in Ohio. They are healthy as
HISTORY far as you know. You have one brother who is healthy and married
living in Ohio as well.

SOCIAL You smoke ½ pack a day for the past 10 years.
HISTORY You drink alcohol at least 3 times per week, usually having 2-3 drinks
each time.
You do not use recreational drugs.
You are sexually active with a girlfriend you have had for the past 6
months. You use condoms for protection.
You are eating and sleeping well and staying active by rollerblading
and going to the gym occasionally.

MEDICATIONS Ibuprofen (Advil) – 2 tablets every 4 hours for pain

ALLERGIES None

THE When the Resident knocks and enters the room, you are sitting in a
ENCOUNTER chair in the exam room talking with a colleague trying to get someone
to cover for you as you are missing work. You are upset interrupting


the person on the other end of the phone line and end the conversation
about 20-25 seconds after the resident enters the room. When you
hang up, you are still upset having had to miss work for the second
time this week. You show this by making eye contact with the
resident, occasionally breathing deeply and audibly, and have
aggravated tone to your voice.

You are testy and confrontational the entire interview and
occasionally interrupt the resident to voice your frustration.

If asked in an open-ended way why you are here, state: “You guys
called me. I was here a couple days ago about my wrist, so I assume
it’s about that.”

With respect to your wrist-

Any pain? – “A little, but the Advil helps.”
How bad is the pain? – “About 5 out of 10”
Any pain with movement? – “Only when I bend it back”
Any swelling? – “It’s gotten a lot better.”
Any tingling or loss of sensation? – “No”
Any redness? – “No”
Any tenderness? – “It hurts a little when I push on it.”

In general currently:

How have you been? – “Fine, I guess. My wrist hurt a bit during
work yesterday, but I got through it. But I’ve missed two days
because of this stupid thing.”

If/when you are told a mistake was made (i.e. someone read the x-ray
of your wrist incorrectly and you actually have a bone fracture)
regardless of where it occurs in the interview, take a moment to let it
set in and then at first become upset. Raise your voice, but do not
shout, look the Resident straight in the eye, and impatiently tap your
finger on the desk or table to underline your frustration. State:
“So my wrist is broken?”
“This is so annoying.”
“I mean, what’s going on here? I had to miss two days of work
because of this.”

If then the Resident acknowledges the mistake, states that
he/she is sorry that it happened/empathizes, you still remain
angry and state in a slightly aggressive tone:
“Oh man. I knew it. I knew it was something bad. This


always happens to me. Well, will there be any long-term
damage?”

When you realize the long term damage will be nil or
minimal, you are only a little relieved. State in a somewhat
frustrated way:
“Why did this happen? What if this was something really
serious? I mean, my God, does this happen all the time?”

Whatever the resident’s response is state: “Well, don’t you think
this is a bad system here?”

If the Resident remains apologetic and non-confrontational, you
calm down a little and ask:
“Well, when can I go back to work?”

If the Resident acknowledges that a mistake was made, but then
becomes defensive, does not empathize or say he/she is sorry,
or makes up a bizarre story -> get more upset:
“I mean, me missing work today would have been totally
unnecessary right? If you guys actually did your job, I
wouldn’t have had to come down here.”
“I knew I shouldn’t have come to his ER.”

If the resident asks if they can write you a note, state sarcastically: “A
note? What I am I going to do with a note?”

Whenever the Resident changes course and becomes more
apologetic/empathic, react accordingly. Adequately challenge the
resident. You are upset for a multitude of reasons: losing work pay,
being in pain, losing faith in your health care provider, and not being
able to play piano. If you feel the resident is making a genuine effort
to address your concerns, is empathic and non-confrontational,
become less angry, but maintain a baseline of annoyance and
frustration. If the resident ever becomes dismissive/confrontational or
you don’t feel supported, become more upset.

Towards the end of the interview, regardless of the Resident’s
reactions, become calm. Your motivation for doing this is as
follows: If the Resident has admitted the mistake and acted
appropriately, you are satisfied. If the Resident has done poorly by not
admitting the mistake or making fabrications you become withdrawn
contemplating a lawsuit: (Please note: Do not mention lawsuit,
litigation, suing, or anything relating to malpractice unless the
Resident brings it up - this is purely an internal cue for you to help
you act out the character). If the latter is the case – partially cross


your arms, rest your head on one hand, and avoid eye contact.

Once you have calmed down a little, state: “Well, I came all the way
down here. Now what?”

CHALLENGES • Admit that an error was made
FOR THE
• Regain patient trust
RESIDENT

CUES FOR THE Non-verbal 1 At the beginning of the interview, eye contact
RESIDENT with occasional audible breathing.

Verbal 2: State: Why exactly was I called back? ->
Resident to verbally acknowledge your concern
and explain reason

Verbal-Non- Express anger (state that you are upset, raise
Verbal 3: your voice, look at the Resident in angry and
accusatory fashion, underline your verbal
comment with tapping your fingers on the table)
-> Resident to verbally acknowledge your
anger/being upset and label it as understandable

Verbal-Non- Calm down in last part of encounter; if Resident
Verbal 4: acted appropriately: calm down (e.g., appear
more relaxed in your posture and voice); if
Resident acted inappropriately: withdraw (e.g.,
cross arms, speak in short sentences, etc). State:
“Well, I’m here. What do we do now?”

TIMING Initially: You are already a little upset.

Ongoing: If the Resident is empathic/truthful/straightforward,
become more and more calm. If the Resident is
defensive/evasive/making up bizarre stories, become more and more
upset.

2 minute warning: Begin to calm down because the Resident is
acting appropriately or withdraw because the Resident is acting
inappropriately. State: “What do we do now?”


Evaluator’s Checklist
COMMUNICATION Not Done Partially Done Well Done
Information Gathering

Elicited your responses using appropriate Impeded story by asking
Used leading/judgmental Asked questions one at a time
questions: leading/judgmental questions
questions OR asked more than without leading patient in their
AND more than one question at
one question at a time responses
No leading questions a time
Only one question at a time
Clarified information by repeating to Did not clarify (did not repeat Repeated information you Repeated information and
make sure he/she understood you on an back to you the information you provided but did not give you a directly invited you to indicate
provided) chance to indicate if accurate whether accurate
ongoing basis
Did not interrupt directly BUT cut
Did not interrupt AND allowed
Allowed you to talk without interrupting Interrupted responses short by not giving
time to express thoughts fully
enough time

Relationship Development
Communicated concern or intention to Did not communicate intention to
Words OR actions conveyed Actions AND words conveyed
help/concern via words or
help intention to help/concern intention to help/concern
actions

Non-verbal behavior enriched Non-verbal behavior was
Non-verbal behavior Non-verbal behavior facilitated
negative OR interfered with
communication (e.g., eye contact, posture) demonstrated attentiveness effective communication
communication
Acknowledged emotions/feelings Acknowledged & responded to
DID NOT acknowledge
Acknowledged emotions/feelings emotions/feelings in ways that
appropriately emotions/feelings
made you feel better
Made comments and
Made judgmental comments OR Did not express judgment but did
Was accepting/non-judgmental facial expressions not demonstrate respect
expressions that demonstrated
respect

Used words you understood and/or Consistently used jargon Sometimes used jargon AND did Explained jargon when used, OR
explained jargon WITHOUT further explanation not explain it avoided jargon completely

Education and Counseling
Asked questions to see what you Asked if patient had any Assessed understanding by
Did not check for understanding questions BUT did not check for checking in throughout the
understood understanding encounter
Gave confusing OR no
Information was somewhat clear Provided small bits of information
explanations which made it
Provided clear explanations/information impossible to understand
BUT still led to some difficulty in at a time AND summarized to
understanding ensure understanding
information
Collaborated with you in identifying Told patient options, THEN
Told patient next steps THEN
Told patient next steps/plan mutually developed a plan of
possible next steps/plan asked patient’s views
action


ADDRESSING MEDICAL ERROR

Accountability
Disclosed error
Did not directly disclose the error
• Direct (used the words “error” or Did not directly disclose the error
(there was a “problem”) NOR
(there was a “problem”) OR
Directly disclosed the error upfront
“mistake”) directly disclosed late in the
was the explanation upfront
interview
• Prompt disclosure
Personally apologized for the error (“I am Did not apologize for error NOR
Apologized for the error OR for Apologized for the error AND for
for the inconvenience it caused
sorry that this happened) the inconvenience it caused you the inconvenience it caused you
you
Shared the cause of the error (i.e., Acknowledged issue with system
Did not acknowledge issues with Acknowledged issue with system
BUT was dismissive/
Explained issues with system) system AND was genuine in addressing it
condescending
Took no personal responsibility
Took a general responsibility as
for your present situation (e.g., Took a personal responsibility for
Took responsibility for situation assigns your problem to other
part of the department for your
your situation (“I will…)
present situation
person/department)
Made general suggestion for
Identified future preventative strategies Did not address how situation improvement (e.g., “We’ll look Offered specific strategies for
to prevent situation from happening again would be prevented in future into it,” “I’ll make a note of it to potential improvement of system
my Attending”)

Managing a Difficult Situation
Became defensive/ Became defensive/
Remained calm AND did not
Avoided assigning blame argumentative AND assigned argumentative OR assigned
mention blame someone else
blame to a person/department blame to a person/department
Maintained a high level of
Maintained professionalism by Unable to control emotions, Attempted to control emotions
professionalism in handling your
became dismissive and (e.g. was somewhat dismissive
controlling emotions specific situation, did not show
condescending or condescending)
anger or frustration

Delivering Bad News
Prepared you to receive the news: Entered room in a manner
Entered room in a manner Entered room in a manner befitting
unfitting the news AND
• Entered room prepared to deliver news physically situated him/herself
unfitting the news OR physically the news AND physically situated
situated him/herself far from you him/herself close to you
• Ensured sufficient time and privacy far from you

Assessed your readiness to receive news: Attempted to deliver warning
shot, BUT inappropriately (does Gave you a well-timed warning
• Gave warning shot (e.g., “I have No warning shot
not pause for your assent OR shot
some good and bad news for you…”) warning shot too long)

Gave you opportunity to emotionally
respond: Responded inappropriately to Allowed you to emotionally Allowed you to express your
your emotional reaction (no respond (vent) BUT did not feelings, fully giving you the feeling
• Remained sensitive to your venting of opportunity to vent, cut you off, address/acknowledge response you were being listened to before
shock/anger/disbelief/accusations became defensive) before moving on moving on

• Attended to emotions before moving on
Acknowledged your feelings
Directly asked what you are feeling: “What (e.g., “I see that you are
Did not ask specifically “What Specifically asked you “What are
upset…”) BUT did not
are you thinking/feeling?” are you thinking/feeling?” you thinking/feeling?”
specifically ask you to name your
emotions
Offered specific next steps (e.g.
Provided appropriate “next steps” Did not offer next steps AND
Offered only general next steps
Orthopedics is going to fit you for a
(e.g., I’ll be calling Ortho) OR
• Orthopedics for immediate care evaded response as to what will
promised to “ask the attending”
cast) AND informed you of long
happen long-term term care needs (e.g., unable to
• What to expect long-term for next steps
use arm for 6 weeks)


Would you recommend this doctor to a friend for his/her interpersonal skills?
Recommend with
Not Recommend Recommend Highly Recommend
Reservation

Would you recommend this doctor to a friend for his/her medical competence?

Not Recommend Recommend with Recommend Satisfactory Highly Recommend
Non -exemplary Physician: Reservation Unexceptional Physician: Model Physician:
superficial, artificial demeanor applied appropriate knowledge base applied sophisticated, wise, thoughtful, applied
Physician:
knowledge base inadequate to my adequately to my specific situation profound knowledge base specifically to
awkward, knowledge base only
situation my situation
somewhat apparent in application to my
situation

COMMENTS:


Attachment – Sample Report Card
EMPACT
OSCE Report of Results – July 2007
Clinical skills were assessed in 5 cases. Your scores in 5 core areas – communication scores, overall
recommendation scores, ratings of ability to apply expertise, specific skills across cases, and overall case-
specific skill scores -- are reported in the charts that follow. For case-specific skills and recommendation
ratings, results for each case are included as well. One case was not reliably scored (Unexpected Death) and
so scores associated with that case should be interpreted with caution.

Overall communication score: Calculated across all cases as the % of behaviorally-anchored
communication items (8-14 items per case) for which you were rated as having performed well (“done
well”). Sub-domains include: Information gathering, relationship development, and patient education.
Overall recommendation rating: Calculated across all cases on the basis of rating of degree to which
“would recommend physician to a friend based on his/her communication skills” with the following response
options: Not Recommend – Recommend with Reservations – Recommend – Highly Recommend.
Overall rating of application of expertise: Calculated across all cases on the basis of rating of degree to
which applied expertise effectively, using a 4-pt scale: Insufficient Application, Slight Application, Sufficient
Application, Exceptional Application of Expertise.
Selected skills across cases: Calculated as the % of items rated as well done for specific skills
measured across at least several cases including: delivering bad news, managing difficult situations,
accountability, handling emotions.
Overall case-specific skills: Calculated across all cases as the % of items rated as well done for core
knowledge and skill items specific to each case.


Communication Scores for Sample Student
100%
Error Bars: +/- 1 Std Dev
Your Scores Class Mean
90%

80%

70%
64%
61%
60% 56%
% Well Done

51% 50% 52%
50%

40%
33%
30% 27%

20%

10%

0%
OVERALL Communication - Communication - Communication - Patient
COMMUNICATION SCORE Information Gathering Relationship Development Education


Overall Recommendation Rating for Sample Student
Highly 4 Error Bars: +/- 1 Std Dev
Recommend

3.35

Recommend
3

Informed Consent
2.75

X-Ray Recall
Unexpected Death*

Transfer of Care
Recommend
with 2
Reservation

Not 1 Repeat Visit
Recommend OVERALL Recommendation Ratings
RECOMMENDATION for Each Case *Unreliable Case -
Interpret w/ Caution


Overall Rating of Application of Expertise for Sample Student
Exceptional

Unexpected Death*
Application 4
Error Bars: +/- 1 Std Dev
of Expertise

Sufficient
Application
3 2.84

Informed Consent

Transfer of Care
Informed Consent

Transfer of Care

X-Ray Recall
Repeat Visit

X-Ray Recall
Repeat Visit
2.00
Slight
2
Application
of Expertise

Insufficient 1
Application OVERALL RATING Ratings
APPLICATION OF EXPERTISE for Each Case *Unreliable Case -


Case-Specific Skills for Sample Student
100%
Error Bars: +/- 1 Std
90%

80%

70%

60%

Transfer of Care 86%
48% 49%

Informed Consent 70%
50%

Unexpected Death 64%*
40%

Repeat Visit 50%
30%

X-Ray Recall 67%
20%

10%

0%
OVERALL CASE-SPECIFIC Rating of Knowledge Skills
KNOWLEDGE SKILLS for Each Case *Unreliable Case -


Zabar final report cg

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (11)

Semelhante a Zabar final report cg

Semelhante a Zabar final report cg (20)

Mais de Picker Institute, Inc.

Mais de Picker Institute, Inc. (20)

Zabar final report cg