SlideShare uma empresa Scribd logo
1 de 4
Baixar para ler offline
Can Unannounced Standardized Patients
Assess Professionalism and Communication
Skills in the Emergency Department?
Sondra Zabar, MD, Tavinder Ark, MSc, Colleen Gillespie, PhD, Amy Hsieh, MPA, Adina Kalet, MD,
Elizabeth Kachur, PhD, Jeffrey Manko, MD, and Linda Regan, MD


Abstract
           Objectives: The authors piloted unannounced standardized patients (USPs) in an emergency medicine
           (EM) residency to test feasibility, acceptability, and performance assessment of professionalism and com-
           munication skills.
           Methods: Fifteen postgraduate year (PGY)-2 EM residents were scheduled to be visited by two USPs
           while working in the emergency department (ED). Multidisciplinary support was utilized to ensure suc-
           cessful USP introduction. Scores (% well done) were calculated for communication and professionalism
           skills using a 26-item, behaviorally anchored checklist. Residents’ attitudes toward USPs and USP detec-
           tion were also surveyed.
           Results: Of 27 USP encounters attempted, 17 (62%) were successfully completed. The detection rate was
           44%. Eighty-three percent of residents who encountered a USP felt that the encounter did not hinder
           daily practice and did not make them uncomfortable (86%) or suspicious of patients (71%). Overall, resi-
           dents received a mean score of 60% for communication items rated ‘‘well done’’ (SD ± 28%,
           range = 23%–100%) and 53% of professionalism items ‘‘well done’’ (SD ± 20%, range = 23%-85%). Resi-
           dents’ communication skills were weakest for patient education and counseling (mean = 43%,
           SD ± 31%), compared with information gathering (68%, SD ± 36% and relationship development (62%,
           SD ± 32%). Scores of residents who detected USPs did not differ from those who had not.
           Conclusions: Implementing USPs in the ED is feasible and acceptable to staff. The unpredictability of
           the ED, specifically resident schedules, accounted for most incomplete encounters. USPs may represent
           a new way to assess real-time resident physician performance without the need for faculty resources or
           the bias introduced by direct observation.
           ACADEMIC EMERGENCY MEDICINE 2009; 16:915–918 ª 2009 by the Society for Academic Emergency
           Medicine
           Keywords: standardized patients, graduate medical education, assessment, OSCE, professionalism,
           communication, assessment




W
            hat options exist for assessing communica-           Accreditation Council for Graduate Medical Education
            tion and professionalism skills? As resi-            (ACGME) Outcomes Project,1 robust modalities to eval-
            dency programs seek to comply with the               uate clinical performance and effectiveness of educa-
                                                                 tion are in high demand. The ACGME’s Toolbox
From the New York University School of Medicine (SZ, TA,         contains numerous tools for assessing communication
CG, AH, AK, EK, JM, LR), New York, NY; and The Johns             skills,2 but many of these rely on self-assessment or
Hopkins University School of Medicine (LR), Baltimore, MD.       trained observers present during patient encounters.
Received February 27, 2009; revision received May 21, 2009;      Patient complaints and postvisit surveys are useful for
accepted May 22, 2009.                                           obtaining information, but offer limited opportunity
Presented at The Gold Foundation Symposium, ‘‘How Are We         for physicians to translate feedback into practice
Teaching Humanism in Medicine and What is Working?’’             change. Unannounced standardized patients (USPs)3–5
September 27–29, 2007, Chicago, IL; and the 9th Annual Inter-    present a method of measuring physicians’ communi-
national Meeting on Simulation in Healthcare (IMSH), January     cation and professionalism skills in a real practice set-
10–14, 2009, Lake Buena Vista, FL.                               ting without the artificiality inherent in observed
Supported by Picker Institute Challenge Grant 2007.              structured clinical exams (OSCEs).6–8
Address for correspondence and reprints: Sondra Zabar, MD;          We hypothesized that USPs can provide a real-time,
e-mail: sondra.zabar@nyumc.org.                                  accurate alternative to direct observation and OSCEs.




ª 2009 by the Society for Academic Emergency Medicine                                                  ISSN 1069-6563
doi: 10.1111/j.1553-2712.2009.00510.x                                                           PII ISSN 1069-6563583   915
916                                                                  Zabar et al.   •   PROFESSIONALISM AND COMMUNICATION


The purpose of this project was to 1) describe the            tion, 6) review and practice with rating checklist, and
process of conducting a USP program in an emergency           7) preparatory observational visit to the ED. Actors
department (ED), 2) determine if implementing USPs in         were compensated at a rate of $25 ⁄ hour for both train-
the ED is feasible, and 3) present preliminary results of     ing and in-ED time.
a USP performance assessment.
                                                              USP Encounter. Unannounced standardized patients
METHODS                                                       met project staff while residents attended a required
                                                              conference. USPs were introduced to the triage nurse,
Study Design                                                  the MR administrator, and the attending. The USPs
This was a prospective, nonrandomized, cohort study           were triaged per standard procedure.
to assess professionalism and communication abilities           During the encounter, USPs complied with any (non-
of emergency medicine (EM) residents using USPs.              invasive) exam and accepted all appointments and
Informed consent was obtained from all participants.          prescriptions, which were canceled postencounter. If
Research activities in this study were approved by the        the resident insisted on any course of action that made
New York University School of Medicine Institutional          the USP feel unsafe, the USP was to ask for the attending,
Review Board through a resident registry wherein resi-        send a short message service (SMS) text message to
dents are asked to consent to allow inclusion of their        project staff, or simply leave the ED. Hospital billing
educational and performance data in a research data-          canceled the visit at the end of the day. Total time in
base. Data, therefore, are reported only for those resi-      the ED was 1.5 to 4 hours ⁄ visit. Immediately following
dents for whom such consent was obtained.                     the encounter, the USP debriefed and completed a
                                                              behaviorally anchored checklist that assessed resident
Study Setting and Population                                  skills and the USP’s satisfaction with the visit.
The Bellevue ED is a busy Level 1 trauma center at an
academic medical center in New York City. The ED sees         Post-USP Survey. At the end of the project, all EM
approximately 100,000 visits per year.                        residents (including those who did not see a USP;
  Fifteen EM residents in their second year of post-          n = 30) were surveyed about their attitudes toward
graduate training (PGY-2) participated in the EM Pro-         USPs using a four-point scale (1 = strongly disagree,
fessionalism and Communication Training (EMPACT)              4 = strongly agree) and open-ended questions. To
Program. At the conclusion of the EMPACT training,            determine detection rates, residents were asked if they
residents were informed that they might be visited by         had encountered a USP and if so to identify the USP’s
USPs during their subsequent time working in the              sex and chief complaint.
ED. However, residents were blinded as to the exact
date of the visit or patient complaint.                       Data Analysis
                                                              Unannounced standardized patients assessed residents’
Study Protocol                                                professionalism and communication skills and their
Logistics. We required involvement from most ED               satisfaction with the patient-centeredness9,10 of the
staff areas including nurses, attending physicians, medi-     visit using a three-point scale: ‘‘not done,’’ ‘‘partially
cal records (MRs), registration, informatics, and radio-      done,’’ and ‘‘well done.’’ Scores were calculated as the
logy. To ensure fidelity for each USP visit, we created a      percentage of well-done items (Table 1). Professional-
preexisting MR with a unique number, patient name             ism and communication skills were scored from 13
and identifying information, prior visits, and test           items and patient centeredness from eight items. Over-
results. Each resident was scheduled to receive two           all recommendation ratings were obtained using a
USPs in urgent care (where residents’ schedules were          four-point scale. Reliability estimates (Cronbach’s
relatively predictable) during the 4 to 6 weeks after the     alpha ‘t’) and descriptive statistics (means, standard
EMPACT curriculum.                                            deviations [SDs], and ranges) are reported. Correla-
                                                              tions (Pearson’s r) between scores earned in the two
USP Scenarios. We used two USP cases previously               separate cases are also reported to assess stability of
validated in OSCEs, representing common ED chal-              performance.
lenges and requiring only communication-based inter-
ventions. In the first case (a misread x-ray), residents
                                                              RESULTS
needed to educate an angry patient recalled for a mis-
read x-ray (skills: delivering bad news, dealing with a       Seventeen of 27 visits were successfully conducted and
challenging patient, accountability), and in the second       evaluated. Resident scheduling problems explained
(a repeat visitor), care for a dissatisfied patient with       most incomplete encounters. Five residents were visited
chronic pain who repeatedly uses the ED (skills: han-         by USPs from both cases, and seven residents from one
dling emotion, patient education, accountability).            case.

USP Training. Eight actors were recruited. On aver-           USP Detection
age, each received seven hours of training consisting of      Seven of 12 residents who encountered a USP pro-
1) discussion of character and situation, 2) calibration      vided information on detection; four of their nine
of emotional tone, 3) role play for standardization,          encounters were detected (44% detection rate). Five of
4) practice with attending and chief residents for realism,   18 residents who did not see a USP indicated that they
5) review of ‘‘ground rules’’ for safety and nondetec-        did (28% false-positive rate). One of the residents who
ACAD EMERG MED • September 2009, Vol. 16, No. 9    •   www.aemj.org                                                              917


Table 1
Resident Performance with USPs


 Domains of Assessment                                     Items                      Mean, %    ±SD, %    Range, %    Reliability
 Communication*                                                                         60         28       23–100        0.91
 Information gathering              Used appropriate questions                          68         36        0–100        0.82
                                    Clarified information
                                    Allowed to talk without interrupting
 Relationship development           Communicated concern                                62         32       20–100        0.85
                                    Nonverbal enhanced communication
                                    Acknowledged emotions
                                    Was accepting ⁄ nonjudgmental
                                    Used words you understood
 Education and counseling           Asked questions to see what you understood          43         31         0–100       0.78
                                    Provided clear explanations
                                    Collaborated with you in identifying next steps
 Professionalism*                                                                       53         20       23–85         0.62
 Accountability                     Disclosed error                                     49         23        0–80         0.60
                                    Personally apologized
                                    Took responsibility for situation
 Manage difficult situation          Avoided assigning blame                             91         16       60–100        0.85
                                    Maintained professionalism
 Giving bad news                    Prepared you to receive news                        42         34         0–83        0.63
                                    Gave you opportunity to emotionally respond
                                    Provided appropriate next steps
 Treatment plan and management      Assessed resources                                  50         39         0–100       0.66
                                    Arranged for follow-up
                                    Discussed plan
 Patient Centeredness*              Fully explored my experience                        43         29         0–75        0.91
                                    Explored my expectations
                                    Came to an agreement
                                    Took a personal interest in me
                                    Earned (regained) my trust
                                    Acknowledged impact of error
                                    Didn’t make me feel wasting time
                                    I was given enough information
 Recommendation                                                                          2.3        0.9     1.0–3.5       0.90

 n = 12 residents, 17 visits; reliability assessed with Cronbach’s alpha. Four-point scale: 1 = not recommend; 2 = recommend with
 reservations; 3 = recommend; 4 = highly recommend.
 USPs = unannounced standardized patients.
 *Mean percentage of items rated as ‘‘well done’’ (not, partly, or well done).
  ’’Would you recommend this physician to a family member of friend?’’




reported a false detection reported ignoring that                     think more (29%), or led them to be more self-aware
patient.                                                              (43%).

USP Performance                                                       DISCUSSION
The reliability of scores (Table 1) suggests adequate
internal consistency (a > 0.60). Residents performed                  Our results show that developing and implementing a
better in the misread x-ray case than in the repeat                   USP program in the ED is feasible and acceptable to
visitor case in professionalism (70% vs. 35%, t = 2.81,               residents. Considering the drawbacks of OSCE assess-
p = 0.048) and patient-centeredness (66% vs. 40%,                     ment and direct observation, combined with increasing
t = 1.96, p = 0.05). Communication (r = 0.73, p = 0.16)               demands on faculty time and decreasing funding, USPs
and recommendation scores (r = 0.81, p = 0.09) were                   may offer an objective, cost-effective method for evalu-
highly, albeit not significantly, correlated between the               ating accurate practice skills.
two cases, but professionalism (r = 0.24, p = 0.70), and                 The biggest challenge faced while implementing the
patient-centeredness were not (r = 0.08, p = 0.90), sug-              USP program was the unpredictability of the ED. Occa-
gesting case content matters most in these domains.                   sionally, USPs were mistakenly examined by another
                                                                      resident. Both content (highly trained SP, realistic
Postevaluation Survey                                                 cases) and logistic factors (dedicated program coordi-
Eighty-three percent of residents who encountered a                   nator, electronic MRs, team collaboration) are neces-
USP felt that it did not hinder their daily practice                  sary for successful integration. Total cost, in terms of
and did not make them uncomfortable (86%) or sus-                     both time and money, is likely greater up front, with
picious of patients (71%). A minority of those resi-                  decreased workload, time, and expense as USPs and
dents who encountered a USP felt that the encounter                   staff become trained. Further study of the costs is
improved their practice behavior (14%), made them                     needed.
918                                                                   Zabar et al.   •   PROFESSIONALISM AND COMMUNICATION


   Even with a high detection rate, residents reported             at: http://www.acgme.org/outcome/comp/compCPRL.
value in the USP program for learning and patient care.            asp. Accessed Sep 20, 2008.
It is possible that informing residents that USPs would       2.   Accreditation Council for Graduate Medical Educa-
be visiting them in the ED improved performance.                   tion, American Board of Medical Specialties.
More importantly, the majority of residents did not feel           Outcome Project Toolbox of Assessment Methods.
that the possibility of encountering a USP had any neg-            Available at: http://www.acgme.org/outcome/assess/
ative impact on their daily practice, suggesting that              toolbox.asp. Accessed Jun 20, 2009.
USPs in the ED will not risk real patient safety. The         3.   Gorter S, Scherpbier A, Brauer J, et al. Doctor-
case of the resident who reported ignoring a patient               patient interaction: standardized patients’ reflec-
thought to be ‘‘unannounced’’ represents an unantici-              tions from inside the rheumatological office.
pated and anomalous professionalism issue, we believe,             J Rheumatol. 2002; 29(7):1496–500.
not causally related to the use of USPs; it demonstrates      4.   Kravitz RL, Epstein RM, Feldman MD, et al. Influ-
how USPs can provide useful information to program                 ence of patients’ requests for direct-to-customer
directors.                                                         advertised antidepressants: a randomized controlled
                                                                   trial. JAMA. 2005; 293(16):1995–2002.
LIMITATIONS                                                   5.   Ozuah PO, Reznik M. Using unannounced standard-
                                                                   ised patients to assess residents’ professionalism.
There was a small sample size, with a relatively large             Med Educ. 2008; 42(5):532–3.
proportion of failed USP visits. However, the failure         6.   Fiscella K, Franks P, Srinivasan M, Kravitz RL,
rate improved as the project progressed. Even with our             Epstein R. Ratings of physician communication by
small numbers, it appears that two cases and the items             real and standardized patients. Ann Fam Med. 2007;
on the behaviorally anchored checklist can discriminate            5(2):151–8.
residents based on their communication skills.                7.   Talente G, Haist SA, Wilson JF. The relationship
                                                                   between experience with standardized patient
CONCLUSIONS                                                        examinations and subsequent standardized patient
                                                                   examination performance: A potential problem with
With the ACGME placing greater importance on evalua-               standardized patient exam validity. Eval Health Prof.
tion of patient outcomes, we believe that our project              2007; 30(1):64–74.
represents a new way to assess real-time resident perfor-     8.   Srinivasan M, Franks P, Meredith LS, Fiscella K,
mance. Despite being time-consuming and subject to the             Epstein RM, Kravitz RL. Connoisseurs of care?
unpredictability of the ED, implementing unannounced               unannounced standardized patients’ ratings of phy-
standardized patients in the ED is feasible and acceptable         sicians. Med Care. 2006; 44(12):1092–8.
to staff. Future comparison of unannounced standard-          9.   Marshall GN, Hays RD. The Patient Satisfaction
ized patients with observed structured clinical exam               Questionnaire Short Form (PSQ-18). RAND Corpo-
scores will enable educators to determine how well these           ration, Paper P-7865. Available at: http://www.
methods assess performance in actual practice.                     rand.org/pubs/papers/P7865/. Accessed Jun 20, 2009.
                                                             10.   Elwyn G, Edwards A, Wensing M, Hood K, Atwell
References                                                         C, Grol R. Shared decision making: developing the
                                                                   OPTION scale for measuring patient involvement.
 1. Accreditation Council for Graduate Medical Educa-              Qual Saf Health Care. 2003; 12:93–9.
    tion (ACGME). ACGME Outcome Project. Available

Mais conteúdo relacionado

Mais procurados

Changes in quality of life among jordanian colorectal cancer patients a quali...
Changes in quality of life among jordanian colorectal cancer patients a quali...Changes in quality of life among jordanian colorectal cancer patients a quali...
Changes in quality of life among jordanian colorectal cancer patients a quali...Alexander Decker
 
Therapeutic Development Paths for Cognitive Impairment in Parkinson's Disease...
Therapeutic Development Paths for Cognitive Impairment in Parkinson's Disease...Therapeutic Development Paths for Cognitive Impairment in Parkinson's Disease...
Therapeutic Development Paths for Cognitive Impairment in Parkinson's Disease...Lona Vincent
 
SICU Critical Care Safety Study
SICU Critical Care Safety StudySICU Critical Care Safety Study
SICU Critical Care Safety Studyshivabirdi
 
Recomendaciones cdc 2011 infecciones por cateteres
Recomendaciones cdc 2011 infecciones por cateteresRecomendaciones cdc 2011 infecciones por cateteres
Recomendaciones cdc 2011 infecciones por cateteresInes Tudela Sanjuan
 
PDAs for Nursing Students: Technology at Your Fingertips
PDAs for Nursing Students: Technology at Your FingertipsPDAs for Nursing Students: Technology at Your Fingertips
PDAs for Nursing Students: Technology at Your FingertipsCynthia.Russell
 
End of life care, ICU framework
End of life care, ICU frameworkEnd of life care, ICU framework
End of life care, ICU frameworkpbsherren
 
IRJET- Hospital Admission Prediction: A Technology Survey
IRJET- Hospital Admission Prediction: A Technology SurveyIRJET- Hospital Admission Prediction: A Technology Survey
IRJET- Hospital Admission Prediction: A Technology SurveyIRJET Journal
 
Evaluacion preanestesica taskforce 2002
Evaluacion preanestesica taskforce 2002Evaluacion preanestesica taskforce 2002
Evaluacion preanestesica taskforce 2002sandoriver
 
Career choice influences in Indian Anaesthesiologists
Career choice influences in Indian AnaesthesiologistsCareer choice influences in Indian Anaesthesiologists
Career choice influences in Indian AnaesthesiologistsArun Gupta
 
PDSA - Front Board Rev - OFC
PDSA - Front Board Rev - OFCPDSA - Front Board Rev - OFC
PDSA - Front Board Rev - OFCalfred lopez
 
vias venosas difíciles en emergencias.
vias venosas difíciles en emergencias.vias venosas difíciles en emergencias.
vias venosas difíciles en emergencias.Marco Darby
 

Mais procurados (20)

159th publication jamdsr- 3rd name
159th publication  jamdsr- 3rd name159th publication  jamdsr- 3rd name
159th publication jamdsr- 3rd name
 
Changes in quality of life among jordanian colorectal cancer patients a quali...
Changes in quality of life among jordanian colorectal cancer patients a quali...Changes in quality of life among jordanian colorectal cancer patients a quali...
Changes in quality of life among jordanian colorectal cancer patients a quali...
 
SIEDSpaper2016
SIEDSpaper2016SIEDSpaper2016
SIEDSpaper2016
 
Therapeutic Development Paths for Cognitive Impairment in Parkinson's Disease...
Therapeutic Development Paths for Cognitive Impairment in Parkinson's Disease...Therapeutic Development Paths for Cognitive Impairment in Parkinson's Disease...
Therapeutic Development Paths for Cognitive Impairment in Parkinson's Disease...
 
20150300.0 00014
20150300.0 0001420150300.0 00014
20150300.0 00014
 
Parkinsons Research
Parkinsons ResearchParkinsons Research
Parkinsons Research
 
SICU Critical Care Safety Study
SICU Critical Care Safety StudySICU Critical Care Safety Study
SICU Critical Care Safety Study
 
Surgical informed consent
Surgical informed consentSurgical informed consent
Surgical informed consent
 
Recomendaciones cdc 2011 infecciones por cateteres
Recomendaciones cdc 2011 infecciones por cateteresRecomendaciones cdc 2011 infecciones por cateteres
Recomendaciones cdc 2011 infecciones por cateteres
 
Final-PICO-Poster
Final-PICO-PosterFinal-PICO-Poster
Final-PICO-Poster
 
PDAs for Nursing Students: Technology at Your Fingertips
PDAs for Nursing Students: Technology at Your FingertipsPDAs for Nursing Students: Technology at Your Fingertips
PDAs for Nursing Students: Technology at Your Fingertips
 
Medical Record for DRG/CBG Coding Purpose
Medical Record for DRG/CBG Coding PurposeMedical Record for DRG/CBG Coding Purpose
Medical Record for DRG/CBG Coding Purpose
 
End of life care, ICU framework
End of life care, ICU frameworkEnd of life care, ICU framework
End of life care, ICU framework
 
Randomization, Bias, Blinding
Randomization, Bias, Blinding Randomization, Bias, Blinding
Randomization, Bias, Blinding
 
IRJET- Hospital Admission Prediction: A Technology Survey
IRJET- Hospital Admission Prediction: A Technology SurveyIRJET- Hospital Admission Prediction: A Technology Survey
IRJET- Hospital Admission Prediction: A Technology Survey
 
COSM v2
COSM v2COSM v2
COSM v2
 
Evaluacion preanestesica taskforce 2002
Evaluacion preanestesica taskforce 2002Evaluacion preanestesica taskforce 2002
Evaluacion preanestesica taskforce 2002
 
Career choice influences in Indian Anaesthesiologists
Career choice influences in Indian AnaesthesiologistsCareer choice influences in Indian Anaesthesiologists
Career choice influences in Indian Anaesthesiologists
 
PDSA - Front Board Rev - OFC
PDSA - Front Board Rev - OFCPDSA - Front Board Rev - OFC
PDSA - Front Board Rev - OFC
 
vias venosas difíciles en emergencias.
vias venosas difíciles en emergencias.vias venosas difíciles en emergencias.
vias venosas difíciles en emergencias.
 

Destaque

Resident Performance from the Patient's View: Richard Wardrop, MD, PhD, FAAP
Resident Performance from the Patient's View: Richard Wardrop, MD, PhD, FAAPResident Performance from the Patient's View: Richard Wardrop, MD, PhD, FAAP
Resident Performance from the Patient's View: Richard Wardrop, MD, PhD, FAAPPicker Institute, Inc.
 
Improving Patient Rounds (IPR): Medical College of Georgia/Georgia Health
Improving Patient Rounds (IPR): Medical College of Georgia/Georgia HealthImproving Patient Rounds (IPR): Medical College of Georgia/Georgia Health
Improving Patient Rounds (IPR): Medical College of Georgia/Georgia HealthPicker Institute, Inc.
 
Integrating Patient- and Family-Centered Care Principles into a Simulation-Ba...
Integrating Patient- and Family-Centered Care Principles into a Simulation-Ba...Integrating Patient- and Family-Centered Care Principles into a Simulation-Ba...
Integrating Patient- and Family-Centered Care Principles into a Simulation-Ba...Picker Institute, Inc.
 
How to have the conversation: Dementia Training Module
How to have the conversation: Dementia Training ModuleHow to have the conversation: Dementia Training Module
How to have the conversation: Dementia Training ModulePicker Institute, Inc.
 

Destaque (11)

Resident Performance from the Patient's View: Richard Wardrop, MD, PhD, FAAP
Resident Performance from the Patient's View: Richard Wardrop, MD, PhD, FAAPResident Performance from the Patient's View: Richard Wardrop, MD, PhD, FAAP
Resident Performance from the Patient's View: Richard Wardrop, MD, PhD, FAAP
 
Zabar final report cg
Zabar final report cgZabar final report cg
Zabar final report cg
 
Lupus a
Lupus aLupus a
Lupus a
 
HoMeS: Home Medication Support
HoMeS: Home Medication SupportHoMeS: Home Medication Support
HoMeS: Home Medication Support
 
Training Residents and Nurses as Patient-Centered Care Teams by Smith, Dwamen...
Training Residents and Nurses as Patient-Centered Care Teams by Smith, Dwamen...Training Residents and Nurses as Patient-Centered Care Teams by Smith, Dwamen...
Training Residents and Nurses as Patient-Centered Care Teams by Smith, Dwamen...
 
Improving Patient Rounds (IPR): Medical College of Georgia/Georgia Health
Improving Patient Rounds (IPR): Medical College of Georgia/Georgia HealthImproving Patient Rounds (IPR): Medical College of Georgia/Georgia Health
Improving Patient Rounds (IPR): Medical College of Georgia/Georgia Health
 
Slide set for pothol es always events
Slide set for pothol es always eventsSlide set for pothol es always events
Slide set for pothol es always events
 
Always events patien ts
Always events patien tsAlways events patien ts
Always events patien ts
 
Always events pothol es
Always events pothol esAlways events pothol es
Always events pothol es
 
Integrating Patient- and Family-Centered Care Principles into a Simulation-Ba...
Integrating Patient- and Family-Centered Care Principles into a Simulation-Ba...Integrating Patient- and Family-Centered Care Principles into a Simulation-Ba...
Integrating Patient- and Family-Centered Care Principles into a Simulation-Ba...
 
How to have the conversation: Dementia Training Module
How to have the conversation: Dementia Training ModuleHow to have the conversation: Dementia Training Module
How to have the conversation: Dementia Training Module
 

Semelhante a EMPACT: Emergency Medicine Professionalism and Communication Training

Assessment Of Communication And Interpersonal Skills Competencies
Assessment Of Communication And Interpersonal Skills CompetenciesAssessment Of Communication And Interpersonal Skills Competencies
Assessment Of Communication And Interpersonal Skills CompetenciesRick Vogel
 
CGEA2011Patient Centeredness
CGEA2011Patient CenterednessCGEA2011Patient Centeredness
CGEA2011Patient CenterednessAlison Martin
 
Improving Timeliness and Quality: Discharge Summaries Dictated by Internal Me...
Improving Timeliness and Quality: Discharge Summaries Dictated by Internal Me...Improving Timeliness and Quality: Discharge Summaries Dictated by Internal Me...
Improving Timeliness and Quality: Discharge Summaries Dictated by Internal Me...emallin
 
An emergency department quality improvement project
An emergency department quality improvement projectAn emergency department quality improvement project
An emergency department quality improvement projectyasmeenzulfiqar
 
Course 2 the need for a careful and thorough history
Course 2 the need for a careful and thorough historyCourse 2 the need for a careful and thorough history
Course 2 the need for a careful and thorough historyNelson Hendler
 
BJHMR-12476.pdf
BJHMR-12476.pdfBJHMR-12476.pdf
BJHMR-12476.pdfhalay
 
BJHMR-12476.pdf
BJHMR-12476.pdfBJHMR-12476.pdf
BJHMR-12476.pdfhalay
 
final project (nursing major) najah university
final project (nursing major) najah universityfinal project (nursing major) najah university
final project (nursing major) najah universitymahdyvika
 
Appraisal Paper Educ Prim Care 2010 pp445-54
Appraisal Paper Educ Prim Care 2010 pp445-54Appraisal Paper Educ Prim Care 2010 pp445-54
Appraisal Paper Educ Prim Care 2010 pp445-54Mark Rickenbach
 
Conducting behavioral-intervention-research-rural-communities-nhc-16-101
Conducting behavioral-intervention-research-rural-communities-nhc-16-101Conducting behavioral-intervention-research-rural-communities-nhc-16-101
Conducting behavioral-intervention-research-rural-communities-nhc-16-101dynajolly
 

Semelhante a EMPACT: Emergency Medicine Professionalism and Communication Training (20)

Assessment Of Communication And Interpersonal Skills Competencies
Assessment Of Communication And Interpersonal Skills CompetenciesAssessment Of Communication And Interpersonal Skills Competencies
Assessment Of Communication And Interpersonal Skills Competencies
 
manajemen rumah sakit 5
manajemen rumah sakit 5manajemen rumah sakit 5
manajemen rumah sakit 5
 
My dessertation ppt
My  dessertation pptMy  dessertation ppt
My dessertation ppt
 
Changing trends in the medical examinations
Changing trends in the medical examinationsChanging trends in the medical examinations
Changing trends in the medical examinations
 
Nocebo in doctor patient communication
Nocebo in doctor patient communicationNocebo in doctor patient communication
Nocebo in doctor patient communication
 
Changing trends in the medical examinations
Changing trends in the medical examinationsChanging trends in the medical examinations
Changing trends in the medical examinations
 
CGEA2011Patient Centeredness
CGEA2011Patient CenterednessCGEA2011Patient Centeredness
CGEA2011Patient Centeredness
 
Improving Timeliness and Quality: Discharge Summaries Dictated by Internal Me...
Improving Timeliness and Quality: Discharge Summaries Dictated by Internal Me...Improving Timeliness and Quality: Discharge Summaries Dictated by Internal Me...
Improving Timeliness and Quality: Discharge Summaries Dictated by Internal Me...
 
An emergency department quality improvement project
An emergency department quality improvement projectAn emergency department quality improvement project
An emergency department quality improvement project
 
Bereaved Relatives
Bereaved Relatives Bereaved Relatives
Bereaved Relatives
 
Course 2 the need for a careful and thorough history
Course 2 the need for a careful and thorough historyCourse 2 the need for a careful and thorough history
Course 2 the need for a careful and thorough history
 
BJHMR-12476.pdf
BJHMR-12476.pdfBJHMR-12476.pdf
BJHMR-12476.pdf
 
BJHMR-12476.pdf
BJHMR-12476.pdfBJHMR-12476.pdf
BJHMR-12476.pdf
 
OSCE Poster_Final
OSCE Poster_FinalOSCE Poster_Final
OSCE Poster_Final
 
final project (nursing major) najah university
final project (nursing major) najah universityfinal project (nursing major) najah university
final project (nursing major) najah university
 
2005 2008
2005 20082005 2008
2005 2008
 
Patient satisfaction
Patient satisfactionPatient satisfaction
Patient satisfaction
 
Appraisal Paper Educ Prim Care 2010 pp445-54
Appraisal Paper Educ Prim Care 2010 pp445-54Appraisal Paper Educ Prim Care 2010 pp445-54
Appraisal Paper Educ Prim Care 2010 pp445-54
 
MNNELSON RESUME
MNNELSON RESUMEMNNELSON RESUME
MNNELSON RESUME
 
Conducting behavioral-intervention-research-rural-communities-nhc-16-101
Conducting behavioral-intervention-research-rural-communities-nhc-16-101Conducting behavioral-intervention-research-rural-communities-nhc-16-101
Conducting behavioral-intervention-research-rural-communities-nhc-16-101
 

Mais de Picker Institute, Inc.

Always Events Healthcare Solutions Book
Always Events Healthcare Solutions BookAlways Events Healthcare Solutions Book
Always Events Healthcare Solutions BookPicker Institute, Inc.
 
Always Use Teacch Back!- Iowa Health System Always Event
Always Use Teacch Back!- Iowa Health System Always Event Always Use Teacch Back!- Iowa Health System Always Event
Always Use Teacch Back!- Iowa Health System Always Event Picker Institute, Inc.
 
My Story- University of Minnesota Amplatz Children's Hospital: Always Event
My Story- University of Minnesota Amplatz Children's Hospital: Always EventMy Story- University of Minnesota Amplatz Children's Hospital: Always Event
My Story- University of Minnesota Amplatz Children's Hospital: Always EventPicker Institute, Inc.
 
Dartmouth hitchcock ihi storyboard final
Dartmouth hitchcock ihi storyboard finalDartmouth hitchcock ihi storyboard final
Dartmouth hitchcock ihi storyboard finalPicker Institute, Inc.
 
Trauma Team Texting and Guardian Angels: UPMC Always Events
Trauma Team Texting and Guardian Angels: UPMC Always EventsTrauma Team Texting and Guardian Angels: UPMC Always Events
Trauma Team Texting and Guardian Angels: UPMC Always EventsPicker Institute, Inc.
 
Same Page Transitional Care- Planetree Always Event
Same Page Transitional Care- Planetree Always EventSame Page Transitional Care- Planetree Always Event
Same Page Transitional Care- Planetree Always EventPicker Institute, Inc.
 
Always Events Recognition Program Application and Overview
Always Events Recognition Program Application and OverviewAlways Events Recognition Program Application and Overview
Always Events Recognition Program Application and OverviewPicker Institute, Inc.
 
University of Minnesota Amplatz Children's Hospital Always Event: My Story
University of Minnesota Amplatz Children's Hospital Always Event: My StoryUniversity of Minnesota Amplatz Children's Hospital Always Event: My Story
University of Minnesota Amplatz Children's Hospital Always Event: My StoryPicker Institute, Inc.
 
University of Maryland Graduate Medical Education Always Events Poster Presen...
University of Maryland Graduate Medical Education Always Events Poster Presen...University of Maryland Graduate Medical Education Always Events Poster Presen...
University of Maryland Graduate Medical Education Always Events Poster Presen...Picker Institute, Inc.
 
Exempla Saint Joseph Always Events Poster Presentation
Exempla Saint Joseph Always Events Poster PresentationExempla Saint Joseph Always Events Poster Presentation
Exempla Saint Joseph Always Events Poster PresentationPicker Institute, Inc.
 
Wake Forest Graduate Medical Education Always Events Poster Presentation
Wake Forest Graduate Medical Education Always Events Poster PresentationWake Forest Graduate Medical Education Always Events Poster Presentation
Wake Forest Graduate Medical Education Always Events Poster PresentationPicker Institute, Inc.
 
Henry Ford Health System Always Events
Henry Ford Health System Always EventsHenry Ford Health System Always Events
Henry Ford Health System Always EventsPicker Institute, Inc.
 
Picker Institute/Gold Foundation 2012-2013 Graduate Medical Education RFP
Picker Institute/Gold Foundation 2012-2013 Graduate Medical Education RFPPicker Institute/Gold Foundation 2012-2013 Graduate Medical Education RFP
Picker Institute/Gold Foundation 2012-2013 Graduate Medical Education RFPPicker Institute, Inc.
 

Mais de Picker Institute, Inc. (20)

Always Events Program Application
Always Events Program ApplicationAlways Events Program Application
Always Events Program Application
 
Always Events Blueprint for Action
Always Events Blueprint for ActionAlways Events Blueprint for Action
Always Events Blueprint for Action
 
Always Events Healthcare Solutions Book
Always Events Healthcare Solutions BookAlways Events Healthcare Solutions Book
Always Events Healthcare Solutions Book
 
Always Use Teacch Back!- Iowa Health System Always Event
Always Use Teacch Back!- Iowa Health System Always Event Always Use Teacch Back!- Iowa Health System Always Event
Always Use Teacch Back!- Iowa Health System Always Event
 
My Story- University of Minnesota Amplatz Children's Hospital: Always Event
My Story- University of Minnesota Amplatz Children's Hospital: Always EventMy Story- University of Minnesota Amplatz Children's Hospital: Always Event
My Story- University of Minnesota Amplatz Children's Hospital: Always Event
 
Dartmouth hitchcock ihi storyboard final
Dartmouth hitchcock ihi storyboard finalDartmouth hitchcock ihi storyboard final
Dartmouth hitchcock ihi storyboard final
 
Trauma Team Texting and Guardian Angels: UPMC Always Events
Trauma Team Texting and Guardian Angels: UPMC Always EventsTrauma Team Texting and Guardian Angels: UPMC Always Events
Trauma Team Texting and Guardian Angels: UPMC Always Events
 
Same Page Transitional Care- Planetree Always Event
Same Page Transitional Care- Planetree Always EventSame Page Transitional Care- Planetree Always Event
Same Page Transitional Care- Planetree Always Event
 
Picker poster proof ihi 10 11-12
Picker poster proof ihi 10 11-12 Picker poster proof ihi 10 11-12
Picker poster proof ihi 10 11-12
 
Ihi storyboard 9 21-12
Ihi storyboard 9 21-12Ihi storyboard 9 21-12
Ihi storyboard 9 21-12
 
Always Events Recognition Program Application and Overview
Always Events Recognition Program Application and OverviewAlways Events Recognition Program Application and Overview
Always Events Recognition Program Application and Overview
 
MyStory Tool
MyStory ToolMyStory Tool
MyStory Tool
 
University of Minnesota Amplatz Children's Hospital Always Event: My Story
University of Minnesota Amplatz Children's Hospital Always Event: My StoryUniversity of Minnesota Amplatz Children's Hospital Always Event: My Story
University of Minnesota Amplatz Children's Hospital Always Event: My Story
 
University of Maryland Graduate Medical Education Always Events Poster Presen...
University of Maryland Graduate Medical Education Always Events Poster Presen...University of Maryland Graduate Medical Education Always Events Poster Presen...
University of Maryland Graduate Medical Education Always Events Poster Presen...
 
Exempla Saint Joseph Always Events Poster Presentation
Exempla Saint Joseph Always Events Poster PresentationExempla Saint Joseph Always Events Poster Presentation
Exempla Saint Joseph Always Events Poster Presentation
 
Wake Forest Graduate Medical Education Always Events Poster Presentation
Wake Forest Graduate Medical Education Always Events Poster PresentationWake Forest Graduate Medical Education Always Events Poster Presentation
Wake Forest Graduate Medical Education Always Events Poster Presentation
 
Henry Ford Health System Always Events
Henry Ford Health System Always EventsHenry Ford Health System Always Events
Henry Ford Health System Always Events
 
Informed Consent powerpoint
Informed Consent powerpointInformed Consent powerpoint
Informed Consent powerpoint
 
Sharing Bad News powerpoint
Sharing Bad News powerpointSharing Bad News powerpoint
Sharing Bad News powerpoint
 
Picker Institute/Gold Foundation 2012-2013 Graduate Medical Education RFP
Picker Institute/Gold Foundation 2012-2013 Graduate Medical Education RFPPicker Institute/Gold Foundation 2012-2013 Graduate Medical Education RFP
Picker Institute/Gold Foundation 2012-2013 Graduate Medical Education RFP
 

EMPACT: Emergency Medicine Professionalism and Communication Training

  • 1. Can Unannounced Standardized Patients Assess Professionalism and Communication Skills in the Emergency Department? Sondra Zabar, MD, Tavinder Ark, MSc, Colleen Gillespie, PhD, Amy Hsieh, MPA, Adina Kalet, MD, Elizabeth Kachur, PhD, Jeffrey Manko, MD, and Linda Regan, MD Abstract Objectives: The authors piloted unannounced standardized patients (USPs) in an emergency medicine (EM) residency to test feasibility, acceptability, and performance assessment of professionalism and com- munication skills. Methods: Fifteen postgraduate year (PGY)-2 EM residents were scheduled to be visited by two USPs while working in the emergency department (ED). Multidisciplinary support was utilized to ensure suc- cessful USP introduction. Scores (% well done) were calculated for communication and professionalism skills using a 26-item, behaviorally anchored checklist. Residents’ attitudes toward USPs and USP detec- tion were also surveyed. Results: Of 27 USP encounters attempted, 17 (62%) were successfully completed. The detection rate was 44%. Eighty-three percent of residents who encountered a USP felt that the encounter did not hinder daily practice and did not make them uncomfortable (86%) or suspicious of patients (71%). Overall, resi- dents received a mean score of 60% for communication items rated ‘‘well done’’ (SD ± 28%, range = 23%–100%) and 53% of professionalism items ‘‘well done’’ (SD ± 20%, range = 23%-85%). Resi- dents’ communication skills were weakest for patient education and counseling (mean = 43%, SD ± 31%), compared with information gathering (68%, SD ± 36% and relationship development (62%, SD ± 32%). Scores of residents who detected USPs did not differ from those who had not. Conclusions: Implementing USPs in the ED is feasible and acceptable to staff. The unpredictability of the ED, specifically resident schedules, accounted for most incomplete encounters. USPs may represent a new way to assess real-time resident physician performance without the need for faculty resources or the bias introduced by direct observation. ACADEMIC EMERGENCY MEDICINE 2009; 16:915–918 ª 2009 by the Society for Academic Emergency Medicine Keywords: standardized patients, graduate medical education, assessment, OSCE, professionalism, communication, assessment W hat options exist for assessing communica- Accreditation Council for Graduate Medical Education tion and professionalism skills? As resi- (ACGME) Outcomes Project,1 robust modalities to eval- dency programs seek to comply with the uate clinical performance and effectiveness of educa- tion are in high demand. The ACGME’s Toolbox From the New York University School of Medicine (SZ, TA, contains numerous tools for assessing communication CG, AH, AK, EK, JM, LR), New York, NY; and The Johns skills,2 but many of these rely on self-assessment or Hopkins University School of Medicine (LR), Baltimore, MD. trained observers present during patient encounters. Received February 27, 2009; revision received May 21, 2009; Patient complaints and postvisit surveys are useful for accepted May 22, 2009. obtaining information, but offer limited opportunity Presented at The Gold Foundation Symposium, ‘‘How Are We for physicians to translate feedback into practice Teaching Humanism in Medicine and What is Working?’’ change. Unannounced standardized patients (USPs)3–5 September 27–29, 2007, Chicago, IL; and the 9th Annual Inter- present a method of measuring physicians’ communi- national Meeting on Simulation in Healthcare (IMSH), January cation and professionalism skills in a real practice set- 10–14, 2009, Lake Buena Vista, FL. ting without the artificiality inherent in observed Supported by Picker Institute Challenge Grant 2007. structured clinical exams (OSCEs).6–8 Address for correspondence and reprints: Sondra Zabar, MD; We hypothesized that USPs can provide a real-time, e-mail: sondra.zabar@nyumc.org. accurate alternative to direct observation and OSCEs. ª 2009 by the Society for Academic Emergency Medicine ISSN 1069-6563 doi: 10.1111/j.1553-2712.2009.00510.x PII ISSN 1069-6563583 915
  • 2. 916 Zabar et al. • PROFESSIONALISM AND COMMUNICATION The purpose of this project was to 1) describe the tion, 6) review and practice with rating checklist, and process of conducting a USP program in an emergency 7) preparatory observational visit to the ED. Actors department (ED), 2) determine if implementing USPs in were compensated at a rate of $25 ⁄ hour for both train- the ED is feasible, and 3) present preliminary results of ing and in-ED time. a USP performance assessment. USP Encounter. Unannounced standardized patients METHODS met project staff while residents attended a required conference. USPs were introduced to the triage nurse, Study Design the MR administrator, and the attending. The USPs This was a prospective, nonrandomized, cohort study were triaged per standard procedure. to assess professionalism and communication abilities During the encounter, USPs complied with any (non- of emergency medicine (EM) residents using USPs. invasive) exam and accepted all appointments and Informed consent was obtained from all participants. prescriptions, which were canceled postencounter. If Research activities in this study were approved by the the resident insisted on any course of action that made New York University School of Medicine Institutional the USP feel unsafe, the USP was to ask for the attending, Review Board through a resident registry wherein resi- send a short message service (SMS) text message to dents are asked to consent to allow inclusion of their project staff, or simply leave the ED. Hospital billing educational and performance data in a research data- canceled the visit at the end of the day. Total time in base. Data, therefore, are reported only for those resi- the ED was 1.5 to 4 hours ⁄ visit. Immediately following dents for whom such consent was obtained. the encounter, the USP debriefed and completed a behaviorally anchored checklist that assessed resident Study Setting and Population skills and the USP’s satisfaction with the visit. The Bellevue ED is a busy Level 1 trauma center at an academic medical center in New York City. The ED sees Post-USP Survey. At the end of the project, all EM approximately 100,000 visits per year. residents (including those who did not see a USP; Fifteen EM residents in their second year of post- n = 30) were surveyed about their attitudes toward graduate training (PGY-2) participated in the EM Pro- USPs using a four-point scale (1 = strongly disagree, fessionalism and Communication Training (EMPACT) 4 = strongly agree) and open-ended questions. To Program. At the conclusion of the EMPACT training, determine detection rates, residents were asked if they residents were informed that they might be visited by had encountered a USP and if so to identify the USP’s USPs during their subsequent time working in the sex and chief complaint. ED. However, residents were blinded as to the exact date of the visit or patient complaint. Data Analysis Unannounced standardized patients assessed residents’ Study Protocol professionalism and communication skills and their Logistics. We required involvement from most ED satisfaction with the patient-centeredness9,10 of the staff areas including nurses, attending physicians, medi- visit using a three-point scale: ‘‘not done,’’ ‘‘partially cal records (MRs), registration, informatics, and radio- done,’’ and ‘‘well done.’’ Scores were calculated as the logy. To ensure fidelity for each USP visit, we created a percentage of well-done items (Table 1). Professional- preexisting MR with a unique number, patient name ism and communication skills were scored from 13 and identifying information, prior visits, and test items and patient centeredness from eight items. Over- results. Each resident was scheduled to receive two all recommendation ratings were obtained using a USPs in urgent care (where residents’ schedules were four-point scale. Reliability estimates (Cronbach’s relatively predictable) during the 4 to 6 weeks after the alpha ‘t’) and descriptive statistics (means, standard EMPACT curriculum. deviations [SDs], and ranges) are reported. Correla- tions (Pearson’s r) between scores earned in the two USP Scenarios. We used two USP cases previously separate cases are also reported to assess stability of validated in OSCEs, representing common ED chal- performance. lenges and requiring only communication-based inter- ventions. In the first case (a misread x-ray), residents RESULTS needed to educate an angry patient recalled for a mis- read x-ray (skills: delivering bad news, dealing with a Seventeen of 27 visits were successfully conducted and challenging patient, accountability), and in the second evaluated. Resident scheduling problems explained (a repeat visitor), care for a dissatisfied patient with most incomplete encounters. Five residents were visited chronic pain who repeatedly uses the ED (skills: han- by USPs from both cases, and seven residents from one dling emotion, patient education, accountability). case. USP Training. Eight actors were recruited. On aver- USP Detection age, each received seven hours of training consisting of Seven of 12 residents who encountered a USP pro- 1) discussion of character and situation, 2) calibration vided information on detection; four of their nine of emotional tone, 3) role play for standardization, encounters were detected (44% detection rate). Five of 4) practice with attending and chief residents for realism, 18 residents who did not see a USP indicated that they 5) review of ‘‘ground rules’’ for safety and nondetec- did (28% false-positive rate). One of the residents who
  • 3. ACAD EMERG MED • September 2009, Vol. 16, No. 9 • www.aemj.org 917 Table 1 Resident Performance with USPs Domains of Assessment Items Mean, % ±SD, % Range, % Reliability Communication* 60 28 23–100 0.91 Information gathering Used appropriate questions 68 36 0–100 0.82 Clarified information Allowed to talk without interrupting Relationship development Communicated concern 62 32 20–100 0.85 Nonverbal enhanced communication Acknowledged emotions Was accepting ⁄ nonjudgmental Used words you understood Education and counseling Asked questions to see what you understood 43 31 0–100 0.78 Provided clear explanations Collaborated with you in identifying next steps Professionalism* 53 20 23–85 0.62 Accountability Disclosed error 49 23 0–80 0.60 Personally apologized Took responsibility for situation Manage difficult situation Avoided assigning blame 91 16 60–100 0.85 Maintained professionalism Giving bad news Prepared you to receive news 42 34 0–83 0.63 Gave you opportunity to emotionally respond Provided appropriate next steps Treatment plan and management Assessed resources 50 39 0–100 0.66 Arranged for follow-up Discussed plan Patient Centeredness* Fully explored my experience 43 29 0–75 0.91 Explored my expectations Came to an agreement Took a personal interest in me Earned (regained) my trust Acknowledged impact of error Didn’t make me feel wasting time I was given enough information Recommendation  2.3 0.9 1.0–3.5 0.90 n = 12 residents, 17 visits; reliability assessed with Cronbach’s alpha. Four-point scale: 1 = not recommend; 2 = recommend with reservations; 3 = recommend; 4 = highly recommend. USPs = unannounced standardized patients. *Mean percentage of items rated as ‘‘well done’’ (not, partly, or well done).  ’’Would you recommend this physician to a family member of friend?’’ reported a false detection reported ignoring that think more (29%), or led them to be more self-aware patient. (43%). USP Performance DISCUSSION The reliability of scores (Table 1) suggests adequate internal consistency (a > 0.60). Residents performed Our results show that developing and implementing a better in the misread x-ray case than in the repeat USP program in the ED is feasible and acceptable to visitor case in professionalism (70% vs. 35%, t = 2.81, residents. Considering the drawbacks of OSCE assess- p = 0.048) and patient-centeredness (66% vs. 40%, ment and direct observation, combined with increasing t = 1.96, p = 0.05). Communication (r = 0.73, p = 0.16) demands on faculty time and decreasing funding, USPs and recommendation scores (r = 0.81, p = 0.09) were may offer an objective, cost-effective method for evalu- highly, albeit not significantly, correlated between the ating accurate practice skills. two cases, but professionalism (r = 0.24, p = 0.70), and The biggest challenge faced while implementing the patient-centeredness were not (r = 0.08, p = 0.90), sug- USP program was the unpredictability of the ED. Occa- gesting case content matters most in these domains. sionally, USPs were mistakenly examined by another resident. Both content (highly trained SP, realistic Postevaluation Survey cases) and logistic factors (dedicated program coordi- Eighty-three percent of residents who encountered a nator, electronic MRs, team collaboration) are neces- USP felt that it did not hinder their daily practice sary for successful integration. Total cost, in terms of and did not make them uncomfortable (86%) or sus- both time and money, is likely greater up front, with picious of patients (71%). A minority of those resi- decreased workload, time, and expense as USPs and dents who encountered a USP felt that the encounter staff become trained. Further study of the costs is improved their practice behavior (14%), made them needed.
  • 4. 918 Zabar et al. • PROFESSIONALISM AND COMMUNICATION Even with a high detection rate, residents reported at: http://www.acgme.org/outcome/comp/compCPRL. value in the USP program for learning and patient care. asp. Accessed Sep 20, 2008. It is possible that informing residents that USPs would 2. Accreditation Council for Graduate Medical Educa- be visiting them in the ED improved performance. tion, American Board of Medical Specialties. More importantly, the majority of residents did not feel Outcome Project Toolbox of Assessment Methods. that the possibility of encountering a USP had any neg- Available at: http://www.acgme.org/outcome/assess/ ative impact on their daily practice, suggesting that toolbox.asp. Accessed Jun 20, 2009. USPs in the ED will not risk real patient safety. The 3. Gorter S, Scherpbier A, Brauer J, et al. Doctor- case of the resident who reported ignoring a patient patient interaction: standardized patients’ reflec- thought to be ‘‘unannounced’’ represents an unantici- tions from inside the rheumatological office. pated and anomalous professionalism issue, we believe, J Rheumatol. 2002; 29(7):1496–500. not causally related to the use of USPs; it demonstrates 4. Kravitz RL, Epstein RM, Feldman MD, et al. Influ- how USPs can provide useful information to program ence of patients’ requests for direct-to-customer directors. advertised antidepressants: a randomized controlled trial. JAMA. 2005; 293(16):1995–2002. LIMITATIONS 5. Ozuah PO, Reznik M. Using unannounced standard- ised patients to assess residents’ professionalism. There was a small sample size, with a relatively large Med Educ. 2008; 42(5):532–3. proportion of failed USP visits. However, the failure 6. Fiscella K, Franks P, Srinivasan M, Kravitz RL, rate improved as the project progressed. Even with our Epstein R. Ratings of physician communication by small numbers, it appears that two cases and the items real and standardized patients. Ann Fam Med. 2007; on the behaviorally anchored checklist can discriminate 5(2):151–8. residents based on their communication skills. 7. Talente G, Haist SA, Wilson JF. The relationship between experience with standardized patient CONCLUSIONS examinations and subsequent standardized patient examination performance: A potential problem with With the ACGME placing greater importance on evalua- standardized patient exam validity. Eval Health Prof. tion of patient outcomes, we believe that our project 2007; 30(1):64–74. represents a new way to assess real-time resident perfor- 8. Srinivasan M, Franks P, Meredith LS, Fiscella K, mance. Despite being time-consuming and subject to the Epstein RM, Kravitz RL. Connoisseurs of care? unpredictability of the ED, implementing unannounced unannounced standardized patients’ ratings of phy- standardized patients in the ED is feasible and acceptable sicians. Med Care. 2006; 44(12):1092–8. to staff. Future comparison of unannounced standard- 9. Marshall GN, Hays RD. The Patient Satisfaction ized patients with observed structured clinical exam Questionnaire Short Form (PSQ-18). RAND Corpo- scores will enable educators to determine how well these ration, Paper P-7865. Available at: http://www. methods assess performance in actual practice. rand.org/pubs/papers/P7865/. Accessed Jun 20, 2009. 10. Elwyn G, Edwards A, Wensing M, Hood K, Atwell References C, Grol R. Shared decision making: developing the OPTION scale for measuring patient involvement. 1. Accreditation Council for Graduate Medical Educa- Qual Saf Health Care. 2003; 12:93–9. tion (ACGME). ACGME Outcome Project. Available