This document explores the assessment of professionalism in healthcare professions. It begins with an icebreaker exercise and then defines professionalism as qualities characteristic of a profession including skill, knowledge, and work practices. The document outlines several domains of professionalism including patient care, medical knowledge, and ethics. It notes professionalism is complex to define and assess given it involves various attributes, behaviors, and attitudes that depend on context. Several tools for assessment are discussed, including written assessments, competency-based approaches like OSCEs, and performance-based methods like multi-source feedback. The document stresses the need for a validated definition of professionalism to guide assessment and emphasizes triangulation of multiple assessment methods over time for reliable and valid evaluation
7. Objectives
1. Define professionalism and its domains
2. Recognize the trouble with assessing
Professionalism
3. List and explain some tools used in
professionalism assessment
4. Articulate the reasons behind the need of
assessing professionalism
8. Professionalism is generally defined as the
body of qualities or features characteristic
of a profession
Oxford English Dictionaries, 1993
9. These qualities include a high degree of skill
and knowledge that is applied to the practice
of work
11. Patient Care
Systems-based Medical
Practice Knowledge
Practice-based
Learning and
Professionalism Improvement
Interpersonal and
Communication
Skills
12. Demonstrate respect, compassion, and integrity; a
responsiveness to the needs of patients and society
that supersedes self-interest; accountability to
patients, society, and the profession; and a
commitment to excellence and on-going professional
development.
13. Demonstrate a commitment to ethical principles
pertaining to provision or withholding of clinical
care, confidentiality of patient information, informed
consent, and business practices
14. Demonstrate sensitivity and responsiveness to a
diverse patient population, including, but not limited
to diversity in gender, age, culture, race, religion, and
disabilities
15. American Board of Internal Medicine. American professionalismin medicine: issues and
opportunities. Definition and objectives.Available at: http://www.abim.org/pubs/p2/definitn.htm.
AccessedOctober 15, 2003.
16. It is very important competence
That means we need to
Teach and Assess
21. Complex construct
It encompasses a variety of
skills, knowledge and other attributes and
have complex relationships to observed
behavior
22. The Elements of Professionalism
By the American Board of Internal Medicine
But are these the only attributes of
professionalism?
NO
23. The Authors’ Classification of Themes
and Subthemes, Arising From Definitions
or Interpretations of Professionalism
Wilkinson et al;2009
24. So… We need to
Understand the various dimensions/attributes
of professionalism
Because
25. Without a validated definition of this construct, assessment of
professionalism within medical education will be compromised
It guides the selection of the educational material and provides the
criteria for any assessment system
30. Context variable
Varies between contexts and individuals and
includes both normative and ideological aspects
It should reflect both local and national contexts
van Mook WNKA et al; 2009
31. Context variable
Professionalism, and the literature supporting it to
date, has arisen from Anglo-Saxon countries.
Caution should be used when transferring ideas to
other contexts and cultures.
International Ottawa Conference (Miami 2010)
32. Context variable
Professionalism is intrinsically related to the social
responsibility of the medical profession.
Thus, definition should reflect societal and health
care changes, and this is an important responsibility
of the profession and its educational institutions to
the public. International Ottawa Conference (Miami 2010)
36. 10 Focus group
What do you mean by professionalism?
To what extent have you developed these
qualities?
37. Admitted were deficient in the acquisition of
professional values
A Professionalism not taught or assessed
Very few teachers as positive role models
38. WHY ?
Negative role modeling by the faculty
Deficiencies in the curriculum
Limited interaction with health team
Absence of feedback
39. Students Quote
I think 40–60% of the students enter medical college with
good professional qualities . . . unfortunately, they gradually
lose their qualities because of the influence of teachers and
senior students . . . they end up in behaving badly and
aggressively . . . (FG-05)
40. Students Quote
Feedback is totally missing in the system . . . Either from
teacher to the student . . . or from student to the teacher . . .
even when feedback is given to the faculty, they may not
implement, . . . they may even victimize the students . . . it
may be a good idea to take feedback anonymously . . . (FG-
10)
41. Context variable
Ginsburg et al conducted interviews with 30
clinicians after the clinicians had watched five
videotaped scenarios of professionally
challenging situations
They were asked what they thought students
should and should not do in these
situations, and they were also asked what
they would do themselves !
The authors found little agreement between clinicians. Ethical principles such as
honesty were defined differently across clinicians and within clinicians across
different scenarios
Suggesting that dishonest behavior could be interpreted as unprofessional or not
unprofessional depending on the context
Ginsburg S et al; 2004
42. Context variable
Ginsburg et al conducted interviews with 30
clinicians after the clinicians had watched five
Society
videotaped scenarios of professionally
challenging situations
They were asked what they thought students
Between individuals should and should not do in these
situations, and they were also asked what
they would do themselves !
Within individual
The authors found little agreement between clinicians. Ethical principles such as
honesty were defined differently across clinicians and within clinicians across
different scenarios
Suggesting that dishonest behavior could be interpreted as unprofessional or not
unprofessional depending on the context
Ginsburg S et al; 2004
45. Mismatch
Attitude Behavior
Unprofessional attitude professional behavior
Faking
Professional attitude Unprofessional behavior
Social pressure to behave in a particular way
46. Mismatch
Attitude Behavior
As a result, we must be carful of making
assumptions about students’ professionalism on
the basis of observed behavior alone
47. Mismatch
Attitude Behavior
Therefore , when devising methods of
assessment; knowledge of the reasoning behind
the action is also required
Attitude
48. An attitude can be defined as “a favorable or
unfavorable evaluate reaction toward
something or someone, exhibited in one’s
beliefs, feelings, or intended behavior
Myers DG. Social Psychology. Boston: McGraw-
Hill College; 1999.
51. Mismatch
Attitude Behavior
Cognitive dissonance theory suggests
“when we behave in a manner contrary to our
attitudes, we experience dissonance, which we
seek to reduce, either by changing our attitudes
to match our behaviors or vice versa.”
Festinger L. A Theory of Cognitive Dissonance. Stanford, CA: StanfordUniversity Press;
1957.
55. Written assessment
Selected • MCQ
response • Questionnaire
Constructed • Essays
response • Short answer questions
56. Written assessment
Ideal to assess
• Knowledge of the judicial, legislative and
administrative processes and ethical principle
• Reflective ability of medical students and junior
doctors
57. Written assessment
Questionnaires
Questionnaires are often based on vignettes or clinical
scenarios and may involve the description of critical
incidents
They show validity and reliability
Boenink AD et al; 2005
58. Written assessment
Critical incident report
This method asks the doctor to reflect on a
critical incident he or she has experienced or
witnessed
59. Written assessment
Critical incident report
• It can encourage reflection and attention to
elements of professionalism
• But it is dependent on the type of incident to
determine which aspect of professionalism is
being assessed
60. Competency-based assessment
It take place in controlled representations of
professional practice such as standard patient
encounters and objective structured, clinical
examinations (OSCEs)
61. Competency-based assessment
Useful component of a systematic approach to
assessing professionalism, especially in the earlier
stages of the curriculum
62. Competency-based assessment
These have the advantage over written assessment in
that they can be used to assess the ‘Showshow’ level
of Miller’s pyramid
63. Competency-based assessment
OSCE
OSCEs are also seen to be fair, as each student carries
out a standardized procedure during the assessment
and have high degrees of reliability because each
student is assessed by many different examiners and
several cases
64. Competency-based assessment
OSCE
BUT
• An artificial environment and therefore may
not reflect actual day-to-day clinical performance
•OSCEs are complex to organize
65. Competency-based assessment
OSCE
BUT
There are problems with the interpretation of student
behaviors by differing assessors, even when calibrated
and well trained
Mazor KM;2007
66. Performance-based assessment
Are those that take place within the natural clinical
setting and include work-based systems with direct
observation of the student
67. Performance-based assessment
• It measures the upper end, i.e. the ‘show’s how’ or even
‘does’, of Miller’s pyramid
•Authentic
•It enable professionalism to be assessed as a second-
order competence
68. Performance-based assessment
BUT
It is time-consuming and requires well-trained
observers and accurate criteria to work well
69. Performance-based assessment
360 feedback
Also known as multi-rater feedback or multi source
feedback (MSF)
Assessment by faculty, nursing or other members of staff
and by patients
70. Performance-based assessment
P-MEX
The Professionalism Mini-Evaluation Exercise (P-MEX)
Modifications of mini-CEX
71. This tool is used to assess a 15- to 30-minute observed
snapshot of a doctor/patient interaction that is conducted
within actual patient-care settings using real patients and
that has a structured marking sheet that covers predefined
generic areas
72. Assess four discrete areas : doctor–patient relationship skills
,reflective skills, time management, and interprofessional
relationship skills
78. Wilkinson et al;2009
Blueprinting
For example :
The blueprint demonstrates that direct
observations (through the mini-CEX
and P-MEX) and collated views
(through MSF and patients’ opinions) are
crucial elements because they capture
many aspects in reliable, valid, and
feasible ways
79. Wilkinson et al;2009
Gaps
Attributes that would not be well assessed using
the current methods
81. Criteria For Evaluation Of Instruments To Assess
Professionalism
What is the sample size, location, and demographics?
How are data recorded?
How is it scored?
Is it reliable?
Is it valid?
Were standards set/classifications made?
What is the feasibility? Length, Cost?
Do the data derived from the tool seem amenable to
change?
Does the tool educate users about the construct being
assessed?
What components of professionalism is this tool
measuring?
82. How should it be?
Less than half the articles retained by Jha et
al. demonstrated reliability or validity.
Many of the problems of reliability derive
from the fact that the assessment tools have
been developed for different purposes and
in different circumstances, perhaps
reflecting the varying contexts of
professionalism !
Jha et al;2007
83. How should it be?
NO single method of assessment has yet
emerged that is reliable and valid !
Triangulation of multiple assessments
by multiple assessors over time
89. Do you think the results of
professionalism assessment
could change the attitude ?
2
90. Many medical educators hope that by constantly
monitoring students’ professional
behaviors, the students will eventually come to
internalize appropriate attitudes
Charlotte et al;2007
Reflecting on the writings of Immanuel
Kant, Sherman puts it another way:“decorum
can, in some cases, change inner states”
Sherman; 2005
91. What if someone ‘fails’ in
professionalism, what are the
penalties of ‘failing’? What
remedial measures do we have?
Would medical school
fail a student purely
based on professional
issues?
3
93. Are the assessment tools looking
for a pattern of behaviors, or a
single behavior or a single
incident?
Are there tools for students to
evaluate faculty professionalism?
Is the purpose of the instrument to
identify professionalism or
unprofessional behaviors?
• Are we assessing professionalism
or personality?
• What is the public’s view of
professionalism?
95. Elements of professionalism are vast and
include: individual
(attributes, characteristics, attitudes, behaviou
rs, identities), interpersonal (relations, group
dynamics, etc) and societal
(economic, political, etc).
96. There is need to develop concrete and
operationalizable definitions, and from them
effective teaching methods and defensible
assessment approaches are designed
97. A true evaluation of professionalism must
focus on the reasons for a behavior, rather
than just the behavior itself
98. Triangulation of multiple kinds of measures, by
multiple observers, synthesized over time with
data gathered in multiple, complex and
challenging contexts is likely to be appropriate
at all levels of analysis
100. No single method exists for the reliable and valid
evaluation of professional behavior
101. The overall assessment program is more important than
the individual tools. The best programs use a variety of
tools in a safe climate, provide rich feedback, anonymity
(when appropriate) and follow-up of behaviour change
over time
102. It may be more important to increase the depth and
quality of reliability and validity of a program existing
measures in various contexts than to continue to
develop new measures for single contexts
104. References
• Shaw D. Ethics, professionalism and fitness to practise: three concepts, not one. Br
Dent J 2009: 207: 59–62.
• Irvine D. The performance of doctors. I: professionalism and self-regulation in a
changing world. BMJ 1997: 314: 1540–1542.
• van Mook WNKA, de Grave WS, Wass V, et al. Professionalism: evolution of the
concept. Eur J Intern Med 2009: 20: e81–e84.
• American Board of Internal Medicine. American professionalismin medicine: issues
and opportunities. Definition and objectives.Available at:
http://www.abim.org/pubs/p2/definitn.htm. AccessedOctober 15, 2003.
• Messick S. The interplay of evidence and consequences in the vali-dation of
performance assessments. Educ Res 1994: 23: 13–23
• Verkerk MA, De Bree MJ, Mourits MJE. Reflective professional-ism: interpreting
CanMEDS’ ‘‘professionalism’’. J Med Ethics
• 2007: 33: 663–666
• Boenink AD, Jonge P, Small K, Oderwald A, Tilburg W. The effects of teaching
medical professionalism by means of vignettes: an exploratory study. Med Teach
2005: 27: 429–432.
• Mazor KM, Zanetti ML, Alper EJ, et al. Assessing professionalism in the context of
an objective structured clinical examination: an in-depth study of the rating
process. Med Educ 2007: 41: 331–340.
105. References
•International Ottawa Conference (Miami 2010) ;Professionalism Theme Working
Group (IOc-PwG) ; Post-Conference Draft Recommendations and Revisions v6
•Wilkinson TJ, Wade WB, Knock LD. A blueprint to assess professionalism: results
of a systematic review. Acad Med. 2009 May;84(5):551–8.
•Zijlstra-Shaw S, Robinson PG, Roberts T. Assessing professionalism within dental
education; the need for a definition. Eur J Dent Educ. 2012 Feb;16(1):e128–136.
•Cruess R, McIlroy JH, Cruess S, Ginsburg S, Steinert Y. The Professionalism Mini-
evaluation Exercise: a preliminary investigation. Acad Med. 2006 Oct;81(10
Suppl):S74–78.
•Rees CE, Knight LV. The trouble with assessing students’ professionalism:
theoretical insights from sociocognitive psychology. Acad Med. 2007
Jan;82(1):46–50.
Ginsburg S, Regehr G, Lingard L. Basing the evaluation of professionalism on
observable behaviors: a cautionary tale. Acad Med. 2004; 79:S1–S4.
Notas do Editor
Animated recolored picture fades in over black and white copy(Advanced)To reproduce the picture effects on this slide, do the following:On the Home tab, in the Slides group, click Layout and then click Blank. On the Insert tab, in the Images group, click Picture. In the Insert Picture dialog box, select a picture, and then click Insert. Under PictureTools, on the Format tab, in the Size group, click the Size and Position dialog box launcher. In the Format Picture dialog box, resize or crop the image so that the height is set to 3.58” and the widthis set to 8”. To crop the picture, click Crop in the left pane, and in the right pane, under Crop position, enter values into the Height, Width, Left, and Top boxes. To resize the picture, click Size in the left pane, and in the right pane, under Size and rotate, enter values into the Height and Width boxes.Under Picture Tools, on the Format tab, in the Adjust group, click Color, and then under Recolor click Dark Blue, Text color 2 Dark (second row, first option from the left). Under Picture Tools, on the Format tab, in the Picture Styles group, click Picture Effects, point to Shadow, and then under Inner click Inside Diagonal Top Left (first row, first option from the left).Drag the picture so that it is positioned above the middle of the slide. On the Home tab, in the Clipboard group, click the arrow next to Copy, and then click Duplicate.Press and hold CTRL and select both pictures on the slide. On the Home tab, in the Drawing group, click Arrange, point to Align, and then do the following:Click Align to Slide.Click Align Center. Click Align Selected Objects. Click Align Middle. Select only the duplicate (top) picture. Under PictureTools, on the Format tab, in the Size group, click the Size and Position dialog box launcher. In the Format Picture dialog box, resize or crop the image so that the widthis set to 2.33”. To crop the picture, click Crop in the left pane, and in the right pane, under Crop position, enter values into the Height, Width, Left, and Top boxes. To resize the picture, click Size in the left pane, and in the right pane, under Size and rotate, enter values into the Height and Width boxes.Under Picture Tools, on the Format tab, in the Adjust group, click Color, and then under Recolor, click No Recolor. On the Home tab, in the Drawing group, click Shapes, and then under Rectangles click Rectangle (first option from the left). On the slide, drag to draw a rectangle. Select the rectangle. Under Drawing Tools, on the Format tab, in the Size group, do the following:In the Shape Height box, enter 7.5”.In the Shape Width box, enter 2.33”.Select the rectangle. Under Drawing Tools, on the Format tab, in the Shape Styles group, click Shape Outline, and then click No Outline.Under DrawingTools, on the Format tab, in the ShapeStyles group, click ShapeFill, point to Gradient, and then click MoreGradients. In the Format Shape dialog box click Fill in the left pane, select Gradient fill in the Fill pane, and then do the following:In the Type list, select Linear. In the Angle box, enter 90.Under Gradient stops, click Add gradient stops or Remove gradient stops until two stops appear in the slider. Also under Gradient stops, customize the gradient stops as follows:Select the first stop in the slider, and then do the following: In the Position box, enter 0%.Click the button next to Color, and then under Theme Colors click White, Background 1 (first row, first option from the left).In the Transparency box, enter 55%. Select the secondstop in the slider, and then do the following: In the Position box, enter 100%.Click the button next to Color, and then under Theme Colors click White, Background 1 (first row, first option from the left).In the Transparency box, enter 100%. On the slide, drag the rectangle to cover the duplicate picture. Select the rectangle. On the Home tab, in the Drawing group, click Arrange, and then do the following:Point to Align, and then click Align to Slide.Point to Align, and then click Align Middle. Click Send Backward. On the Home tab, in the Drawing group, click Shapes, and then under Rectangles click Rectangle (first option from the left). On the slide, drag to draw another rectangle. Select the rectangle. Under Drawing Tools, on the Format tab, in the Size group, do the following:In the Shape Height box, enter 4”.In the Shape Width box, enter 2.67”.Under Drawing Tools, on the Format tab, in the Shape Styles group, click Shape Fill, point to Gradient, and then click No fill. Under Drawing Tools, on the Format tab, in the Shape Styles group, click the Format Shape dialog box launcher. In the Format Shape dialog box, click Line Color in the left pane, select Solid line in the Line Color pane, and then do the following:Click the button next to Color, and then under Theme Colors click White, Background 1 (first row, first option from the left). In the Transparency box, enter 70%.Also in the Format Shape dialog box, click Line Style in the left pane, and then do the following in the Line Style pane:In the Width box, enter 0.75 pt. Click the button next to Dash type, and then click Square Dot (third option from the top).Drag the dotted rectangle on top of the small, full-color picture. Press and hold SHIFT and select the dotted rectangle, the small picture, and the large picture on the slide. On the Home tab, in the Drawing group, click Arrange, point to Align, and then do the following:Click Align Selected Objects. Click Align Middle. On the Insert tab, in the Text group, click Text Box, and then on the slide, drag to draw the text box.Enter text in the text box, select the text, and then on the Home tab, in the Font group, select Gill Sans MT Condensedfrom the Font list, select 24 from the Font Size list, click the button next to Font Color, and then under Theme Colors click White, Background 1 (first row, first option from the left).On the Home tab, in the Paragraph group, click Center to center the text within the text box.On the slide, drag the text box below the dotted rectangle.To reproduce the background effects on this slide, do the following:Right-click the slide background area, and then click Format Background. In the Format Background dialog box, click Fill in the left pane, and then select Solid fill in the Fill pane. Also in the Fill pane, click the button next to Color, and then under Theme Colors click Black, Text 1, Lighter 15% (fifth row, second option from the left).To reproduce the animation effects on this slide, do the following:On the View tab, in the Zoom group, click Zoom, and then in the Zoom dialog box, in the Percent box, enter 70%. (Note: Make sure that Fit is not selected in the Zoom dialog box.)On the slide, select the dotted rectangle. On the Animations tab, in the Advanced Animations group, click Add Animation, and then, under Motion Paths, clickCustom Path.Press and hold SHIFT to conform the path to a straight, horizontal line, and then do the following on the slide:Click the center of the dotted rectangle to create the first motion-path point.Click approximately ½” beyond the right edge of the rectangle to create the second motion-path point. Double-click approximately 2” beyond the left edge of the slide to create the third and final motion-path point. On the slide, right-click the freeform motion path, and then click Reverse Path Direction. On the Animations tab, in the Timing group, in the Start list, select WithPrevious.On the slide, select the gradient-filled rectangle. On the Animations tab, in the Advanced Animations group, click Add Effect, and then click More Entrance Effects. In the Add Entrance Effect dialog box, under Subtle, click Fade, and then click OK. On the Animations tab, in the Timing group, in the Start list, select WithPrevious.On the Animations tab, in the Timing group, in the Duration box, enter 0.5. On the slide, select the gradient-filled rectangle. On the Animations tab, in the Advanced Animations group, click Add Effect, and then click More Motion Paths. In the Add Motion Path dialog box, under Lines and Curves, click Down, and then click OK. On the Animations tab, in the Timing group, in the Start list, select WithPrevious.On the Animations tab, in the Timing group, in the Duration box, enter 2. On the slide, right-click the down motion path and click ReversePathDirection.On the slide, select the smaller, full-color picture. On the Animations tab, in the Advanced Animations group, click Add Effect, and then click More Entrance Effects. In the Add Entrance Effect dialog box, under Subtle, click Fade, and then click OK. On the Animations tab, in the Timing group, in the Start list, select WithPrevious.On the Animations tab, in the Timing group, in the Duration box, enter 2. On the Animations tab, in the Timing group, in the Delaybox, enter 1.5. On the slide, select the text box. On the Animations tab, in the Advanced Animations group, click Add Effect, and then click More Entrance Effects. In the Add Entrance Effect dialog box, under Subtle, click Fade, and then click OK. On the Animations tab, in the Timing group, in the Start list, select WithPrevious.On the Animations tab, in the Timing group, in the Duration box, enter 1.
This template can be used as a starter file for a photo album.
Where assessment tools are to be used in new contexts, re-validation with attention to cultural relevance is imperative.
Where assessment tools are to be used in new contexts, re-validation with attention to cultural relevance is imperative.
Where assessment tools are to be used in new contexts, re-validation with attention to cultural relevance is imperative.
Where assessment tools are to be used in new contexts, re-validation with attention to cultural relevance is imperative.
Where assessment tools are to be used in new contexts, re-validation with attention to cultural relevance is imperative.
Direct observation, especially during clinical performance, has many advantages. It measures the upper end, i.e. the‘show’s how’ or even ‘does’, of Miller’s pyramid . It goessome way to providing authenticity and the context to enableprofessionalism to be assessed as a second-order competence.Unfortunately, it is time-consuming and requires well-trainedobservers and accurate criteria to work well