RBCW Presentation

1. Choosing right Assessment method
Assessment of Clinical & Practical Skill
Dr. Hironmoy Roy
MD, MBA, ACME, FAIMER Fellow, MIPHA
Dip. Hospital Management
Copyright preserved Hironmoy Roy

2. Copyright preserved Hironmoy Roy

3. Assessment
• Assessment is a systemic procedure, for
measuring a trainee’s progress/the level of
achievement
• Against a defined criteria
• To provide judgement for the trainee
Copyright preserved Hironmoy Roy

4. Assessment vs Evaluation
Assessment
• Formal action to
measure the
competence or
performance of the
students.
• Used in context of
students (where the
student is now?)
Evaluation
• Value judgment, on a
quality or worth
• Used in context of any
educational/national
programmes
Copyright preserved Hironmoy Roy

5. Why we assess?
• Pass- Fail ?
Copyright preserved Hironmoy Roy

6. Copyright preserved Hironmoy Roy

7. We need to choose right assessment method
• For a given---
(1) Competency
(2) Objective
(3) Learning methodVALID
RELIABLE
STANDARDISED
Copyright preserved Hironmoy Roy

8. Choosing a tool…
A tool will be
• Valid: Does it really measure, what it is to measure?
• Reliable: Does it measure consistently?
• Educational impact: Does it has effect on teaching-
learning?
• Acceptable: Is it really acceptable to all stakeholders/
staffs/ teachers-students?
• Doable in the infrastructure
• cheap
Copyright preserved Hironmoy Roy

9. Copyright preserved Hironmoy Roy

10. Copyright preserved Hironmoy Roy

11. Some well known tools…
• Essay type question (EQ)
• Structured essay type question (SEQ)
• Short answer questions (SQ)
• Multiple choice questions (MCQ)
Copyright preserved Hironmoy Roy

12. Some well known tools…
• Traditional Long case/ short case
• OSCE/OSPE
• Mini CEX
• Case based discussion (CBD)
• 360 degree assessment
• Simulation (real/virtual)
• DOPS
Copyright preserved Hironmoy Roy

13. Does
Shows How
Knows how
Knows
Copyright preserved Hironmoy Roy

14. Our tradition......
Copyright preserved Hironmoy Roy

15. Long case
• A case is given
• Student needs to take history, do general survey, do
systemic examination, to arrive in a diagnosis
• Assessor asks reasoning and management plan
• Usual time 60 mins= (40 mins + 20 mins crossing)
• Largely unobserved
Copyright preserved Hironmoy Roy

16. Short case
• A case is given
• Student needs to do the systemic survey
• To arrive in a diagnosis
• Assessor asks reasoning
• Time= 20 mins (10 mins + 10 mins)
• Largely unobserved
Copyright preserved Hironmoy Roy

17. Usual queries.....
“What is your diagnosis?”
“Sir....This is a case of XYZ”
“Why you are telling so?”
“Sir....it is because of .....”
Copyright preserved Hironmoy Roy

18. Does
Shows How
Knows how
Knows
Critical reasoning
and Patient
management
problem (PMP)
Copyright preserved Hironmoy Roy

19. How to address the “Shows How” and
“Does”?
Copyright preserved Hironmoy Roy

20. Does
Shows How
Knows how
Knows
Copyright preserved Hironmoy Roy

21. OSCE
KSAC (mainly basic clinical skill)
• Different stations= discrete activities=
competencies
• Each students is exposed in all stations
• “Checklist” is the life of it.
• Built upon the SLO
• 8 to 18 stations are ideal
Copyright preserved Hironmoy Roy

22. OSCE
KSAC (mainly basic clinical skill)
• In each station student needs to perform a
scheduled task in scheduled time
• Tasks may be from K domain, may be from
AC domain
• Student needs to perform the task on
simulated situations (mannequin/patient)
Copyright preserved Hironmoy Roy

23. Copyright preserved Hironmoy Roy

24. Copyright preserved Hironmoy Roy

25. Does
Shows How
Knows how
Knows
Copyright preserved Hironmoy Roy

26. Copyright preserved Hironmoy Roy

27. Does
Shows How
Knows how
Knows
Copyright preserved Hironmoy Roy

28. Work Place Based Assessment
• Workplace-based assessment refers to the
assessment of working practices based on
what doctors actually do in the workplace,
and is predominantly carried out in the
workplace itself
Copyright preserved Hironmoy Roy

29. Copyright preserved Hironmoy Roy

30. 360 degree feedback
KSAC
a multi-source feedback to individual’s
competency from multiple perspective, which is
objectively collected by a predefined tool.
Who are the evaluators?
Peers
Subordinate
Student
Patient
Patient party
administrator
Copyright preserved Hironmoy Roy

31. Copyright preserved Hironmoy Roy

32. Mini CEX (Mini Clinical Examination)
• KSAC
• 15-20 minutes shot to assess a component
(competencies) of a long case
• Immediate & structured feedback
• Competencies those are usually assessed-
1. History taking
2. Physical examination
3. Counseling
4. Communication
5. Professionalism
Copyright preserved Hironmoy Roy

33. DOPS
• KSAC
• Same as Mini CEX
• Focuses only on procedural skill
Copyright preserved Hironmoy Roy

34. Does
Shows How
Knows how
Knows
Copyright preserved Hironmoy Roy

35. Does
Shows
How
Knows how
Knows
Mini CEX, DOPS, WPBA
360 degree assessment
OSCE/OSPE
Long case/Short
case
Essay Q, Mod EQ,
Short AQ, MCQ,
Oral-Viva
Copyright preserved Hironmoy Roy

36. Copyright preserved Hironmoy Roy

37. TIME
Given
Traditi
onal
Oral
Viva
Traditi
onal
Long
case
CBA OSCE Mini
CEX
Prob
Based
Assess
ment
MCQ
1 hr 0.50 0.60 0.68 0.47 0.73 0.36 0.62
2 hr 0.69 0.75 0.73 0.64 0.84 0.53 0.76
4 hr 0.82 0.86 0.84 0.78 0.92 0.69 0.93
8 hr 0.90 0.88 0.82 0.96 0.93 0.82 0.93
Copyright preserved Hironmoy Roy

38. 0
0.2
0.4
0.6
0.8
1
1.2
1 hr 2 hr 4 hr 8hr
Traditional Oral Viva Traditional Long case CBL OSCE Mini CEX PBL MCQ
Copyright preserved Hironmoy Roy