Tips for constructing objective written exams (MCQs, Short answer questions, Modified essay questions, True/False and Matching questions) for assessing medical students.

1. TIPS FOR CONSTRUCTING OBJECTIVE WRITTEN EXAM QUESTIONS Presented by: Dr. Soha Rashed Member of Medical Education Department Professor of Community Medicine Faculty of Medicine- Alexandria University 12.1.2009

2. OBJECTIVE WRITTEN EXAM QUESTIONS Multiple Choice Questions (MCQs) Short Answer Questions (SAQs) Modified Essay Questions (MEQs) True/False Questions (TF) Matching Questions

3. MCQs

4. Guidelines for writing MCQs

5. How MCQ items are constructed? Incomplete statement, direct question, problem situation (clinical scenario), diagram, graph, image, radiologic image, histopathological section, laboratory findings, etc. Lead-in (question) Alternatives (options or choices): Onecorrect answer, and several plausible incorrect/wrong answers. The incorrect responses are called distractors. divert unsure students from the correct answer To distract those students who are uncertain of the answer

6. Tips for writing clear MCQs Write simple stems that clearly state the central problem: Present the problem as precisely as possible, and include only the necessary/relevant information. Ensure accuracy in grammar and sentence construction. Use familiar words, and avoid uncommon terminology and abbreviations. Avoid double negatives.

7. The stem should contain as much of the item as possible. If a phrase can be stated in the stem, it should not be repeated in the options. Ideally, the item should be answerable without having to read all of the options (cover test).

9. Understanding

10. Application of knowledge

12. Contextual MCQ with the same themes (better example) A 55-year-old patient with chronic renal failure undergoing dialysis. He appears to be pale. A full blood count shows the following red cell indices: Hemoglobin: 8.7 gm/dl Hematocrit: 26% MCV: 92 fL (expected range 80-100 fL) MCH: 33 pg (expected range 27-31 pg) MCHC: 33 gm/dl (expected range 32-36 gm/dl) Reticulocyte count: 0.2% Which of the following is the most appropriate therapy? (a) Erythropoietin (b) Ferrous sulphate (c) Folic acid (d) Vitamin B 6 (e) Vitamin B 12 (cyanocobalamin) Note the following features in this MCQ: • Combines basic science knowledge with clinical science knowledge • Students need to connect multiple themes (in this case the role of the kidney in erythropoiesis and the changes in RBC morphology) • The lead-in (question) focuses on only one aspect of the condition • The MCQ can be answered without looking at the options (cover test)

13. Mrs. B. is 20 years old and had an IUD inserted a month ago. She came to the health center 2 days ago with vaginal discharge and abdominal and pelvic pain. She reports that she does not have any fevers or chills. What is your plan? A. Gather history, send vaginal cultures, remove the IUD. B. Gather history, send vaginal cultures, follow up in 3 days. C. Gather history, treat with antibiotics for presumptive PID. D. Gather history, send vaginal cultures, treat with antibiotics for PID, and remove the IUD if the woman wishes.

14. Several multiple-choice questions can be created for the same scenario, but these questions must be independent of each other—the answer to one cannot be dependent on the answer to the others. This type of multiple-choice question requires a higher level of thinking.

17. the history (duration of signs and symptoms),

18. physical findings,

19. results of diagnostic studies,

20. initial treatment,

22. Anatomy For each patient with the following neurologic abnormalities, select the artery(ies) that is (are) most likely to be involved. 1. A 72-year-old right-handed man has weakness and hyperreflexia of the right lower limb, an extensor plantar response on the right, normal strength of the right arm, and normal facial movements. *A. Left anterior cerebral artery B. Right anterior cerebral artery C. Left middle cerebral artery D. Right middle cerebral artery 2. A 68-year-old right-handed man has right spastic hemiparesis, an extensor plantar response on the right, and paralysis of the lower two-thirds of his face on the right. His speech is fluent, and he has normal comprehension of verbal and written commands. A. Left posterior cerebral artery B. Right posterior cerebral artery *C. Left lenticulostriate arteries D. Right middle cerebral artery E. Right lenticulostriate arteries What area is supplied with blood by the left anterior cerebral artery?

23. Anatomy A middle-aged male complains of difficulty climbing the stairs. He describes weakness without pain in his right lower limb. He is able to place his right leg on each step without experiencing any problem, but has difficulty climbing the step, and must grasp the hand-rail to pull himself up. Climbing the next step with his left leg occurs normally. You also notice that his gait on a flat surface appears nearly normal; there is no weakness in extending the right knee against a considerable load. You suspect damage and/or malfunction in the A. Obturator nerve B. Tibial nerve C. Superior gluteal nerve D. Femoral nerve E. Inferior gluteal nerve (correct option)

24. Avoid using negative terms in the stem (such as no, never, none, and not) because students sometimes misread or misunderstand them. If you must use them, bold-face and/or underline the term to draw attention to it. Example: “Which of the following is not the. . . .” Example: “Which of the following is NOT the. . . .”

25. Provide a coherent list of possible answers: Ideal number is 4-5 options. Use similar style and keep the responses similar in length. The list of responses should be grammatically consistent in form. Use an “a/an” at the end of your stem to ensure that each of the responses will complete the statement correctly, regardless of whether or not the first word of each response begins with a vowel.

26. Write reasonable distractors: Avoid obviously wrong or silly distractors. Use distractors that are clear⎯they are not meant to be trick questions or confuse the student. All distractors should be of similar length to avoid giving clues to the correct response. Numerical values should be: listed in order, of the same format , with no overlap. You may get ideas for good distractors by looking at previous tests and using students’ most frequent errors.

27. Distractors (Options) should be homogenous: Symptoms Signs Diagnosis Risk factors Treatments

29. Too complicated, requiring the examinee to constantly keep the answer code in mind.

31. Write clear directions that indicate how and where the students are to respond. Should they indicate the correct responses on the test or on a separate form? Is there only one correct answer or one best answer? What is the point value of each item? Example: Directions: Each of the items on this examination is followed by four possible responses. For each item, select the best response. Indicate your answer by circling the appropriate letter next to your answer. Each correct response is worth one point.

32. Recommended practice Use of blueprint* Context or clinical scenario-based MCQ Use a standard checklist prior to submission of MCQ Invite peers to review the question Analyze MCQ by difficulty and discriminatory indices Effect and rationale Improves content validity Assessment of higher order knowledge Efficient in identifying the problem and providing feedback to the item-writer Peer review will detect fine hidden problems Quality assurance *Blueprinting refers to the process where test content is carefully planned against the learning objectives. The examination blueprint specifies the objectives that are to be tested in the given examination as well as their relative weight on the examination.

33. Imprecise and Difficult Terms to Avoid in MCQs Several terms should be avoided while writing MCQ as they are imprecise, difficult to quantify, and give away clue to the correct answer. Examples of such words are: • Never • Always • Sometime • Generally • Commonly • Usually • Same as • Can be • May be • Can appear • Possible (possibly)

36. Short Answer Questions (SAQ) Description A practical alternative to the long essay question. SAQ is an open ended, semi-structured question format. They require the student to spontaneously respond (provide one or several responses to a question or situation) rather than choose from a selection of responses. A structured, pre-determined marking scheme improves objectivity.

37. Short-Answer Questions SAQ The questions can incorporate clinical scenarios. This type of question assesses higher-level thinking, but is less reliable because a teacher must interpret the responses. They are not as easy to complete or score as MCQ or true-false questions. A similar format is also known as modified essay question (MEQ)

38. Short Answer Questions SAQ

39. Tips for writing short-answer questions Make the questions clear and easy to understand. Write precise questions so that students know exactly what response is expected. Prepare a structured marking sheet. For each question, list all of the acceptable responses, and be prepared to consider other answers that may be equally acceptable. Make the marking sheet easy to understand so that other teachers can use it if necessary.

40. Write some questions that involve possible multiple responses. This demands more from students and can address broader content. Example: Family planning methods most appropriate for adolescents are: a._______________, b. _______________, c._______________. Write clear directions for short-answer questions. Clearly state if a point will be given for each correct response or if the student must have all the correct responses to obtain one point.

41. Modified essay question (MEQ) e.g., Case vignette (Clinical scenario) You are a medical officer in pediatrics. You are asked to review a one-hour-old baby for increasing respiratory rate and sub-costal retraction. The baby was born at 35 weeks to a 29-year-old mother via elective LSCS. The indication for LSCS was uncontrolled BP. The mother had regular follow-up during her antenatal period. She had gestational diabetes and pre-eclamsia. Question 1: What are the most likely diagnoses? (Name two) Question 2: What are the preliminary investigations that you would like to perform at this point? (Name three) Question 3: For each of the diagnoses list one primary pathophysiological mechanism. Model answer: Question 1: Hyaline membrane disease; transient tachypnea of newborn (TTNB); (two marks) Question 2: Full blood count; chest X-ray; and arterial blood gas (three marks) Question 3: deficiency of surfactant; failure to reabsorb lung fluid (two marks)

42. Advantages • Better content coverage as compared to long essay question • Improved objectivity as the marking scheme can be structured and predetermined • Less laborious to mark • Higher chance for assessment of clinical reasoning Limitation • If a large amount of knowledge needs to be tested, it is more efficient to use MCQs

43. Practical Tips in Writing SAQ Choose a relevant clinical scenario Link the questions directly to the clinical scenario Ensure that questions cannot be answered without the case Specify the number of responses Specify the mark assigned to each question Incorporate basic science principles (e.g. pathophysiological , mechanism) in the case scenario

44. True-False Questions

45. Tips for writing good true-false questions The language of true-false test items should be clear, concise, and understandable: Avoid words such as more, few, large, and good, because these are relative and may confuse the students. Avoid using negatively stated items (e.g., It is not recommended that. . . .) because students may not read the question accurately and may misunderstand the meaning. Use short statements that contain only one idea.

46. Use statements that are either completely true or completely false: Avoid a statement that is partially true and partially false. True-false items should focus on one idea and should challenge, but not trick, the students. Make the true statements equivalent in length and number to the false statements.

47. Another type of true-false item requires the student to correct false statements to make the statements true. This can be accomplished by underlining one or two key terms and asking the students to change the underlined term(s) if they decide the statement is false. With this type of item, you will need to determine if the revised terms are correct, which adds some effort and subjectivity to the scoring process.

48. Example: Read the following statement. Circle “T” if the statement is true. Circle “F” if the statement is false and write in the blank space the word(s) that would replace the underlined word(s) to make the statement true. T F Spermicide is the family planning method recommended for preventing sexually transmitted infections and HIV. ________ ________

49. Write clear directions that indicate how and where the students are to respond. Should they indicate the correct responses on the test or on a separate form? What is the point value of each item? Example: Directions: For each of the following statements, write a capital T in the block to the left of the number if the statement is true or a capital F if the statement is false. You will receive one point for each correct answer.

50. Matching Questions Two lists of words and phrases that are to be matched. The first list is known as a list of premises, the second, as a list ofresponses. Items within each list should be similar (e.g., possible diagnoses, medications, etc.).

52. Tips for writing good matching questions Focus the matching question on one subject. Keep the list of statements brief (no more than 5 to 7 statements). If the list is long, students may spend considerable time on one matching item even if they have a rather clear idea of what the response should be. Keep the matching exercise on one page; it is confusing when it runs onto the next page.

53. Arrange the premises and responses in a logical order: If the premises or responses are names or titles, they may be arranged alphabetically. If they are numbers, they may be arranged sequentially. Any logical order can be used.

54. Write clear directions that indicate the basis on which the lists are to be matched and how the student is to respond. Can answers be used more than once? Are the answers to be indicated on the test or on another sheet? Are there distractors? What is the point value of each item?