This document discusses techniques for testing and evaluation in fire service training. It covers four levels of evaluation (reaction, learning, transfer, business results), the difference between summative and formative evaluation, and various types of tests including written, oral, practical, and performance evaluations. Guidelines are provided for constructing written, multiple choice, true/false, matching, completion and essay tests. Sources for test materials are also discussed.
10. Do we ever get a final evaluation? In teaching you cannot see the fruit of a day's work. It is invisible and remains so, maybe for twenty years. ~Jacques Barzun
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25. http://rubistar.4teachers.org/index.php?screen=WhatIs&module=Rubistar Heidi Goodrich, a rubrics expert, defines a rubric as "a scoring tool that lists the criteria for a piece of work or 'what counts.'" So a rubric for a multimedia project will list the things the student must have included to receive a certain score or rating. Rubrics help the student figure out how their project will be evaluated. Goodrich quotes a student who said he didn't much care for rubrics because "if you get something wrong, your teacher can prove you knew what you were supposed to do." Generally rubrics specify the level of performance expected for several levels of quality. These levels of quality may be written as different ratings (e.g., Excellent, Good, Needs Improvement) or as numerical scores (e.g., 4, 3, 2, 1) which are then added up to form a total score which then is associated with a grade (e.g., A, B, C, etc). Many rubrics also specify the level of assistance (e.g., Independently, With Minimal Adult Help; With Extensive Adult Help) for each quality rating. Rubrics can help students and teachers define "quality". Rubrics can also help students judge and revise their own work before handing in their assignments. What is a Rubric?
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
Editor's Notes
Levels established by educator Donald Kirkpatrick. Take a look at http:// www.businessballs.com/kirkpatricklearningevaluationmodel.htm http://www.science.ulster.ac.uk/caa/presentation/kirkpatrick/tsld014.htm
Typically, this is done through the use of an evaluation form. For discussion, âwhen is the best time to have an evaluation form completedâ? Some think the very last thing in a class. Unfortunately, so often, the students are in a big hurry to get done and out the door that the evaluations are really not honest.
Exams, tests, and skill assessments help an instructors evaluate if what they have taught is actually absorbed by the student and if they are able to adequately recall.
Six weeks to six months down the road. Typically, we just donât do follow up like this. How often have you said that the true evaluation of your course is the safe retirement of the student and possibly even those they have supervised? This should be a good slide for discussion. Can you think of an easy (or even reasonable) way to evaluating how much a student retains six weeks to six months post class?
How did the department benefit from the training. Was it a train the trainer scenario? Did the student share the information? How did the student utilize the training for the benefit of the organization?
Is the true evaluation completed with a safe retirement of the teacher, the student and all the students of the students? This slide is simply for consideration and discussion. Another thought: A teacher affects eternity; he can never tell where his influence stops. ~Henry Brooks Adams
Compares overall goals, objectives, and content to the student performance. Think about the first day of a Course Delivery class and how the students handle the introduction process. Look at the progress about mid-way and then again at the end.
An activity should take place during the discussion of testing. Why not simply take a chapter from this course and ask one group to write a specified number of test questions (with possible exception of essay) and share them with the class. It serves as a good course review and you can evaluate the questionsâŚ..hey not to mention, you may use them for future classes. When writing any test questions, use caution with local vernacular wording.
All of the above responses can and have been successfully challenged. Choice âAâ fits the answer and the rest of the question is not read. Choice âAâ is actually a correct answer. None of the above questions tend to trick or cause misreading of questions that suddenly change from looking for positive to looking for negative responses.
Hopefully, true/false tests do not become a flip of the coin. Just for FYI, I wrote a 20 question T/F quiz once. The first 19 responses were true (maybe it was false) and the last one was false. I thought the class was going to lynch me.
Question: Do you provide more possible answers than there are matches?
Donât forget the term synonym. Does your response have to be exact or would a word with the same meaning be acceptable. You need to know that ahead of grading.
Suggest the General Writing Rubric (or one you like better) be handed out as an example. Copy in your âmiscâ folder.
As an instructor, you need to be watching and evaluating the group projects. If you donât, it is possible that a small number of the group will actually be doing while the others are skating.
Rote learning is a learning technique which avoids understanding of a subject and instead focuses on memorization . The major practice involved in rote learning is learning by repetition . The idea is that one will be able to quickly recall the meaning of the material the more one repeats it. Although it has been criticized by some schools of thought , rote learning is commonly used in the areas of mathematics, music, and religion. (Wikipedia). In practical settings this might be an excellent way of doing things. We have learned the âpractice makes perfectâ. It is likely less beneficial in classroom settings.
Situational skills testing can give a more true to life evaluation.
Student is being evaluated on using his bunker gear as flotation until he is able to properly remove it all. Check list would be difficult since they are in the water. Students were adequately briefed on what was expected of them prior to their âleap of faithâ; stepping off the side of the pool in full bunker gear and air pack.
Photo from Extreme Extrication class at Great Florida Fire School 2007.
Do not forget, if there is a degree of potential hazard, have a safety officer.
Some argument, obviously, could be made for any of these. Oral; however, would have the most negative consequences related to unexpected distractions.
Your choice may have been transfer; however, that looks at a much closer time frame of 6 weeks to 6 months.