The document discusses various types of assessment questions that can be used for computer-aided assessment (CAA) and their suitability for different levels of Bloom's Taxonomy. It describes the advantages and disadvantages of true/false, matching, multiple choice, short answer, calculation, essay, "problem based", simulation, and performance questions. The document recommends that any CAA system developed by the university should be easy to use, supported, standards-based, secure, and scalable to meet the needs of assessing higher-order thinking skills.

1. E-learning and Assessment<br />Version 4.<br />Reference http://wiscinfo.doit.wisc.edu/teaching-academy/Assistance/course/questions.htm<br />The education strategy (March 2004) has highlighted that Bristol University:-<br />Is a “research intensive” university where the student learning experience is informed by research and is delivered by research active staff.<br />Wishes to retain its high quality teaching.<br />Needs to widen access and to look at diverse progression routes into HE.<br />Needs to expand postgraduate numbers – particularly on taught masters programmes<br />To retain their highly rated international research output and to deliver high quality teaching, requires that teaching is cost effective.<br />Whilst E-learning has a relevance to all of these points, this paper will focus on point 5 i.e. the requirement for teaching to be cost effective. The contribution of E-learning to assessment is a major area where substantial gains in efficiency can be achieved. <br />Currently staff spend a significant amount of time <br />setting examinations <br />ensuring that they match course learning objectives, <br />liasing with the external examiner<br />marking<br />exam related student administration<br />Much of this activity is repeated at least twice per year and sadly the outcome is often only summative rather than formative. Several staff within the university have already adopted CAA (computer aided assessment) as a means of addressing this problem and have found pedagogic as well as efficiency gains. However this approach has not yet become embedded. Reasons for this limited uptake include:-<br />Lack of staff time<br />Lack of priority by university managers<br />Lack of reward<br />Techno phobia<br />Lack of knowledge/understanding of how e-learning can help. Specifically many staff still believe that CAA is only capable of “simple multi choice questions” which test recognition.<br />A multitude of software packages which are evolving and for which support is varied. Academic staff whose interest is primarily the pedagogy of their own subject, find the technological developments both confusing and frustrating.<br />Academic staff perception that the technology is driving the pedagogy, rather than the student learning experience being informed by research and delivered by research active staff.<br />It is therefore essential that any development of CAA within the University of Bristol must address these issues. Specifically the technology must:-<br />Match the needs of academic staff <br />Be relatively simple to use, be well supported and not evolve rapidly in the short term. It is therefore important to get the specification right first time and not be continually updating it. The current design should therefore match not just current (short term) but also our future (medium term) needs. <br />Be capable of assessing at all levels of Blooms Taxonomy (see later)<br />Fit within our portal (MLE)<br />Support the latest technical interoperability standards to facilitate easy transfer of content to another CAA application if required at a later date<br />Be secure, both meet Data Protection/privacy requirements and have robust mechanisms in place to deal with machine or network failures<br />Be scalable - while any CAA might initially be implemented in one or two areas it should have the potential to be used across the University <br />Before examining question and assessment types, it's beneficial to establish a common language for discussing cognitive processes. To assess specific levels of student mastery of course content, needs a framework for describing those levels. Bloom's taxonomy is a system created to improve testing precision by categorising cognitive functioning into distinct levels. Appropriate questions can then be developed to assess the desired level. Psychologist Benjamin Bloom developed this system at the University of Chicago in the late 1940s. His goal in formulating this classification was to increase precision in the discussion of educational goals among teachers, administrators, and researchers.<br />Bloom's levels are: <br />654321<br />Knowledge <br />Comprehension <br />Application <br />Analysis <br />Synthesis <br />Evaluation<br />quot;
Taxonomyquot;
is simply a system of categorizing and organizing. In this case, the taxonomy is hierarchical; each level is subsumed by the higher levels. In other words, a student functioning at the quot;
applicationquot;
level has also mastered the material at the quot;
knowledgequot;
and quot;
comprehensionquot;
levels. <br />Before taking a closer look at the levels, it should be noted that Bloom's taxonomy is not the only system of classification for cognitive domains. However, it is a widely accepted system, and is useful in illustrating how certain question types are better choices for assessing different levels of student mastery. What's important is the idea that student knowledge is not all equal; there are levels of mastery ranging from simple recitation of facts, to formulating informed opinions regarding complex issues. An awareness of these levels can help one determine how well do students really know course content. <br />Table 1 : Blooms Taxonomy of Learning<br />LevelLearner ActionQuestion CuesKnowledgeRecall content in the exact form that it was presented. Memorisation of definitions, formulas, or procedures are examples of knowledge-level functioningList, define, label, identify, nameComprehensionRestate material in their own words, or can recognise previously unseen examples of a concept.Describe, associate, categorise, summariseApplicationApply rules to a problem, without being given the rule or formula for solving the problem.Apply, calculate, illustrate, solveAnalysisBreak complex concepts or situations down into their component parts, and analyse how the parts are related to one another.Analyse, compare, separate, order, explainSynthesisRearrange component parts to form a new whole.Combine, modify, rearrange, quot;
what-ifquot;
EvaluationEvaluate or make judgments on the worth of a concept, object, etc. for a purposeAssess, decide, grade, recommend, explain, judge<br />Introduction to Exam Question Types<br />True/False <br />Matching <br />Multiple Choice<br />Short Answer<br />Calculation<br />Translation <br />Essay<br />“Problem based”<br />Comprehension - 'Gobbets'<br />quot;
Simulation testingquot;
<br />quot;
Performancequot;
<br />True/false, matching and multiple choice are examples of closed-ended questions. That is, they do not require the students to construct an answer themselves. Rather, the answer is among the possible options. Closed-ended questions test recognition; that is, they evaluate a student’s ability to recognize the answer. However questions that are built around a scenario, some data, a graph or a table of numbers can require students to ‘deal’ with the information in order to determine the correct response.<br />Open-ended questions include short answer and essay types. These questions require the creation of a written response. Open-ended questions test student recall; the student is asked to come up with the answer on their own, as opposed to picking it out from a list of potential options. <br />quot;
Problem basedquot;
, quot;
simulation testingquot;
and quot;
performancequot;
all assess the student's ability to undertake a process and therefore need to assess skills as well as knowledge. Traditional written examination formats clearly are not an appropriate form of assessment. E-learning provides an opportunity to develop assessment tools which are matched to these learning objectives.<br />True/False Questions<br />Advantages:True/false questions are among the easiest to write, and can be scored electronically.<br />Disadvantages:True/false questions are limited in what kinds of student mastery they can assess. They have a relatively high probability of student guessing the correct answer (50%). True/false also assesses recognition of information, as opposed to recall. However properly crafted True/False questions require students to recall relevant knowledge, reason or make some deduction about the mechanism and so arrive at their most likely correct response which hopefully is one of the choices. <br />Most Appropriate For:Factual information and naturally dichotomous information (information with only two plausible possibilities). Dichotomous information is quot;
either/orquot;
in nature. Examples include male/female, analog/digital, and internal/external.<br />Matching Questions<br />Advantages:Matching items can assess a large amount of information in a confined space on the exam page, relative to multiple-choice questions; if developed carefully, the probability of guessing is low. To decrease that probability further, avoid equal-sized lists by including a few “distractor” items in the second (answer) column.<br />Disadvantages:Matching assesses recognition rather than recall of information.<br />Most Appropriate For:Assessing student understanding of related information. Examples of related items include Symptoms and diagnosis, intervention and risks, terms and definitions, tools and uses, and events and dates.<br />TABLE 2 Question types<br />Question typeDescriptionBlooms levelsTrue/falsePresent a statement, and prompt the student to choose whether the statement is truthful Knowledge Comprehension Matching questionsInvolve paired lists that require students to correctly identify, or “match,” the relationship between the itemsKnowledgeComprehensionMultiple-choiceAsk students to choose from a list of possible answers. Most multiple-choice questions feature one correct answer, and two to four quot;
distractorquot;
choices that are not correct. KnowledgeComprehensionApplicationAnalysisShort-answerOpen-ended questions that require students to create an answer. Short-answer items typically require responses of one word to a few sentences. “Fill in the blank” and “completion” questions are examples of short-answer question types.KnowledgeComprehensionApplicationAnalysisCalculationPresent a statement which includes variables from a formula that is being assessed. The particular instance has variables replaced by specific numbers so the calculation can be performed. Can also test understanding of dimensions appropriate to the calculation. Can also handle inexact responses - so making students aware of the precision of answers.Knowledge Comprehension ApplicationAnalysisTranslationStudents are required to translate a passage from one language to another specified language observing the grammatical structure, idiomatic expression and literary tone of the piece. Knowledge, Comprehension, Application, SynthesisEssayLike short-answer, are constructed-response questions. However, essay answers are typically much longer than those of short-answer, ranging from a few paragraphs to several pages.Knowledge Comprehension Application Analysis SynthesisEvaluationComprehension'Gobbets'Students must respond to a passage or image, often without the prompt of a specific question. They will have to interpret (and sometimes identify) the passage/image and should be able to relate it to its wider context. These questions specifically test the student's ability to create answers and to make sensible links between specific items and wider academic debates.Knowledge, Comprehension, Application, Analysis, Synthesis, Evaluation“Problem based”Students are asked to interpret information and make an informed decision as to what further information is required so that judgements, decisions and course of action can be decided upon. Knowledge Comprehension Application Analysis SynthesisEvaluationquot;
Simulation testingquot;
Use of a simulation capable of testing (in a controlled way) a student's ability to carry out a process.Knowledge Comprehension Application Analysis SynthesisEvaluationquot;
Performancequot;
Examination by performance raises particular difficulties of assessment and documentation. Performance-based assessments test the student's interpretation and presentation of material and observation of the conventions of genre (dramatic genre, scholarly presentation etc). Students are also assessed on their ability to work as a group and to communicate to an audience.Knowledge Comprehension Application Analysis SynthesisEvaluation<br />Multiple Choice Questions<br />Advantages:Multiple-choice questions are the most versatile of the closed-ended question types. This versatility stems from the fact that the questions can contain more elaborate scenarios that require careful consideration on the part of the student. The probability of student guessing is also relatively low.<br />Disadvantages:When compared to true/false and matching, multiple-choice items can be more challenging to write. They also require the creation of plausible “distractors,” or incorrect answer options. As with other closed-ended questions, multiple-choice assesses recognition over recall.<br />Most Appropriate For:Factual, conceptual, or procedural information. Unlike true/false and matching, multiple-choice is not limited to a particular type of information for which it is best suited. Because of the versatility of multiple-choice questions, the examples in this section will focus on demonstrating how an effective multiple-choice question can test for each of the first four levels of Bloom’s Taxonomy. <br />Knowledge LevelThe value that occurs most frequently in a given data set is the:<br />Mean<br />Standard deviation<br />Mode<br />Median<br />The answer is “c.” This is a knowledge-level question because it requires only that the student remember the definition of mode.<br />Comprehension LevelWhich of the following measures involves ordinal data?<br />The score of a football game<br />Football player’s jersey number<br />A team’s rank in league standings<br />Number of fans in attendance<br />The answer is “c.” This is an example of a comprehension-level question, in that it requires the student to recognize a previously unseen example of ordinal data.<br />Application Level<br />The standard deviation of the sample data 6, 9, 10, 12, 13 is closest to:<br />3<br />5<br />8<br />10<br />The answer is “a.” This question assesses application-level mastery, or the ability to apply a rule or formula in a new context.<br />Analysis Level<br />Susan, a student in Mr. Smith’s statistics class, asks Mr. Smith what her average score is for the three exams he has given the class. He replies that her average is +1.7. Which of the following assumptions about the students’ scores on these tests is most plausible?<br />The standard deviations of scores on all three tests were similar.<br />None of the tests produced extremely skewed distributions.<br />All of the students did poorly on at least one of the tests.<br />The correlations between the three sets of test scores were low. <br />Answer is “d.” This item requires the test-taker to recognize unstated assumptions and relationships. The test-taker has to recognize the steps Mr. Smith has taken (converted scores into Z-scores) and why he would do so (the tests probably have very unlike distributions or lack of variation in one or more tests). <br />Short-Answer Questions <br />Advantages:Short-answer questions assess unassisted recall of information, rather than recognition. Compared to essay questions, they are relatively easy to write.<br />Disadvantages:Short-answer items are only suitable for questions that can be answered with short responses. Additionally, because students are free to answer any way they choose, short-answer questions can lead to difficulties in scoring if the question is not worded carefully. It’s important when writing short-answer questions that the desired student response is clear.<br />Most Appropriate For:assessing the correct spelling of items, or in cases when it is desirable to ensure that the students have committed the information to memory (medical students, for example, will require recall of information more than recognition by the nature of their jobs). <br />Calculation Questions<br />Advantages:For simple formula this is a very easy and obvious question type. It can generate a large set of potential questions automatically so each test can be different. Handling dimensions and marking that aspect separately is another useful feature. Method also allows for inexact responses so making student aware of imprecision.<br />Disadvantages:For more complex formula the capabilities might be too limited. Only suitable for formula based questions.<br />Most Appropriate For:Scientific material based on formulae that have a single solution.<br />Essay Questions<br />Advantages:Essay questions can effectively assess all six levels of Bloom’s Taxonomy. They allow students to express their thoughts and opinions in writing, granting a clearer picture of the level of student understanding. Finally, as open-ended questions, they assess recall over recognition.<br />Disadvantages:There are two main disadvantages to essay questions: time requirements and grading consistency. Essays are time-consuming for students to complete, and require careful instructions on the part of the test writer. Scoring can be difficult because of the variety of answers, as well as the “halo effect” (students rewarded for strong writing skills as opposed to demonstrated mastery of the content). Students may also plagiarise published work and present that as their essay. Currently available software can be used to screen student essays to deter plagiarism.<br />Most appropriate for:Assessment that cannot be accomplished with other question types. Because essays are the only question types that can effectively assess the highest levels of student mastery, they are the only option if the goal of testing is the assessment of synthesis and evaluation levels.Essay questions can be written for all of Bloom's taxonomy, but their true advantage is the ability to assess the highest levels. Because essay questions can be time-consuming for student to complete and difficult for instructors to grade, this question type should be chosen carefully. If the desired knowledge can be assessed with a different question type, it should be considered. However, if higher-level processing is a goal of a course (as it likely will be), well-developed essay questions are the key to accurate assessment. <br />“Problem based”<br />Advantages :Similar to an essay question can effectively assess all six levels of Bloom’s Taxonomy. It is particularly suited to assessing how effectively students can apply knowledge gained in a novel situation. The student’s ability to make effective decisions is assessed and thus it is appropriate to Blooms level 5 & 6 (synthesis and evaluation). For example you can ask a question and the next question depends upon the answer given to the former. The result is an assessment based as much on the path taken through the questions as it is on the final question answer. It can identify if a student can choose a good strategy or not. You can get the student to enter into a dialogue with the computer, and record the responses. Time stress can be applied which has relevance in some areas. <br />Disadvantages :Difficult questions to write, requiring detailed data/information to be presented to the student in a structured way. The question is likely to need to integrate information from several sources and have several “pathways” open for the student to choose. (see figure 1)<br />Most appropriate for :Suited to differential diagnosis of clinical cases. Presenting problem can be presenting clinical symptoms, which requires provisional diagnosis (possibly more than one). This needs to be refined by clinical examination and laboratory tests (pathology, microbiology, biochemistry, histology, ECG, ultrasound, X ray etc.) The diagnosis once refined may need further tests before a conclusive diagnosis can be made. This assessment process is highly relevant to the principles of differential diagnosis and evidence based medicine.Once a diagnosis has been made then a suitable treatment needs to be decided upon and a prognosis made. Each of these steps can be assessed and ranked in order of importance. Negative marking will be required if an option is taken which would actually harm the patient.<br />Provisional hypothesisPresenting informationwhich defines the problemDecision (several options)Course of action choice(several options)Modified hypothesisFigure 1 Outline of a problem based assessmentAdditional informationrequested by the student[To be self selected from several sources, but which is ranked in terms of significance. If inappropriate or excessive information is requested then student looses marks] <br />quot;
Simulation testingquot;
<br />AdvantagesMany things we teach actually are high level processes that must be learnt - interpreting, translating, writing, manipulating, etc. Simulation of the actual process can be a powerful tool for facilitating learning, and can be used for both summative and formative assessment. However the assessment of these processes can often be difficult and complex. The development of virtual reality tools and E-learning provides an exciting opportunity to overcome many of these problems and to be cost effective at the same time. <br />DisadvantagesSimulations can be costly to develop both in staff time and the cost of appropriate equipment. Currently the aerospace and defence world uses simulations (flight simulators) as part of both learning and assessment. But in this very expensive example, it is worth building a simulation because the cost of making a mistake on a real one is huge. This is not always (often) the case. There are cheaper ways of simulating however, a good example being the CD test currently used to test driving students hazard awareness skills. Another example being developed at Glasgow University is an interactive model used to test a students ability to pregnancy diagnose a cow.<br />Most appropriate for :Testing students skills, which currently is often not done because of the financial cost, demands upon staff time and safety implications.<br />quot;
Performancequot;
<br />Advantages Almost by definition the performing arts require both formative and summative assessment tools capable of making judgements about student learning outcomes based upon a performance or a piece of creative work. E-learning tools such as video or image capture can play an important role in storing the evidence of student outcomes (on which the assessment is based) and in supporting formative feedback.<br />Disadvantages<br />There are few shortcuts for marking performance and elearning tools cannot automatically mark work as they can for multiple choice type questions for example. Performance marking can be seen as subjective and the marking criteria needs to be properly thought out. Performances are ephemeral events, so unless the performance is captured in some way, (for example with video recording equipment) the performance is lost. If it is a group performance, it can be difficult to mark individual students for what is essentially a group effort. <br />Most appropriate for : <br />Performance tests practical application of theory and methodology, gauging creative interpretation of source material. Performance can also test communication skills, for example in medicine a students ability to communicate with patients (often actors) can be assessed. There are parallels with science practicals, but in arts subjects such as music, it the performance itself that is assessed, rather than the results of the practical.<br />The University of Bristol/ILRT has been proactive both in developing itself and in buying in technologies capable of delivering on-line assessment (see attached appendix 2). <br />Whatever the decision as to which tools will be chosen for the next step, it will be important that they can be used to assess all 6 levels of Blooms taxonomy.<br />Appendix 1: Examples of CAA <br />which evaluate differing levels of Blooms taxonomy<br />LEVEL 1: knowledge/recognition<br />Question:Newton's three famous laws of motion include these concepts (select three):<br /> <br />Speed<br />Velocity<br />Mass<br />Vector<br />Energy<br />Gravity<br />Question:For each of the following dates, match the event or person associated with it. <br /> <br />Question Items Answer Items<br /> 1. 1066 A. Norman Conquest of Britain<br />2. 1100s B. Angkor Wat is built<br />3. 1215 C. Magna Carta signed<br />4. 1347 D. Bubonic Plague spreads in Europe<br />5. 1455 E. Gutenberg's movable-type printing press produces the Bible<br />6. 1517 F. Martin Luther initiates the Reformation<br />7. 1664 G. Newton's theory of universal gravitation<br />8. 1764 H. Mozart (aged 8) writes first symphony<br />9. 1867 I. Japan ends 675-year shogun rule<br />10. 1947 J. Gandhi's civil disobedience movement leads to an independent India<br />LEVEL 2: Comprehension<br />Question: Indicate what class is this drug? <br />Alkyl phenol<br />Steroidal anaesthetic<br />Barbiturate<br />Non-steroidal analgesic<br />Muscle relaxant<br />Question: Which of the following statements is correct?<br /> <br />stereotypies are repetitive behaviours that develop into redirected behaviours<br />stereotypies sometimes develop from redirected behaviours and approach / avoidance behaviour<br />apathetic behaviour, redirected behaviour and aggressive behaviour are all examples of stereotypies<br />stereotypies only develop in animals that have already shown vacuum behaviours<br />LEVEL 3 : Application<br />Question: Indicate what is the effect of this suture pattern on the wound edges<br />Apposes edges<br />Inverts edges<br />Relieves tension at edges<br />Question: <br />Pet cats show hunting behaviour. According to control theory models of motivation involving feedback mechanisms, if the functional consequence of this hunting behaviour (food. is given to the cat, the behaviour should stop. <br />True or False<br />Level 3/4 Application/Analysis<br />The following data were obtained from an experiment in which four diets were fed to growing rats. Three diets were formulated to contain protein (CP) entirely as casein, fishmeal or kidney bean meal. The fourth diet was protein-free. Food intake, faecal and urinary excretion were measured over 5 days. The following results were obtained. <br />Which diet has the highest true protein digestibilty<br />casein<br />fishmeal<br />kidney bean<br />Incorrect Answer Feedback Example of casein <br />where EFCP = endogenous faecal crude protein <br />Thus casein has the highest true protein digestibility<br />Question: The following data were obtained from a digestibility trial with growing cattle weighing 300kg. <br />Nutritive value of food<br />Dry matter (DM) concentration0.25<br />Gross energy (GE)18.8 MJ/kgDM<br />Neutral detergent fibre550g/kgDM<br />Animal data<br />Food intake13.5/kg fresh weight<br />Faecal output1.22kgDM/day (GE = 15.2MJ/kgDM)<br />Urinary output150gDM/day (GE= 16.1MJ/kgDM)<br />Methane output160 litres (STP)/day (GE = 39.5kj/lSTP)<br />First calculate the ME value in MJ/kgDM of the feed and use it to calculate the ME intake. Use this value to determine the <br />Indicate ME intake : ME requirement ratio.<br />1.5<br />1.3<br />1.0 <br /> <br />Incorrect Answer Feedback<br />Diet Metabolisable Energy concentration (MJ/kg/DM) is calculated from <br />which equals<br /> <br />thus ME concentration = 10.72 <br />ME intake = food intake (fresh) x DM xME concentration <br />= 13.5 x 0.25 x 10.72 = 36.18 <br />ME requirement = 0.5 x W0.75 = 0.5 x 3000.75 = 36.04 <br />Therefore ME/Mem ration = 36.18/36.04 = 1.0<br />LEVEL 4 : Analysis<br />Question A sow is introduced to another sow on one occasion and shows no oestrous response but, a few minutes later, shows a strong oestrous response when introduced to a boar, even though the two are not allowed to mate. The same procedure is followed again a week later and this time the sow shows no oestrous response to the boar. These observations indicate that: <br />oestrous behaviour is controlled by external causal factors only<br />oestrous behaviour is controlled by internal causal factors only<br />both external and internal causal factors control oestrous behaviour<br />the identity of the boar strongly influences whether oestrous behaviour is shown<br />Question The graph below shows the relationship between levels of stereotypic circling behaviour and baseline circulating corticosterone observed in 20 randomly selected singly housed mice. These results indicate that: <br />Circling behaviour helps the animals to cope with the environment by reducing stress<br />There is no discernible relationship between circling and corticosterone<br />Corticosterone levels play a causal role in the development of circling behaviour<br />Animals that show high levels of circling may be more stressed<br />High activity could be linked to elevations in corticosterone levels<br />LEVEL 4/5 Analysis/synthesis<br />Question The following results were obtained from routine haematology / biochemistry (normal ranges appear after each value):<br />Total protein 60 g/l (57 - 78) Albumin 28 g/l (32 - 38)Potassium 2.5 mmol/l (3.5 - 4.5)White blood cells 4 x10^9/l (6 - 18)Neutrophils 1.5 x10^9/l (3 - 12)Haematocrit 0.58 (0.35 - 0.55)<br />Which of the following differentials is most likely in this case?<br />Hepatic failure<br />Chronic renal failure<br />Auto immune haemolytic anaemia<br />Parvovirus infection<br />Small intestinal foreign body <br /> LEVEL 5/6 Synthesis/Evaluation<br />Question On day 1 of a study, 6 groups of captive otters (treatment A) are prevented for swimming for 7 days while 6 other groups are not (treatment B). On day 8, otters in treatment A are allowed to swim once more. The graph below shows how much time (mean and standard deviation. otters from the two treatments spent swimming on day 8. These results indicate that: <br /> <br />otters are highly motivated to swim and this motivation accumulates since they last swam<br />otters are motivated to swim<br />deprivation of swimming is stressful for otters<br />the control of otter swimming behaviour is unlikely to be as predicted by Lorenz's psychohydraulic model<br /> <br />Question In an experiment on learning, 10 rats were injected with lithium chloride 2h after consuming their daily ration of familiar rat pellets. A further 10 rats were handled and given a sham injection. The rats were then replaced in their individual home cages with ad libitum familiar rat pellets and water. After 24 h they were given a choice between familiar rat pellets, and pellets flavoured with a novel but palatable substance. What happens?<br /> <br />The lithium chloride stimulates the rats' appetite and the experimental rats eat more of both types of pellets than control animals.<br />The lithium chloride induces a temporary sickness and a specific food aversion for the familiar rat pellets is formed. In the test, only novel pellets are consumed.<br />The lithium chloride induces a temporary sickness but no specific food aversion learning takes place because the pellets were familiar. Rats eat the familiar pellets in the test.<br />No food aversion learning takes place, because the injection was given 2h after food consumption. For learning to occur the injection would have had to be given immediately after food consumption. Rats eat the familiar pellets in the test.<br />The lithium chloride induces a feeling of sickness that persists for 24h. In the test, nothing is consumed. <br />LEVEL 6: Evaluation<br />This example is produced using CALnet and currently can only be used for instruction since it does not include the facility for assessment. It is planned to provide this as an add-on. Basically it consists of a problem solving exercise using clinical material – in this case a horse presenting with symptoms of Colic. <br />Stepwise the student needs to :<br />make a provisional diagnosis based upon the presenting symptoms<br />carry out a clinical examination – by asking pertinent questions of the software. In the assessed version (to be developed) students will be assessed on the amount of irrelevant information they request. Material that can be provided to the student can be quite diverse in its format including, text, numeric data, audio, pictures and video.<br />make a diagnosis<br />Decide upon treatment<br />Assess likely outcome of the case.<br />The following screen shots give a flavour of how the case looks for a student.<br /> <br />The following are the answers provided in pop-up boxes when individual treatments have been selected : Note some are appropriate, others would be bad news for the case!<br />AnalgesiaAnalgesia is always indicated in colic to control pain and reduce self-inflicted injury and exhaustion. The pain of impaction colic tends to be intermittent, so that analgesics are usually given intermittently to effect. ProkineticsDrugs which promote peristalsis are contraindicated because in the presence of an obstruction they tend to increase pain and, at worst, they may cause gut rupture. Liquid ParaffinLiquid paraffin is a mild laxative which softens the mass and lubricates the gut. It is delivered by nasogastric tube in large volumes, twice daily if necessary, until the mass starts to break down and liquid paraffin appears in the faeces. Because of its viscosity it is easier to deliver a suspension of the oil in warm water or 0.9% saline (50:50 volumes). This may be done under gravity using a funnel or by the cautious use of a pump. Volume guide: pony 2-3L (total); horse 4-5L; large horse 6-7L. Intravenous FluidsIntravenous polyionic fluids may be useful in the first day or two of treatment to ensure proper hydration of gut tissues. They are positively indicated in dehydrated patients.AnthelminticsAn optional treatment but it is unlikely that parasites are implicated here. Nevertheless, the presence of strongyle larvae and tapeworms may interfere with normal gut motility. The mare's history recounts use of a proprietary benzimidazole every six weeks to control alimentary parasites, but at normal therapeutic doses BMZs are not effective against migrating strongyle larvae or tapeworms. If considering anthelmintics as part of colic therapy, use products which are effective against strongyle larvae and tapeworms. AntibioticsIf the impaction were associated with peritonitis, then antibiotics would be indicated. However, they are not indicated in the clinical work up of this case.Teeth raspingThe teeth should be examined as part of the clinical work up. Sharp points on the cheek teeth can be associated with inadequate chewing of hard feed, which can predispose or exacerbate impactions. Teeth rasping should be considered. Pathological changes, particularly vascular compromise of the gut, is reflected in the colour changes seen in peritoneal fluid. In this case: normal clear yellow fluid<br />3. Progress and conclusion of the case<br />This mare was hospitalised on a bedding of shavings and treated with low doses (one-third doses) of flunixin as required. Five litres of liquid paraffin in warm 0.9% saline were given twice daily by nasogastric tube. No food was offered for 36 hours, after which wet bran mashes were given twice daily. After 48 hours scant hard faeces were produced, followed by a larger volume which was seen to contain liquid paraffin. At this point the administration of liquid paraffin was stopped and grass was introduced into the diet as a soft feed. No signs of colic recurred. Two days later the faeces showed an absence of liquid paraffin and a rectal examination confirmed disappearance of the impaction. Hay was gradually introduced into the diet over a period of three days, during which time the mashes were continued to regulate the faecal consistency. Thereafter the usual diet was introduced and the clients were warned not to use straw bedding again. No further problems were reported at a six month follow up. Appendix 2 : Comparison of various CAA tools<br />One of the key elements in any eLearning strategy will be the tools to deliver eLearning programmes or tutorials. A variety has been used within the University to date (and the following list is not exhaustive) partly because of ad-hoc experimentation but more importantly due to cost and because one single product has not been found to meet all needs.<br />Blackboard - has the all important element of central University support but its eLearning capabilities consist of the ability to store documents or the means to deliver quizzes and questionnaires which are limited in their format and content and which only allow results for the most recently taken quiz to be stored. However, Blackboard and other popular VLEs do not specifically support eLearning content creation - merely its delivery (i.e. VLEs, to a large extent, assume a priori existence of eLearning content).<br />TAL - is a Bristol-produced computer-based assessment package capable of producing online and offline tests. The online tests can be authored and delivered in both Blackboard directly or via CALnet. TAL focuses on summative assessment and does not allow inclusion of standalone questions inside formative eLearning content.<br />CALnet - enables non-technical staff to create interactive eLearning content in the form of online tutorials that support a much broader pedagogy than the minimal built-in features of VLEs. However, CALnet currently lacks any form of student tracking or feedback to the lecturer. CALnet was developed at ILRT and is available free of charge to University of Bristol staff.<br />CalScribe - is an offline tutorial authoring tool on top of the commercial off-the-shelf ToolBook multimedia authoring tool and as such brings a high degree of technical and pedagogical flexbility but at the unrealistic cost of requiring academics to become novice programmers. Every user requires a ToolBook licence in order to use CalScribe. LTSS has helped CalScribe users to switch to CALnet but at the loss of some authoring features.<br />WinEcon - eLearning tools built on top of CALnet for the WinEcon II project, feature richer interaction modes and question types than either CALnet, CalScribe or Blackboard but currently in a form that is specific to the WinEcon II project; the WinEcon tools have never been ported back to CALnet for wider use by the university.<br />The ideal eLearning tool would include at least the following characteristics:<br />Free at the point of use to staff and students (at least the perception of being free for the individuals involved)<br />Delivered via the Web and usable across all commonly used browsers<br />Enable creation and deployment of online programmes of work that lead students through a mixture of text, images (still and moving), links to online resources, questions and feedback<br />Provide the option to have questions and quizzes scored and have these results stored over time<br />Tutorials/Assessments which are repeatable… <br />Students can take a tutorial more than once and can compare their scores<br />Tutorials can be copied by staff to form the basis of a new tutorial<br />Allows lecturers to review their students' tutorials/assessment<br />Conform to the emerging standards for interoperability of content (IMS, SCORM) so that no one is tied to a particularly piece of software as tutorials can be moved between software packages<br />Conforms to guidelines and legal requirements for accessibility and usability<br />Allows local customisation of tutorials to the extent that the look and feel can be altered to reflect departmental styles but not to the extent that accessibility and usability are compromised<br />Adequately supported by online documentation (downloadable documents and/or online help) for both tutorial authors and students to the extent that direct training is not required<br />Can be hosted centrally so that both tutorials and results are held on a secure server whose data is regularly backed-up<br />Can be technically supported by the University with no, or very minimal, increase in IT staff or hardware <br />Can potentially be linked to University databases (e.g. Datahub for student records)<br />Ideally within the technical control of University staff (open source / in-house developed) to better facilitate enhancement and extension of features<br />