4. Confirmation Bias and the Erebus Disaster “ Those on the flight deck interpreted features of the topography ahead in such a way as to confirm their mindset, not challenge it. The flat slopes of Ross Island and Beaufort Island were misinterpreted as features associated with McMurdo Sound instead of Lewis Bay at the foot of Erebus.” (McGregor, 2006, p. 21).
5.
6.
7.
8. Possible Relationships between H and T H T H H H T T T Disjoint Overlapping Embedded Surrounding U U U U PTS is only really problematic when H is embedded in T
9.
10.
11.
12.
13.
14. Chance level * * ** * * Significantly worse than chance ** Significantly better than chance Mean Disconfirmatory Responses (out of 3)
15. Summary of findings for all studies t (28) = -2.20, p < .05 .71 1 2 13 13 29 example t (16) = -1.57, n.s. .69 0 3 6 8 17 no instruction t (45) = -2.73, p < .05 .70 1 5 19 21 46 cognitive psychology students 5. example and no instruction t (17) = -.89, n.s. .78 2 2 4 10 18 student pilots 4. example t (20) = 6.48, p < .001 2.00 5 11 5 0 21 orienteers 3. no instruction t (35) = -4.78, p < .001 .56 0 1 18 17 36 cognitive psychology students 2. lecture on confirmation bias t (65) = -4.90, p < .001 .58 0 8 22 36 66 student pilots 1. no instruction 3 2 1 0 Difference from chance M disconfirmatory choices n disconfirmatory choices (0 to 3) Participants Study
19. Railway line with branch line leading off east Large pine plantation either side of road and railway 24.1% Flying Picnic area with stream running under road Picnic area with stream running under road 46.5% Motorcycle Small aircraft landing behind town High bush clad peak behind town 8% Sea Most common choice Correct landmark Correct Scenario
20.
21.
22.
Notas do Editor
Abstract When pilots become lost they are often faced with a situation that is analogous to Wason's (1960) famous rule discovery task. Wason showed that people often attempt to tackle the task using suboptimal strategies that make it difficult to discover the underlying rule. We examined the possibility that student pilots’ domain specific knowledge of navigation would protect against the use of these suboptimal strategies. In a series of five navigation-oriented studies we found that not only did pilots and non-pilot students exhibit classic confirmation bias but that even quite explicit instructions to use optimal strategies did not improve performance. Interestingly we found evidence to suggest that domain-specific experience can protect against suboptimal strategy use – a sample of orienteers performed significantly above chance in the same navigation tasks. The theoretical and practical significance of these findings is discussed.
Roughly speaking, confirmation bias is tendency to seek out or interpret new information in a way that confirms one's preconceptions, but avoid information that contradicts them. Confirmation bias means finding what one is looking for – the idea is often considered to be a, possibly unconscious, motivational bias where one has a goal of bolstering a preferred hypothesis. But most research on confirmation bias shows that CB, if it occurs at all, is more likely to be a cognitive bias – a byproduct of our normal way of reasoning about the world.
Air New Zealand flight TE901, 28 th November 1979, 257 people died. “ Those on the flight deck interpreted features of the topography ahead in such a way as to confirm their mindset, not challenge it. The flat slopes of Ross Island and Beaufort Island were misinterpreted as features associated with McMurdo Sound instead of Lewis Bay at the foot of Erebus.” (p. 21 of McGregor, A. (March, 2006). Accidents, failures, mistakes, and leaky buildings . A Paper presented at the National IPENZ Conference, Wellington, New Zealand.
The term confirmation bias (CB) is used to describe a lot of different ideas (Nickerson, 1998) The kind of CB we are interested in cognitive not motivational in nature.
Wason referred to the tendency for Ps to ask ‘yes’ questions in order to confirm their hypothesis as enumerative induction (the use of confirming evidence only). But Wetherick (and later Klayman & Ha) pointed out that eliciting a ‘no’ answer could also be consistent with a confirmatory strategy (and a ‘yes’ answer might be consistent with a disconfirmatory strategy). One might try to confirm H1 by using a triple NOT consistent with H1 and seeking a no response. One might try to disconfirm H1 by using seeking a positive response to an inconsistent (with H1) triple. Thus, one needs to distinguish between a positive test strategy (which can lead to disconfirmation/falsification) and an attempt to confirm one’s favoured hypothesis. In the RDT these two phenomena are highly correlated but in other situations they need not be. Poltiek (1996) showed that participants find it difficult to make sense of the instruction to use a falsificatory strategy (try to prove false) one’s best guess. She argues that this is an incoherent idea – one’s best guess is, by definition, what one thinks is likely to be the case; thus, one can be expected to find the correct rule by attempting to disprove one’s best guess about that rule. Participants use both positive and negative testing to try and gather evidence for their best guess. Evans’ (1989, 2006) heuristic/analytic framework shows that confirmation bias (a tendency to think one’s hypothesis is correct because of the strategies used to evaluate it) occurs, not because we have a confirmation bias tendency, but rather because of the operation of three principles: singularity (people typically can only focus on one hypothesis), relevance (the hypothesis is the most plausible or probable in the context), and satisficing (the hypothesis is evaluated in terms of current goals and accepted if satisfactory).
When PTS goes wrong … When H is embedded in T … PTS cannot distinguish R T from R H Only a negative testing strategy (NTS) can In context of the Rule Discovery Task PTS: {14, 16, 18} or {8, 10, 12} All triples consistent with H are also consistent with T NTS: {3, 5, 7} or {9, 10, 11} or {20, 30, 40} By selecting a triple that one does NOT expect to be consistent with R H and getting a YES answer one discovers that is R H false
Check the time elapsed since the last position fix and estimate the distance covered in that time. On the chart draw a line of position [LOP] , across the track (the original or an intercept), at the estimated distance from the last fix. The line should extend about 1 nm either side of track, for each 5 minutes flown since the fix. i.e. if it is 30 minutes since your last positive fix then the line will extend roughly 6 nm either side. Then draw a rough circle with the LOP as the diameter (refer diagram below) and your most probable position [MPP] is somewhere within that circle of uncertainty. Find the most prominent features on the map within the circle and then try to locate them on the ground. The 1 nm per 5 minutes is based on ground speeds around 50 or 60 knots, if ground speeds are around 100 knots then make it 2 nm per 5 minutes. 'Most probable' means maybe an 80% chance. &quot;Read from ground to map!&quot; Normally in flight the navigator should be continually identifying features on the map and waiting for the next one to come up on track, within an estimated time. When uncertain of position the procedure is reversed, look for two or more large features on the ground and then identify features on the chart that are in the same juxtaposition. Prominent line features are best although, quite often, a spot feature is easily identified – for example the names of grazing or farming properties are shown on the charts and their owners, particularly those with an airstrip, often paint the name on a roof, in large letters. If you see a prominent line feature, then fly along it until you can derive a fix from an intersect or a verifiable landmark.
We carried out 5 separate studies in which participants were asked to imagine they were lost, although they were probably located within a red circle drawn on the map (the circle represented their theorised location). Participants were told they could see three distinctive features from where they were truly located and to select one feature that was most useful to determine whether they were in the circled location. They were told there could be dire consequences of wrongly believing they were in the circle, if in reality they were not.
Diagnosticity = how good a test is at distinguishing between two options.
Why are orienteers better than pilots? Why were pilots no better than non-experts? Evidence mixed on power of expertise to improve performance on RDT and related tasks Gorman et al. clergy better than scientists on RDT But expert software engineers better than novices (Teasley et al. 1994) – although still strong evidence for positivity bias and insufficient testing by experts. Theorists argue that people find it very difficult to naturally consider multiple hypotheses Evans’ principle of singularity But this does increase performance (dual goal studies). DG = DAX and MED examples. Allows positive testing to be used successfully Are orienteers inclined to consider dual goals? Are they used to being put in situations where they are ‘tricked’ to think they are located where they are not? Could this drive a natural inclination to dual goal use?
What is the best solution for training pilots? Encouraging negative testing is difficult Ps often don’t even think it (a falsificatory strategy) is possible (Poltiek, 1996) Klayman & Ha (1987) suggest we have a positivity bias because PTS works well in so many situations Difficult to consider more than one hypothesis (Principle of singularity) Many lost situations are perfectly amenable to a single hypothesis, positive testing strategy Therefore not a lot of demand for dealing with ‘embedded situations’ Compare that to orienteering where embedded situations may well be common
Use highly diagnostic landmarks wherever possible (e.g., hills with trees on is less useful than farm homestead with the name ‘Old McDonald’ on the top) Use multiple landmarks to increase chance of use of a unique landmark Individuate landmarks as much as possible to increase diagnosticity (a map may symbolise two different looking landmarks in the same way – e.g., two settlements. Thus, a pilot can use their own experience to further individuate landmarks or, if this is not possible, try to use landmarks that ARE individuated on a map – e.g., odd shaped roads or rivers).