How to Reduce Analytical Bias by Applying Analysis of Competing Hypotheses (ACH)
1. The Intelligence Collaborative
http://IntelCollab.com #IntelCollab
Poweredby
How to Reduce Analytical Bias
by Applying Analysis of
Competing Hypotheses (ACH)
A Complimentary Webinar from Aurora WDC
12:00 Noon Eastern /// 24 July 2013
~ featuring ~
Dan Mulligan Michel Bernaiche
2. The Intelligence Collaborative
http://IntelCollab.com #IntelCollab
Poweredby
Dan Mulligan
Dan Mulligan is on the faculty of the Academy of
Competitive Intelligence (ACI) and has presented at SCIP
conferences every year since 2007. At the 2011 SCIP
Conference in Orlando, Florida, he was recognized with the
“Best Conference Presentation”. His corporate experience
includes work in the medical, pharmaceutical,
manufacturing, and consulting industries. Through the Dale
Carnegie organization, he trained, taught, and consulted
with over 200 of the Fortune 500 companies over the past
19 years, focusing on leadership and communications.
Email: DanMulligan82@gmail.com
The Intelligence Collaborative is the online learning and networking
community powered by Aurora WDC, our clients, partners and other friends
and dedicated to exploring how to apply intelligence methods to solve real-
world business problems.
Apply for a free 30-day trial membership at http://IntelCollab.com or learn
more about Aurora at http://AuroraWDC.com – see you next time!
3. The Intelligence Collaborative
http://IntelCollab.com #IntelCollab
Poweredby
α Use the Questions pane on your
GoToWebinar control panel and all questions
will be answered in the second half of the
hour.
α You are welcome to tweet any comments on
Twitter where we are monitoring the
hashtag #IntelCollab or eavesdrop via
http://tweetchat.com/room/IntelCollab
α Slides will be available after the webinar for
embedding and sharing via
http://slideshare.net/IntelCollab
α To view the recording and download the PPT
file, please register for a trial membership at
http://IntelCollab.com.
Questions, Commentary & Content
4. The Intelligence Collaborative
http://IntelCollab.com #IntelCollab
Poweredby
Agenda
► Competitive Intelligence Process
► Analytic problems in CI
► Analysis of Competing Hypotheses (ACH)
► Examples
► Summary, Q&A and Discussion
7. The Intelligence Collaborative
http://IntelCollab.com #IntelCollab
Poweredby
Information and Intelligence
7Introduction to Intelligence
Information Intelligence
Scattered Bits of Data
Evaluated, Filtered, Distilled,
Analyzed Information
Unrefined
End Product of a Complex
Process
Raw & Unfinished Basis for Action
19. The Intelligence Collaborative
http://IntelCollab.com #IntelCollab
Poweredby
Analysis Of Competing Hypotheses
►Richards J. Heuer Jr.
►45yr. CIA veteran
►Developed ACH after
studying cognitive
psychology Richards J. Heuer Jr.
Photo courtesy the Center for the Study of Intelligence
21. The Intelligence Collaborative
http://IntelCollab.com #IntelCollab
Poweredby
Analysis Of Competing Hypotheses
• Reduces cognitive
biases
• Logical
• Auditable
Strengths
• Snapshot in time
• Dependent on validity of
evidence
• Time consuming
Weaknesses
25. The Intelligence Collaborative
http://IntelCollab.com #IntelCollab
Poweredby
PARC Software
►Developed by the
Palo Alto Research
Center (PARC)
►ACH2.0.3
►Developed in
collaboration with
Richards J. Heuer,
Jr.
D
d http://www2.parc.com/istl/projects/ach/ach.html
44. The Intelligence Collaborative
http://IntelCollab.com #IntelCollab
Poweredby
Dan Mulligan
Thank you! Now how about a
little Q&A?
Email: DanMulligan82@gmail.com
The Intelligence Collaborative is the online learning and networking
community powered by Aurora WDC, our clients, partners and other friends
and dedicated to exploring how to apply intelligence methods to solve real-
world business problems.
Apply for a free 30-day trial membership at http://IntelCollab.com or learn
more about Aurora at http://AuroraWDC.com – see you next time!
Notas do Editor
Richards Heuer is the author of Psychology of Intelligence Analysis. Heuer spent 45 years gaining experience in intelligence and security working for the Central Intelligence Agency (CIA). In addition to publishing work on the psychology of intelligence analysis, he also published works on counterintelligence, deception, analytical methodologies, and personnel security issues.
The scientific method is a proven process based on the rejection of hypotheses in which the analyst accepts only those which are not disproven. Traditional intelligence analysis does not seek to refute hypotheses, but tends to seek confirmation of initial judgments. Cognitive biases are mental errors caused by simplifying information processing strategies. Emotional or intellectual predispositions toward certain judgments do not cause cognitive biases; rather, cognitive biases are the result of subconscious mental shortcuts for processing information.Analysis of Competing Hypotheses ACH assists analysts in identifying and questioning basic assumptions providing a means to test hypotheses. ACH is an eight-step procedure that allows analysts to leave an audit trail, unlike other decision-making processes. In order to prevent distortions caused by cognitive biases, Heuer’s method requires analysts to explicitly identify all reasonable hypotheses. Hypotheses compete for the analyst’s favor, eliminating satisficing because the analyst is working with all possible rational hypotheses. Typically, analysts identify what appears to be the most promising hypothesis before collecting and organizing information based on whether it supports or refutes the hypothesis the analyst has chosen. Often incongruent information is not fully exploited, nor are other hypotheses tested. When testing one hypothesis at a time, biases and mindsets are harder to dispel.
AuditableWith ACH, analysts note all evidence and hypotheses on paper, making the path from collection to final estimate easier to follow.Overcomes cognitive biasesIndividually analyzing each piece of evidence can give analysts a skewed view of the overall picture. Using ACH, disproving hypotheses provides a more effective method for avoiding established mental roadblocks.LogicalThe competitive aspect of ACH forces analysts to examine hypotheses more logically than separating them for reviewing one at a time.ACH is similar to Multi-Criteria Decision Making (MCDM).Although ACH is more effective than less-systematic, traditional approaches, no process is without drawbacks.Snapshot in timeThe evidence used in the analysis is static, while events are dynamic. In order to provide a timely analytic product, analysts must determine a point at which to halt collection in order to prepare the estimate. Dependent on validity of evidenceAnalysis is only as good as the evidence that goes into the process. Unreliable evidence can undermine even the most thoughtful process execution. As evidence can be ambiguous and prone to the pitfalls of deceptive tactics, it is difficult to determine reliability and validity of the inputs. No better than the skill of the analyst using the processThe degree to which ACH can reduce cognitive biases reflects the skill of the analyst using it. A successful analyst must be imaginative when generating hypotheses, keeping his mind open to all possibilities, rather than attempting to anticipate an outcome before examining the evidence. He must also carefully weigh the reliability and relevance of the evidence.Time consumingPerforming the ACH process manually can be long and arduous, especially when there are numerous hypotheses and a large body of evidence. Due to the amount of time that it takes to do this by hand and the time pressures analysts usually encounter, they do not apply the method as often as might be appropriate. For this reason, automation of ACH is critical and will be discussed in the next lesson.
Structure is provided by delving deeper into the previous hypothesis and further exploiting the evidence, or gathering more.Automation, and structure with PARC Software facilitates the analysis of the second level of hypothesis testingIn other words, structure enables analysis of complex problems consisting of several questions by deconstructing the complex question into simple questions with either yes, no answers or mutually exclusive answers. Then ACH is applied to the root question, and the remaining questions are added to the winning hypothesis and ACH is applied again. (Added by NSA, 1Sep06)
Using SACH eases the process for analysts from start to finish. SACH closes evidence gaps, creating a tighter finished product.Documenting every step in SACH later helps analysts easily follow the estimative process. Because SACH provides a trail of auditable information, should the analyst provide an incorrect estimate, it is easier to evaluate what went wrong.In conclusion:SACH allows analysts to coordinate which evidence to test with the hypotheses the analyst is testing. It also allows the analyst to deal more effectively with large amounts of evidence.SACH assists managers in supervising and monitoring analysts’ progress.SACH provides decisionmakers with higher-quality products, and makes it easier for them to see exactly how the analyst arrived at the final estimate. When presenting to decisionmakers, analysts will more effectively produce documents with the bottom lines up front (BLUF) because the automation of SACH helps analysts discover and prove the bottom line.SACH improves oversight, as products are more easily defendable by clearly tracing all thought processes back to the original evidence.
The software takes an analyst through a process for making a well-reasoned, analytical judgment. Usefulfor issues that require weighing of alternative explanations of what has/is/or may happenIt helps the analyst overcomesome cognitive limitations that make prescient intelligence analysis difficult. ACH is grounded in psychology, decision analysis, and the scientific method. It helps analysts protect themselves from avoidable errorImproves their chances of making a correct judgment
Analysis of Competing HypothesesClassic Intelligence Techniques
AuditableWith ACH, analysts note all evidence and hypotheses on paper, making the path from collection to final estimate easier to follow.Overcomes cognitive biasesIndividually analyzing each piece of evidence can give analysts a skewed view of the overall picture. Using ACH, disproving hypotheses provides a more effective method for avoiding established mental roadblocks.LogicalThe competitive aspect of ACH forces analysts to examine hypotheses more logically than separating them for reviewing one at a time.ACH is similar to Multi-Criteria Decision Making (MCDM).Although ACH is more effective than less-systematic, traditional approaches, no process is without drawbacks.Snapshot in timeThe evidence used in the analysis is static, while events are dynamic. In order to provide a timely analytic product, analysts must determine a point at which to halt collection in order to prepare the estimate. Dependent on validity of evidenceAnalysis is only as good as the evidence that goes into the process. Unreliable evidence can undermine even the most thoughtful process execution. As evidence can be ambiguous and prone to the pitfalls of deceptive tactics, it is difficult to determine reliability and validity of the inputs. No better than the skill of the analyst using the processThe degree to which ACH can reduce cognitive biases reflects the skill of the analyst using it. A successful analyst must be imaginative when generating hypotheses, keeping his mind open to all possibilities, rather than attempting to anticipate an outcome before examining the evidence. He must also carefully weigh the reliability and relevance of the evidence.Time consumingPerforming the ACH process manually can be long and arduous, especially when there are numerous hypotheses and a large body of evidence. Due to the amount of time that it takes to do this by hand and the time pressures analysts usually encounter, they do not apply the method as often as might be appropriate. For this reason, automation of ACH is critical and will be discussed in the next lesson.