Mais conteúdo relacionado
Semelhante a Smu seminar 2014_03_26 v3 (20)
Smu seminar 2014_03_26 v3
- 1. Fundamentals Matter – A Brief
Introduction to Risk Analysis for
Information Security
Southern Methodist University, March 26, 2014
Heather Goodnight, President
Patrick Florer, CTO
Risk Centric Security, Inc.
www.riskcentricsecurity.com
Authorized reseller of ModelRisk from Vose Software
Risk Centric Security, Inc. Confidential and Proprietary .
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Risk Analysis for the 21st Century®
- 2. • Introductions
• What we are going to talk about
o Why Fundamentals Matter / Current State
o Definitions
• Risk and the Risk Landscape
• Possibility and Probability
• Measurement
• Variability and Uncertainty
• Precision vs. Accuracy
• Scales of Measurement: Qualitative vs. Quantitative
• Not Enough Data
• Monte Carlo Simulation
• Modeling Expert Opinion and PERT distributions
Agenda
Risk Centric Security, Inc. Confidential and Proprietary .
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 3. Heather Goodnight is an accomplished Global Sales and Business Development
Consultant. Over the years, her unique, practical insight into problems of risk and
opportunity have provided important guidance for organizations both large and
small. She is a cofounder of Risk Centric Security and currently serves as President
of the Corporation. In 2010, she was appointed to the RIM Council (Responsible
Information Council) of the Ponemon Institute. In addition to her role at Risk
Centric Security, she serves as Business Development Manager at Triumfant, Inc.,
a vendor of advanced anti-malware products.
Patrick Florer has worked in information technology for almost 35 years. For 17
years, he worked a parallel track in medical outcomes research, analysis, and the
creation of evidence-based guidelines for medical treatment. His roles have
included IT operations, programming, and systems analysis. From 1986 until now,
he has worked as an independent consultant, helping customers with strategic
development, analytics, risk analysis, and decision analysis. He is a cofounder of
Risk Centric Security and currently serves as Chief Technology Officer. He is a
member of the Ponemon Institute RIM council. In 2012, he was appointed
Distinguished Fellow of the Ponemon Institute.
Introductions
Risk Centric Security, Inc. Confidential and Proprietary .
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 4. The Current State of Confusion …
.
Risk Centric Security, Inc. Confidential and Proprietary .
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 6. What is Risk?
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 7. What Risk Isn’t!
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Vulnerability Threat
- 8. Risk = Frequency x Impact
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Frequency
Impact
Risk
- 9. Risk and Opportunity
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 10. Possibility and Probability: Possibility
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 11. Possibility and Probability: Probability
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 12. What is a Measurement?
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 13. Properties of Measurement
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Validity
Reproducibility
Detail
- 14. Sources of Error in Measurement?
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Random Error
Errors from Bias
- 15. Variability and Uncertainty
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Variability
Uncertainty
- 16. Precision and Accuracy
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 17. Scales of Measurement
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Qualitative Quantitative
- 18. Qualitative Scales
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Nominal/Categorical
IntervalOrdinal
HIGH - Red
MEDIUM - Orange
LOW - Green
First, Second, Third … On a scale of …
- 20. Problems with Qualitative Scales
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
My Scale
High
Medium
Low
Red
Orange
Green
Your Scale
High
Medium
Low
Red
Orange
Yellow
Green
(RED – GREEN + MEDIUM) / Somewhat Likely
= ???
Mismatched Scales
Meaningless Calculations
Assessor Disagreements
- 21. Problems with Qualitative Scales
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Boundary Problems
$2.5M Loss Exposure = Moderate = Yellow
$2.5M Loss Exposure = Moderate = Yellow
$2.5M Loss Exposure = Moderate = Yellow
$7.5M Aggregate Loss Exposure = not so Moderate !
Issues with Loss of Information
- 22. Quantitative Scales
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
2 + 2 - 1 = 3
360 * 10 = 3,600
Sqrt(25) = 5
f(x) = y
etc.
- 23. Qualitative Methods - Problems
Difficulty with arithmetic and statistical operations
From ISO 17999
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 24. Qualitative Methods - Problems
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 25. Qualitative Methods - Problems
On a scale of 1 to 5,
where 1 = least and 5 = most,
please rate …
Likert scale (From Wikipedia, the free encyclopedia)
When responding to a Likert questionnaire item, respondents
specify their level of agreement or disagreement …
In so doing, Likert scaling assumes that distances
on each item are equal …
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 26. Data
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Good Data Bad Data
Big Data
Little Data
- 27. How much data is enough data?
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
How do I get to the mall?
How do we build this?
vs.
- 28. Data from Calibrated Estimates
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
More often than you might think, the data we have to work with
comes from Subject Matter Experts (SME’s).
How can we improve the accuracy of these SME’s – to a 90%
confidence level?
With calibration.
Example: How much does an iPhone 5s weigh?
- 29. Monte Carlo Simulation
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
The average = $12,500
$2,500 $12,500 $32,000
The range is:
The distributions are:
- 30. Monte Carlo Simulation
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 31. The Beta Pert Calculator
Minimum:
What is the least or lowest (best or worst) numerical
estimate that you believe to be reasonable? This will be the
smallest number you come up with.
Most Likely:
What is the most likely or most probable numerical estimate
in your opinion? This number must fall between the
minimum and maximum. It may equal either the minimum
or the maximum, but should not equal both
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 32. The Beta Pert Calculator
Maximum:
What is the greatest or highest (best or worst)
numerical estimate that you believe to be
reasonable?
Note that “best” or “worst” case estimates could be
either minimum or maximum values, depending upon
the scenario.
In a risk / loss exposure scenario, lower is better, so the
minimum represents the lowest loss, or best outcome.
The maximum represents the highest loss, or worst
outcome.
In a sales or opportunity scenario, it’s the reverse:
lower is not better, so the minimum represents the
worst case. Higher is better, so the maximum
represents the best case.
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 33. The Beta Pert Calculator
Confidence:
On a scale that includes “Very Low”, “Low”, “Average”,
“High”, and “Very High”, how confident are you in the
accuracy of your estimates?
This parameter controls the sampling around the most likely
value, and thereby also controls the height of the histogram
or slope of the cumulative plot.
For most analyses, using “Average” for the confidence
parameter works well. In this instance, “Average” really
means having no strong feeling about the matter – being
evenly divided between under-confidence and over-
confidence.
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 34. The Beta Pert Calculator
Percentile Tables
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 35. The Beta Pert Calculator
Percentile Tables
1% of values are <= 10,044 and 99% are > 10,044
10% of values are <= 11,120 and 90% are > 11,120
20% of values are <= 11,658 and 80% are > 11,658
50% of values are <= 13,025 and 50% are > 13,025
The 50th percentile has another name - it’s
called the Median.
The Median is the mid-point in a list of values -
half of the values in the list are less and half
are greater than the Median.
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 36. The Beta Pert Calculator
Histogram
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 37. The Beta Pert Calculator
Cumulative Plot
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 38. Thank you !
Heather Goodnight
President and Cofounder
Patrick Florer
CTO and Co-founder
Risk Centric Security, Inc
patrick@riskcentricsecurity.com
214.828.1172
Authorized reseller of ModelRisk from Vose Software
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
Risk Analysis for the 21st Century ®
- 39. ”We don’t have enough data!” - Sources
Open Security Foundation: datalossdb and osvdb
http://www.opensecurityfoundation.org/
Office of Inadequate Security:
http://www.databreaches.net/
Identity Theft Resource Center:
http://www.idtheftcenter.org/
ISACA: www.isaca.org
ISSA: www.issa.org
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 40. ”We don’t have enough data!” - Sources
Mitre Corporation: www.mitre.org
OWASP: http://owasp.com/index.php/Main_Page
Privacy Rights Clearing House:
http://www.privacyrights.org/
SANS: www.sans.org
The Ponemon Institute: www.ponemon.org
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 41. ”We don’t have enough data!” - Sources
Conference procedings: Black Hat, RSA, Source
Conferences, BSides
Internet tools:
Search engines: Google, Bing, Yahoo, Ask.com
Trend Analyzers:
Google trends:
http://www.google.com/trends
Twitter Trends: www.trendistic.com
Amazon:
http://www.metricjunkie.com/
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.
- 42. ”We don’t have enough data!” - Sources
Securitymetrics.org – mailing list
Society of Information Risk Analysts (SIRA)
Books:
How to Measure Anything – Hubbard
The Failure of Risk Management – Hubbard
Risk Analysis: A Quantitative Guide – Vose
Clinical Epidemiology and Biostatistics – Kramer
Data-Driven Security: Analysis, Visualization and Dashboards – Jacobs and
Rudis
Risk Centric Security, Inc. Confidential and Proprietary.
Copyright © 2014 Risk Centric Security, Inc . All rights reserved.