This is a talk about data science operations and the applications of Risk I/Os insights to the security industry - how we went about mining insights from our large dataset
11. “It is a capital mistake to theorize
before one has data.
!
!
!
!
Insensibly, one begins to twist
facts to suit theories, instead of
theories to suit facts.”
12. C(ommon) V(ulnerability) S(coring) S(ystem)
“CVSS is designed to rank information
system vulnerabilities”
Exploitability/Temporal (Likelihood)
Impact/Environmental (Severity)
The Good: Open, Standardized Scores
13. FAIL 1: A Priori Modeling
“Following up my previous email, I have tweaked my
equation to try to achieve better separation between
adjacent scores and to have CCC have a perfect (storm) 10
score...There is probably a way to optimize the problem
numerically, but doing trial and error gives one plausible set
of parameters...except that the scores of 9.21 and 9.54 are
still too close together. I can adjust x.3 and x.7 to get a
better separation . . .”
14. 2: Data Fundamentalism
Since 2006 Vulnerabilities have declined by 26 percent.”
http://csrc.nist.gov/groups/SNS/rbac/documents/vulnerability-trends10.pdf
!
!
The total number of vulnerabilities in 2013 is up 16 percent
so far when compared to what we saw in the same time
period in 2012. ”
http://www.symantec.com/content/en/us/enterprise/other_resources/b-
intelligence_report_06-2013.en-us.pdf
23. I Love It When You Call Me Big Data
50,000,000 Live Vulnerabilities
1,500,000 Assets
2,000 Organizations
24. I Love It When You Call Me Big Data
15,000,000
Breaches
25.
26.
27. Baseline Allthethings
Probability
(You Will Be Breached On A Particular Open Vulnerability)?
=(Open Vulnerabilities | Breaches Occurred On Their CVE)
/(Total Open Vulnerabilities)
2%
28. Probability A Vuln Having Property X Has Observed Breaches
RANDOMVULN
CVSS 10
CVSS 9
CVSS 8
CVSS 6
CVSS 7
CVSS 5
CVSS 4
Has Patch
0.000 0.010 0.020 0.030 0.040
33. Defend Like You’ve Done It Before
Groups,
Motivations
Exploits
Vulnerability
Definitions
Asset
Topology,
Actual Vulns
on System
Learning
from
Breaches
34. Probability A Vuln Having Property X Has Observed Breaches
RandomVuln
CVSS 10
Exploit DB
Metasploit
MSP+EDB
0.0 0.1 0.2 0.2 0.3
35.
36. Data is Everything and Everything is Data
Spray and Pray = 2%
CVSS 10 = 4%
Metasploit and Exploit DB = 30%