DevSecOps, Application security at scale. Delta Analysis and edgescan. Full stack security. OWASP California 2016

1. Skillful, Scaleful Fullstack
Security in a state of constant
flux
with……

2. Eoin Keary
CTO/Founder edgescan.com
OWASP Leader/Member/Ireland Founder
OWASP Global Board Member (2009-2014)

3. One problem, Many solutions
DAST – Peoples front of Judea
RASP – Judean peoples front
IAST - Judean Popular People's Front
SAST - Popular Front of Judea
VA - Vulnerability Assessment / Known
Vulnerability Scanning

4. Web Risk
• Application Security
• Host Security
• Both / Either / Or
• It’s all software right.

5. Full-Stack!

6. Appsec VA
Threat
Intel
ENdPoint

7. Market Driven Approaches to a Market
Driven Problem.

8. Segregated Industry
• Developers Vs Security
• Admin Vs Developers
• Security Vs Admin

9. Convergence
Developer ←→ Security ←→ Admin
DevSecOps
AppSec + HostSec -> Fullstack

10. Divergence
Application Security Industry
Vs
Vulnerability Analysis Industry
Market Driven Separation Vs Risk

11. Do it earlier
Yadda, Yadda Yadda
Cheaper, efficient, coverage etc etc etc
10x, 20x better – 1998 IBM
20 years later – same sh1t

12. Agile Risk Model
Fail Early – Fail Often
“Push Left”

13. Continuous what?
CI -> Continuous Integration
CD -> Continuous Deployment
TDD -> Test Driven Development
Continuous Maintenance
Continuous Security

14. Continuous Security
“Keeping up” with development
Assisting secure deployment
Catching bugs early – Push Left
Help ensure “change” is secure

15. Host/Server/Framework
Building bricks – Frameworks / Components
Spring, JQuery, Jade, Angular, Hibernate
> 30 billion Open source downloads 2015
90% of application code is framework
63%* don’t monitor component security
43%* don’t have open source policy
* http://www.sonatype.com/about/2014-open-source-software-development-survey

16. Components
As of October 2015 -
Spring (3.0-3.05) – CVE-2011-2894 – Code exe
7,000,000 downloads since vuln discovered
CVSS: 6.8
Apache Xerces2 – CVE-2009-2625 – DoS
4,000,000 downloads since vuln discovered
CVSS: 5
Apache Commons HttpClient 3.x - CVE-2012-5783 – MiTM
4,000,000 downloads since vuln discovered
CVSS: 4.9
Struts2 (2.0-2.3.5) – CVE-2013-2251-Remote Cmd Injection
179,050 downloads since vuln discovered
CVSS: 10

17. “63% of vulnerabilities discovered in 2015 by
edgescan were outside of software developer
control – Operating System CVE, Component CVE,
Misconfiguration etc ..”
- edgescan Vulnerability Statistics Report 2015

18. AppSec/Component Sec
• “If you're not doing component vulnerability
management you’re not doing appsec…”
– 90% of application code is framework
• “If you’re not doing full-stack you are not doing
security…”
– Hackers don’t give a S*#t

19. Problems?
Security in a constant state of flux.

20. “We Can” scale..
Automation of assessment
Depth
Coverage / Breadth
Rigour

21. SCALE!
Automation
Event Driven
Frequent/Scheduled
Build Build Build

22. Automation!!
• Jenkins, Hudson, Bamboo
– Event driven
– Scheduled
– Incremental
• CHEF, Puppet, Cloud(immutable)
Sounds great…. but

23. Automation and Integration
• Automation can detect technical
vulnerabilities
– Misuse of code
– Coding Bugs
– Implementation Mistakes

24. Automation and Integration
• Automation can NOT detect Logical
vulnerabilities
– Business Logic
– Backdoors (E.g. Juniper, Fortinet)
– Provide Risk measurement
– Business Context

25. Accuracy/Information/Context
The “Anti-Scale”
Risk/ Business Context
Information Vs Data
Human Decisions and Intel
Technical constraints
-> Chokepoints

26. The “Anti-Scale”
New languages and programming methods
Growth of interpreted languages with no strong typing
(Javascript, Ruby,…) – “hurts” SAST
Few automated tools to test APIs / RESTful APIs
Testing Window is squeezed, manual testing is
doomed!?

27. Fighting The “Anti-Scale”
Accuracy
“Rule Tuning” – DAST & SAST
Build Fails!
White Noise Suppression
Real Security Vs “Best Practice”
Updates to Rules
Scale
“Delta Analysis”
Previous Vs Current
Changes
FP’s / FN’s

28. SAST Integration
• Analysis without Runtime - SAST
• More than just tooling
• Management Lifecycle
– Rule Management & Tuning / False Positives
• Cant cover Vuln Taxonomy –Blindspots

29. SAST Blindspots
• Storage and transmission of confidential
information
• Logic: Authentication, brute force attacks,
effectiveness of password reset etc.
• Logic: Privilege escalation and insufficient
authorization. Business Logic
• Data privacy: data retention and other compliance
(e.g. ensuring credit card numbers are masked
when displayed) - context

30. CI Integration
• Rule Tuning!!
• Technical Vulnerabilities
• Logical Vulnerabilities
• Feedback Loop
• Build Fails
• Root Cause Metrics
• All Vulns are not equal!

31. DAST Tool/Runtime Vulnerability
Management - Pitfalls
• Coverage Depth – can be shallow
• App Complexity - enemy
• Logical vulns – poor
• “Trial and error” testing

32. Vulnerability Assessment (Host)
• Easy to perform, Harder to manage
• First assessment
– higher work effort
– establish coverage (Reduce FN’s)
– Weed out FP’s
• Delta Analysis – Previous Vs Current
• Vulns beyond your control

33. Component Security
Don’t forget….
• Unpredictable (Like Host Security).
• Requires frequent/continuous vigilance.
• Fix can be difficult and not backward
friendly

34. Delta-Metrics
• Vuln type - (CVE, OWASP, WASC,
SANS..)
• Tech Stack - (Code, F-work, Host etc)
• Layer - (App/Host)
• Root Cause - (Code, Patch, Config /
Deploy)
– Technical, Logical/Behavioural Vuln

35. Continuous Asset Profiling
• Detect Global Estate Changes
– New / Dead active IP’s
– Service Changes (Ports open / enabled).
– Perimeter Change – Firewall/ACL changes
– Rogue deployments

36. Fighting The “AntiScale” - Delta Analysis
Measure of change in a target environment.
Focusing on change in risk posture compared to last assess
-> Closed, New, False Positives

37. Fighting The “Anti-Scale”-
Testing like a Developer
Break testing into little pieces
Smoke / Incremental Vs full regression
testing
“Early and Often”
– Continuous, on demand
– Testing duration drives testing frequency

38. Business & Behavioural Testing
At scale:
Can be Difficult …..
Technical Security is covered by “tuned”
Automation…..
More Time to “Deep Dive”

39. “Future of Pentesting”
• Push towards Technical Vulnerabilities rooted
out using technical methods/services …..
• Push from time chasing Top 10 (SQLI, XSS,
etc) -To- Behavioural, Logical, Business flow
assessment.
• Constant flux requires constant assessment.
• Point-in-time is dead?

40. FIN
• We can scale but not everything is [easily]
scalable
• Discover Tech Vulns using Tech
• Consider full-stack, don’t let marketing dictate risk.
• Lets test to mirror DevOps
• Convergence is necessary to address issue.
@eoinkeary
eoin@bccriskadvisory.com