Ast in CI/CD by Ofer Maor at DevSecCon Boston 2017

1. Join the conversation #DevSecCon
OFER MAOR
*AST in CI/CD
Making it Work!
Director, Enterprise Solutions
@OferMaor
linkedin.com/in/ofermaor
ofer.maor @ gmail.com

2. Speaker
• Enterprise Solutions @ Synopsys
• Over 20 Years in Cybersecurity
• Hacker at Heart
• Longtime OWASPer
• Pioneer of IAST
• DevSecOps/DevOpsSec Fan!
• Avid Photographer
Singapore Skyline, Feb 2017. Taken at DevSecCon Singapore!

3. Glossary
• Interactive Application Security Testing (IAST)
• Dynamic Application Security Testing (DAST)
• Static Application Security Testing (SAST)
• Building Security In Maturity Model (BSIMM)
• Maturity Action Plan (MAP)
• Software Security Initiative In-a-Box (SSIB)
• Continuous Integration/ Continuous Delivery and
Deployment (CI/CD)
About Synopsys

4. Agenda
• Background – The *AST Landscape
• The CI/CD Challenge
• *AST Solutions for CI/CD – What works and what doesn’t
• Building the right mix – How to make it Work!

5. The *AST Landscape

6. Software Security is a Journey
Evolving• Augmenting internal teams with external resources for
scalability
• Identify and prioritize vulnerabilities for remediation
• Integrating with DevOps
TACTICAL STRATEGIC
BOLT
ON
BUILT
IN
Starting
• Pen testing to find vulnerabilities
• Compliance driven
• Low level testing
• Programmatically managing risk across your software release cycles
• Driving efficiencies through SDLC integration
• Purposeful blend of automated and manual testing processes
Optimizing
DEVELOPMENT WORKFLOW
FIND REMEDIATE PREVENT

7. The *AST Landscape (and some more…)
• SAST
• DAST
• IAST
• MAST
• SCA
• Fuzz Testing
• Penetration Testing
• …

8. Confused?
You’re not Alone!
Let’s first figure
out what it all
means.

9. SAST – Static Application Security Testing
AKA: Static Code Analysis for Security
• Analyzes code to identify vulnerabilities
• Most prevalent AST solution today
• Challenges
• Potential FPs
• May require tuning and configuration
• Hard for use for security professionals
• Offered in various flavors:
• Analysis of (uncompiled) source code
• Analysis of code & build
• Analysis of binary code
• Managed Service / Tool / IDE Plugin
cond2
*a = b free(a)*a = b free(a)
cond2
b = 10a = malloc(10)
cond1

10. DAST – Dynamic Application Security Testing
AKA: Web Application Scanner (Black Box)
• Sends HTTP tests to test running application
• Longest used AST technology
• Challenges
• Accuracy of results
• Not suited for dev – no code guidance
• Performance (long testing times)
• Offered in various delivery forms:
• On Premise
• Cloud
• Managed Services
• Included in Professional Services
?????

11. IAST – Interactive Application Security Testing
AKA: Runtime Code Analysis
• Runtime code analysis through instrumentation
• Youngest AST technology
• Challenges
• Deployment of agents on tested servers
• Requires integration into dev/devops environments
• Coverage influences by what’s executed
• Comes with various “interpretations”
• Inline/Passive IAST (Based on existing traffic)
• Active IAST (Including HTTP Inducer)
• DAST Add-on Only
• RASP Add-on
Database
Back End
Front End
HTTP/s
DATA WS
SQL DATA
ODBC
IAST

12. MAST – Mobile Application Security Testing
AKA: It’s really a bunch of stuff bundled in one name…
• Under MAST, we can find….
• Server side WS analysis – with SAST/DAST/IAST/etc.
• Mobile code SAST
• Mobile binary analysis (3rd party too…)
• Mobile Behavioral Analysis (3rd party too…)
• Reputation testing
• SDK & Opensource
• More…
• Challenges
• Broad set of problems
• Still evolving…

13. SCA – Software Composition Analysis
AKA: Open Source Library Scanning
• Searches known open source (and closed
source) components in applications
• Rapidly growing testing segment
• Challenges
• Additional technology on top of other *AST
• Very broad scope
• Offered in different flavors
• Binary Analysis for Supply Chain and 3rd Parties
• Source Analysis for home grown security and
licensing
• On-premise / Cloud options

14. How Does it All Fit?
REQUIREMENTS
& DESIGN
Architecture Risk
Analysis
Security Code
Design Analysis
Threat Modeling
TRAINING
Core Security Training
Secure Coding Training
eLearning
SAST (IDE)
SAST (Build)
SCA (Source)
IAST
IMPLEMENTATION
SAST (Managed)
Fuzz Testing
SCA (Binary)
Mobile Testing
VERIFICATION
DAST (Managed)
Pen Testing
Network Pen Testing
RELEASE

15. The CI/CD Challenge

17. CI/CD, DevOps, DevSecOps, WHAT?!

18. Continuous Everything

19. CD Extreme
• Multiple production
updates per day
• Multiple CI streams
• A/B UAT Testing
• Parallel testing and
deployment
• No place for outsiders
Source: Wikipedia

20. *AST in CI/CD

21. Criteria
• Speed
• Integration
• Ease of Use
• Relevance (Accuracy)
• Actionability

22. SAST – Static Application Security Testing
AKA: Static Code Analysis for Security
• Speed Instant to Hours (by Flavor)
• Integration IDE, Build, Binary
• Ease of Use Varies. Can be Complex
• Relevance Can be overwhelming
• Actionability Right on. Points to Line of Code

23. SAST Flavors
• IDE “Spellchecker” Lightweight, Instant
• In-IDE Incremental Pre-checkin, Minutes
• Integration/Build CI, Minutes to Hours
• Binary Analysis Post Build. Hours
• Managed Service External. Days

24. • Speed Hours to Days
• Integration Not Really…
• Ease of Use Requires some security skills
• Relevance Focus on Front end (but some FPs)
• Actionability Difficult.
DAST – Dynamic Application Security Testing
AKA: Web Application Scanner (Black Box)

25. • Speed Instant to Hours (by Flavor)
• Integration Test Automation
• Ease of Use Easy (once deployed)
• Relevance Very relevant. Actual executed LoC
• Actionability Right on. Points to Line of Code
IAST – Interactive Application Security Testing
AKA: Runtime Code Analysis

26. IAST Flavors
• Inline/Passive Lightweight, Instant
Integrates with Existing Tests
• Active Minutes (Incremental) – Hours
Requires dedicated testing

27. • Speed Minutes to Hours
• Integration IDE, Build, Binary
• Ease of Use Fairly Easy
• Relevance Hard to determine actual impact
• Actionability Not always straight forward
SCA – Software Composition Analysis
AKA: Open Source Library Scanning

28. Making it all WORK!

29. Key Principles
• If You Can’t Beat Them – Join Them!
• Automation, Automation, Automation
• Alt-Ctrl, Shift-Left (But not just…)
• Multiple Technologies, Multiple Flavors, Multiple Times!
• Parallel Processes in Parallel Speeds
• You’re Going to HAVE TO Live with some Risk

30. Risk Appetite
“Amount and type of risk
that an organization is
prepared to pursue, retain
or take”
Source: ISO 31000 risk management standard

31. Understanding Risk
• Exploitable Vulnerabilities
• Misc. Vulnerabilities
• Potentially Vulnerable Code
• Insecure Coding Practice
• Bad Coding Practice
• Public Front End
• Limited Access Front End
• Back End / Internal

32. Making it All Work!
• Use “Instant/Passive” solutions as much as possible
• In-IDE ”Spell Checker” Static Analysis
• Inline IAST
• Define PRACTICAL policies for ”Hard” and “Soft” gates:
• Hard Gates – Stop the process
• Soft Gates – Put in motion a correction process
• Use ”Layers” of testing at different stages

33. IDE “Spell
Checker”
Incremental
IDE SAST
Inline
IASTIn IDE
SCA
Verification
DAST
Active
IAST
Full Scan
SAST
Full Scan
SCA

34. Fast vs Slow
• Rely heavily on integrated/fast technologies
• Key criteria – “does not get in the way”
• Define practical blocking criteria – be realistic
• All the rest – In the backlog

35. Accept A/B Testing
• Gradual A/B Testing is replacing “Test Environments”
• Manage A/B Testing exposure as part of risk management
• Use it! A/B Testing gives you the best test environment
• Create the right “Retro” gates by risk:
• High – Block propagation and roll back
• Medium – Block propagation until fix is delivered (but don’t roll back)
• Low – Continue propagation but with a fix following right up

36. Summary
• Software Security testing is complex, even more so in CI/CD
• Unfortunately – There’s no “One Ring to Rule them All”
• You have to build your *AST workflow and pipeline:
• Work closely with R&D & DevOps
• Use multiple tools and multiple technologies
• Work in parallel tracks and speeds
• Manage your risk!

37. Join the conversation #DevSecCon
Thank You!
Questions?
@OferMaor
linkedin.com/in/ofermaor
ofer.maor @ gmail.com
Solar Eclipse
San Francisco
2017