This document summarizes a talk on using hackers versus security tools in the software development lifecycle (SDLC). It discusses how hackers can provide a unique perspective in requirements, design, development, testing, and production by thinking creatively about edge cases and security implications, though they do not scale as well as tools. Tools are better for automation, high-volume testing, and preventing known issues, but may miss more complex vulnerabilities. An informed approach uses both hackers and tools throughout the SDLC.
2. Why this talk?
• Our goal is to build secure software
• What does an SDLC that considers security throughout look
like?
• Where can you automate security controls in your SDLC?
• What are the implications of building 1 application vs. managing
hundreds?
• Learn to think more like a hacker
3. Whoami
• Geoffrey Vaughan @MrVaughan
• Security Engineer @SecurityInnovation
• Appsec pentesting/advisory at all areas of SDLC
• Former High School/Prison/University Teacher
• Occasionally I’m let out of my basement
• Travelled from Toronto to be here with you today
5. Qualities
Qualities of a Hacker
• Develops creative solutions to
complex problems
• Researches and deeply
understands the problem
• May leverage tools in the
pursuit of a solution
Qualities of a (Security) Tool
• Helps solve problems fast
• Automates the mundane
• Can use signatures,
behaviors, or analytics
• Great for high volume testing
(large problems and large
number of test cases)
6. Securing your SDLC
• At various points in your SDLC,
you may want to use a hacker
and/or a tool to help secure your
product
• Hackers are great at thinking
about problems from a different
perspective
• Great for finding design flaws
• Tools can be very thorough at
finding/preventing defined
known issues
• Great for doing tedious things
7. Security Requirements
Have you thought of everything?
• How do you confidently know from an early stage that you have
thought of every possible thing that could go wrong with your
application?
• It is a lot cheaper && easier && faster to fix security issues in
the Requirements phase than in Production
• Like 30 to 100X less expensive!
• (Depends who you ask)
8. Security Requirements
Have you thought of everything?
Hacker
• Probably will find things the
tools miss
• Will think of some really
interesting edge cases
• Might not think of everything
Tool
• Checklists
• Threat Modeling
• Processes
10. Design/Architecture
Hacker
• Hacker + Developer in a room
with a flow diagram can often
find many issues in a very
short amount of time
• This approach doesn’t scale
well when the application
becomes infinitely large or
when there is a huge list of
applications to test
Tool
• Threat modeling
• There are not a lot of tools out
there that provide meaningful
value in this space
11. Development
Hacker
• Training
• Manual Code Review
• Can find more complex
vulnerabilities
• Doesn’t scale well
• Peer Code reviews
Tool
• In IDE plugins (code assisted
development)
• Static analysis tools
• Limited vulnerability classes
detectable
• Lots of false positives
(thousands)
• Good coverage for large
applications
• Secure Coding Guidelines
12. What can you find with static
analysis?
Good at finding
• Source Sink issues,
tracking where malicious
input is executed (XSS, SQLi,
and URL Redirects)
• Security misconfigurations
• Insecure randomness
• Some session management
issues
• False Positives!!!!
Not good at finding
• Authorization issues
• Some authentication issues
(password resets, password
brute force)
• Abuse of business rules
• Memory corruption issues
(some)
• Design flaws
13. QA/Testing
• Ideally, it’s best to try to find issues as early in the SDLC as
possible
• In QA, finding and fixing issues is more difficult
• More costly, could introduce delays, sometimes under strict time constraints
• Some issues could require redesign or architecture changes
• First chance to do runtime analysis
14. QA/Testing
Hacker
• Can consider the whole
picture of the application
• Limited by time/best effort
• If combined with source code,
can give best perspective into
finding vulnerabilities
• Hard to cover all
pages/parameters
Tool
• Fuzzing high volume of test
cases
• Crawl/test large applications
with good coverage
• Can do Authenticated vs.
Unauthenticated testing
• Crash analysis, runtime
debugging
• Still has trouble with business
rules
15. Production
Hacker
• Can leverage external
resources (Social
Engineering, Social media,
Google)
• Can leverage
weak/vulnerable users
• May invest significant
time/energy
Tool
• Signature based detection
• Heuristic threat intelligence
• Abnormality detection
• Continuous runtime scanning
16. So What About Agile?
Security Tasks:
1. Every Feature/Story Requirements
2. Every Sprint/Release Requirements
3. Regular Maintenance
17. With Every New Feature / User Story:
• Do the feature requirements consider the security implications
of this feature?
• How will this feature affect the overall threat model
18. Every Sprint / New Release
• Ensure overall security requirements continue to apply across
every new sprint (checklist?)
• Impact on application architecture
• Threat modelling for all new features
• Automated code review
• Manual/Peer code review
• Security Testing of new features
19. Regular Maintenance
• Periodic security testing and scanning to ensure no new issues
arise. The result is a snapshot of current your security posture
• Regular security training for all members of the team
• Takes a big picture look at results from all security testing and
look for areas where issues could have been prevented sooner.
20. Secrets to Doing Agile Security Well
• It takes the whole team thinking about security all the time
• Perform regular checks to identify, address issues, and improve
processes
• Systems and processes are necessary to implement security
controls throughout.
21. Hacker vs. Tool?
• An informed hacker will know to use each tool and when to rely
on their hacker mindset/instincts
• Learn to think more like a hacker to…
• Make better tools
• Attack your application as a hacker might
• Learn the trade, not the tool
22. More Talks today:
I’m also presenting 2 other talks today on completely unrelated
subjects:
Catching IMSI Catchers: Hunting the hunter, can you tell if your
phone’s being captured by a rogue cell phone tower/ IMSI
catcher/ Stingray?
Security Best Practices for Regular Users - What's in your
personal threat model? What assets are you trying to protect?
Learn how to improve your personal security and privacy online
through best practices and security tips.
For the sake of this presentation, a hacker is more of an idea or way of thinking then one particular person. “Hacker Mindset”
Here we are going to look at each area of the SDLC and see how it can benefit from tools and hackers.
The earlier you find a vuln in the SDLC the cheaper / better.
Tools can put systems in place forcing developers to care about security early on. Security requirements as function requirements.
I just like asking questions until I find a path that will allow me to exploit the system. Favorite thing to do! --- !AHA story
There are strong arguments to keeping your architecture as simple as possible
Formal vs. informal threat modelling.
When I am training I try to engage people to get them thinking like a hacker. I don’t teach security, I train hackers.
With peer reviews you can create cohorts of review where you have teams reviewing each other, does scale a bit better then one hacker.
Source code reviews, static analysis, and ide plugins are only good at finding certain classes of vulnerabilities
With any tool you use you need to know what it is capable of finding and what it is not capable of finding.
A real threat actor might devotes weeks/months/years to compromising your system
You might invest 2 weeks worth of effort into QA testing of security. External hackers could invest months or multiple people.
Talking about 3 categories of activities that need to be performed at different frequencies throughout the development of an agile application.
Every new feature could introduce new threats, vulnerabilities, or break other resolved issues
Security posture regression is possible. You may think you were secure but you rush too many new features without proper considerations and new vulnerabilities are in production
Doesn’t break auth models, consistent data storage practices, crypto / communication channels.
Pay Down technical debt and improve processes.
Explain Technical Debt
It takes a village / team of people and processes
One for the devs, one for the hackers, one for the users
