Geoffrey Vaughan, Security Engineer at Security Innovation, discusses the pro's and con's of using a hacker vs. a scanning tool for testing applications.

1. Hacker vs. Tools
Geoffrey Vaughan
Security Engineer
@mrvaughan

2. Why this talk?
• Our goal is to build secure software
• What does an SDLC that considers security throughout look
like?
• Where can you automate security controls in your SDLC?
• What are the implications of building 1 application vs. managing
hundreds?
• Learn to think more like a hacker

3. Whoami
• Geoffrey Vaughan @MrVaughan
• Security Engineer @SecurityInnovation
• Appsec pentesting/advisory at all areas of SDLC
• Former High School/Prison/University Teacher
• Occasionally I’m let out of my basement
• Travelled from Toronto to be here with you today

4. Disclaimer
• Vendor/tool agnostic
• I provide services in all areas of SDLC
• Hacker Biased (I am one)

5. Qualities
Qualities of a Hacker
• Develops creative solutions to
complex problems
• Researches and deeply
understands the problem
• May leverage tools in the
pursuit of a solution
Qualities of a (Security) Tool
• Helps solve problems fast
• Automates the mundane
• Can use signatures,
behaviors, or analytics
• Great for high volume testing
(large problems and large
number of test cases)

6. Securing your SDLC
• At various points in your SDLC,
you may want to use a hacker
and/or a tool to help secure your
product
• Hackers are great at thinking
about problems from a different
perspective
• Great for finding design flaws
• Tools can be very thorough at
finding/preventing defined
known issues
• Great for doing tedious things

7. Security Requirements
Have you thought of everything?
• How do you confidently know from an early stage that you have
thought of every possible thing that could go wrong with your
application?
• It is a lot cheaper && easier && faster to fix security issues in
the Requirements phase than in Production
• Like 30 to 100X less expensive!
• (Depends who you ask)

8. Security Requirements
Have you thought of everything?
Hacker
• Probably will find things the
tools miss
• Will think of some really
interesting edge cases
• Might not think of everything
Tool
• Checklists
• Threat Modeling
• Processes

9. Design/Architecture
Most architecture designs consist of:
• Use cases
• User stories
• Data Flow Diagrams
• Server/Stack layouts

10. Design/Architecture
Hacker
• Hacker + Developer in a room
with a flow diagram can often
find many issues in a very
short amount of time
• This approach doesn’t scale
well when the application
becomes infinitely large or
when there is a huge list of
applications to test
Tool
• Threat modeling
• There are not a lot of tools out
there that provide meaningful
value in this space

11. Development
Hacker
• Training
• Manual Code Review
• Can find more complex
vulnerabilities
• Doesn’t scale well
• Peer Code reviews
Tool
• In IDE plugins (code assisted
development)
• Static analysis tools
• Limited vulnerability classes
detectable
• Lots of false positives
(thousands)
• Good coverage for large
applications
• Secure Coding Guidelines

12. What can you find with static
analysis?
Good at finding
• Source  Sink issues,
tracking where malicious
input is executed (XSS, SQLi,
and URL Redirects)
• Security misconfigurations
• Insecure randomness
• Some session management
issues
• False Positives!!!!
Not good at finding
• Authorization issues
• Some authentication issues
(password resets, password
brute force)
• Abuse of business rules
• Memory corruption issues
(some)
• Design flaws

13. QA/Testing
• Ideally, it’s best to try to find issues as early in the SDLC as
possible
• In QA, finding and fixing issues is more difficult
• More costly, could introduce delays, sometimes under strict time constraints
• Some issues could require redesign or architecture changes
• First chance to do runtime analysis

14. QA/Testing
Hacker
• Can consider the whole
picture of the application
• Limited by time/best effort
• If combined with source code,
can give best perspective into
finding vulnerabilities
• Hard to cover all
pages/parameters
Tool
• Fuzzing high volume of test
cases
• Crawl/test large applications
with good coverage
• Can do Authenticated vs.
Unauthenticated testing
• Crash analysis, runtime
debugging
• Still has trouble with business
rules

15. Production
Hacker
• Can leverage external
resources (Social
Engineering, Social media,
Google)
• Can leverage
weak/vulnerable users
• May invest significant
time/energy
Tool
• Signature based detection
• Heuristic threat intelligence
• Abnormality detection
• Continuous runtime scanning

16. So What About Agile?
Security Tasks:
1. Every Feature/Story Requirements
2. Every Sprint/Release Requirements
3. Regular Maintenance

17. With Every New Feature / User Story:
• Do the feature requirements consider the security implications
of this feature?
• How will this feature affect the overall threat model

18. Every Sprint / New Release
• Ensure overall security requirements continue to apply across
every new sprint (checklist?)
• Impact on application architecture
• Threat modelling for all new features
• Automated code review
• Manual/Peer code review
• Security Testing of new features

19. Regular Maintenance
• Periodic security testing and scanning to ensure no new issues
arise. The result is a snapshot of current your security posture
• Regular security training for all members of the team
• Takes a big picture look at results from all security testing and
look for areas where issues could have been prevented sooner.

20. Secrets to Doing Agile Security Well
• It takes the whole team thinking about security all the time
• Perform regular checks to identify, address issues, and improve
processes
• Systems and processes are necessary to implement security
controls throughout.

21. Hacker vs. Tool?
• An informed hacker will know to use each tool and when to rely
on their hacker mindset/instincts
• Learn to think more like a hacker to…
• Make better tools
• Attack your application as a hacker might
• Learn the trade, not the tool

22. More Talks today:
I’m also presenting 2 other talks today on completely unrelated
subjects:
Catching IMSI Catchers: Hunting the hunter, can you tell if your
phone’s being captured by a rogue cell phone tower/ IMSI
catcher/ Stingray?
Security Best Practices for Regular Users - What's in your
personal threat model? What assets are you trying to protect?
Learn how to improve your personal security and privacy online
through best practices and security tips.

23. Thank you
Geoffrey Vaughan
@mrvaughan
@SecurityInnovation