This webcast will analyze the key differences between the penetration testing and bug bounty models and explore why one company replaced their pen tests over the last three years.
Time Series Foundation Models - current state and future directions
3 Reasons to Swap Your Next Pen Test With a Bug Bounty Program
1. 3 REASONS TO SWAP YOUR
NEXT PEN TEST WITH A BUG
BOUNTY PROGRAM
2. Jason Haddix, Head of Trust and Security Wade Billings, VP of Technology Services
2
YOUR SPEAKERS
3. AGENDA
• Key differences between bug bounties and penetration testing
• Definitions
• Testers
• Coverage
• Model
• Canvas by Instructure Case Study
• Q&A
3
DOWNLOAD OUR REPORT ‘HEAD TO HEAD:
BUG BOUNTIES VS. PENETRATION TESTING”
https://bugcrowd.com/penetration-testing
4. WHAT IS PENETRATION TESTING?
A penetration is…
• A time-boxed, fixed-cost assessment
• External consultants try to find as many vulnerabilities and config issues as possible and exploiting
those vulnerabilities to determine the risks
A penetration is NOT…
• A red team assessment
4
5. | CONFINDENTIAL INFORMATION
WHAT IS A BUG BOUNTY?
3/14/175
Independent security
researchers from all
over the world are
recruited
Vulnerabilities
are found and
reported
Rewards are
exchanged for reporting
vulnerabilities in
company applications
7. TESTERS:
MANY VS. FEW
Not only is the testing pool much
larger, but it is also more diverse,
providing organizations with a broad
set of skills and expertise.
7
9. MANY WAYS TO USE BUG BOUNTY PROGRAMS
9
Start with invite only
private program to
gain experience
Deliver ongoing security assurance with
continuous private and/or public program
Project or app specific On-Demand
Start with invite only
private program to
gain experience
Expand scope to increase
value & researcher
engagement
12. SECURITY AT CANVAS
• Published security notices
• Extensive security testing
• Open security audits since 2011
• Working with independent researchers
13. RESULTS: SIX YEARS OF PUBLIC SECURITY AUDITS
13
0
10
20
30
40
Average pen test findings 2011 - 2013 Average bug bounty findings 2014 - 2016
Non-critical vulnerabilities High-critical vulnerabilities
SC Mag folks will introduce concept, take care of housekeeping and hand it over
Spend a couple of minutes introducing yourselves and backgrounds
Haddix to quickly outline agenda
Plug = download accompanying report that is attached within the webinar screen or at the url listed
Set framework for what we’re discussing–differentiate between red teaming and penetration testing
Make sure to communicate the fact that we understand there are uses for penetration testing (and outline what those uses are)
Also make sure to communicate that we’re not saying testers themselves are flawed, but the model
Haddix
Brief overview of next slides (acts as a summary/table of contents)
Main message: this is why Canvas chose the crowdsourced model – more testers
Stat: On average, public and private, 138 unique researchers submit on bounty programs
Address: Trust
Follow-up questions:
How can you trust bug hunters? Wade, did you run into any internal questions about this?
Do they have the necessary skills/specialized skill sets?
Main message: continuous testing is the key to the improvement over pen testing
Mention Methodology issue
Follow-up questions:
How did you ensure coverage of bug hunters?
Note: I moved this slide up one for a better flow.
Use this slide to talk about the different uses and talk about how Canvas is utilizing the crowdsourced model
Don’t spend as much time on this slide… use this slide as a segue into the case study
Wade to set framework for the Canvas story
NOTE: Graph shows averages of 3 annual pen tests vs. 3 annual bug bounties
Can also talk about each year individually–8x the first year, dip in results the second year, back up the third year
Talk about QUALITY
Wade to expand on learnings
Wade to talk about the future of bug bounties at Canvas
Haddix to talk about the future of the space…segue into logos slide (adoption)
Haddix to expand upon the adoption, talk about private, talk about expanding testing capabilities
Do you think every organization is ready for a bug bounty program?
Can bug bounties replace all pen testing?
What is the signal vs. noise in bug bounties? Is it worth it?
What is the future of penetration testing?
What top tier companies have switched to this model?
Seems risky, how do you control rogue researchers?