This document discusses the importance of continuous security testing throughout the development process. It recommends adopting a security test pyramid approach similar to testing principles like test-driven development. Responsibility for security testing should be shared between developers, testers, and external communities. The document reviews some recent security breaches like Cross-Site Scripting attacks on eBay and data leaks from Cloudbleed, noting vulnerabilities that allowed these threats were not caught earlier. Various tools for security testing like dependency checks, static analysis, and OWASP ZAP are presented, along with caveats to keep in mind regarding tools. The key takeaways are to incorporate security testing into all phases of development and think like an attacker when testing.
2. Agenda
❏ The current state of security testing
❏ The security test pyramid
❏ Sharing responsibility
❏ Some real life security breaches
❏ Some tools and approaches
4. The current state
We tend to concentrate on security during two phases of a
software project
The beginning and the end
We “talk” about security in the beginning.
At the end the common practice is to do a pen test
5. The ideal state
Can we have pen testing done continuously?
How do we incorporate security testing throughout all phases?
6. So, how do we get there?
The test pyramid, TDD etc. are principles/approaches that have
been widely adopted
Could we think along the same lines for security as well?
9. Some security breaches from the recent
past
❏ What were some of the security breaches we have had in
the recent past?
❏ What was its impact?
❏ Could it have been caught earlier? If so, how?
12. The Attack
There was a vulnerability that allowed attackers boobytrap links
on pages
A user, on clicking these, were taken to fake login page,
endangering their user credentials
13. The scary facts about the eBay
vulnerability
❏ The vulnerability (called XSS) first seen in 1990s
❏ The sensitivity of the information stolen
❏ Who discovered the vulnerability
❏ The time frame between introduction to discovery of the
threat
15. What was Cloubleed?
Reverse proxies of Cloudflare leaked HTTP cookies, auth
tokens, HTTP POST bodies etc.
Data from one customer could go to any other customer
Some data was also cached in search engines!
Impact: 2 million websites, like Uber, OkCupid, Fitbit and even a
password manager!
16. What happened?
How? The equality check to see if the end of buffer has been
reached led to a buffer overrun
Mitigation? A ‘>=’ in the equality check instead of ‘==’
Similar to the Heartbleed vulnerability found in 2014
18. Chaining of multiple vulnerabilities
World’s largest dedicated security technology company
SQL-i hall of shame in two consecutive years: 2016 and 2017
VirusScan Enterprise for Linux
Ten vulnerabilities were found, of which four chained together
exposed a threat.
19. The deadly combination..
# An information disclosure flaw allowed remote file reading
# The next flaw allowed creation malicious scripts without the
need for proper authentication or cryptographic controls
# And the final vulnerability, which allowed escalation of
privileges and remote execution of code.
When chained together, these vulnerabilities allow a remote
attacker to execute code with root access control.
20. It was a series of vulnerabilities. Not just one.
The biggest takeaway from this attack?
21. Why were these threats not caught?
● Lack of awareness
● It just wasn’t caught!
24. Static Code Analysis
Does your code conform to industry standards?
Basic security checks can also be done
Example :
- Taint Analysis
- Buffer overflows
25. Vulnerable components
Dependencies may have known vulnerabilities
National Vulnerability Database (NVD) with known vulnerabilities
OWASP dependency checker
26. OWASP ZAP
Free application/api scanner to help find vulnerabilities
Meant to be used by beginners and professionals
Reference 1
Reference 2
27. Caveats to using tools
Tools are meant to be a guide/assistance
A green pipeline doesn't necessarily mean you're 100% "secure"
Tools require maintenance
28. How do we test a feature from a
security standpoint?
29. Upload a File
How would you test in from a functional point of view?
● Happy path?
● No file?
30. Upload a File
What could you test from a security point of view?
● .EXE upload?
● .CSV upload with malicious macros?
31. Login/Logout?
1. Is the password safe?
2. When login fails?
3. Can a user hit back from the login page?
32. Search feature?
1. Can I inject a script into it?
2. Is there any way I could use the search engine to see data
I'm not supposed to see?
34. Key takeaways
Regardless of what you're building, you have to be secure
Focus on security not only at the beginning and end, but
throughout
Think test pyramid
While testing, think like a malicious attacker would.
Use tools to help you, but don't solely rely on them
This talk will cover why security testing should be a continuous process and how to go about it.
We will also talk about how it’s a shared responsibility of the team.
The number of security attacks and it’s impact has increased exponentially over the recent years.
Everything is getting digitized and the all kinds of data are out there that can be misused.
Being a part software delivery process, it is every role’s responsibility to ensure all quality checks, including security is met. Security is a herculean task, it can not just be owned by one role in the team, it’s everyone’s responsibility.
Usually what we see happen is that towards a release, a penetration test will be done by a third party. They’ll generate a report of potential threats. However, finding security threats so close to a release often forces a team to prioritize the threats and then handle them. Some may get resolved. Some might not.
And worse, some might not even be caught.
If you rewind back to a few year’s ago, you’ll see that we had a similar problem with functional testing as well.
Functional bugs were caught towards the end, which was too late in the feedback cycle.
Sounds familiar?
The security test pyramid is no different from a regular test pyramid.
The key thing here is to remember to include a security perspective to your testing.
Push tests as far down as you can
Just like how quality is a shared responsibility across roles, so is security.
Team - Just like quality from a functional view is a shared responsibility
External community - Documents, standards, practices, tools
Ebay is an e-commerce website that facilitates online consumer-to-consumer and business-to-consumer sales
A common use case of the site would involve a user browsing through items on sale, like say perhaps a phone or a book etc.
There was a vulnerability that allowed attackers boobytrap links on pages
A user, on clicking these, were taken to fake login page, endangering their user credentials
The vulnerability (called XSS) was first seen in the 1990s. Which means that this particular type of attack is known, there are widely known solutions to the vulnerability, and it could have been prevented
The sensitivity of the information stolen - User credentials were stolen without the user even being aware of the fact. They would merely enter their username and password into the fake login page, and not know their accounts were being compromised
Who discovered the vulnerability. The vulnerability was not in fact discovered by Ebay, but by someone else who then later reported it to Ebay.
To make matters even worse, this vulnerability went underdiscovered for three months.
So if you look at all this together, it means that Ebay had a known vulnerability for three months, which would compromise the accounts of all their users, and Ebay themselves were not even aware of the problem until someone told them.
Scary right?
So this particular example only affected one set of users. Let's look at an example where multiple user bases were hit.
Cloudflare is a company that provides a content delivery network, Internet security services and distributed domain name server services.
Heartbleed vulnerability was found in the OpenSSL cryptography library which was also caused by a buffer overrun.
If you look at all three of these examples, or most other attacks that have occurred, all were because of known vulnerabilities. So why were they not caught?
Security usually comes as an afterthought. Between delivery commitments and deadlines, a lot of things tend to get missed.
Why did we choose these set of attacks?
Not just credit cards
Everyone makes mistakes
There are some tools out there that can help you a lot. These tools are very useful, they will help bring to light a lot of common vulnerabilties
Unit level -
Static Code Checkers - They basically check simple “rules” that might indicate possible security vulnerabilities
Examples :
BOON applies integer range analysis to determine whether a C program can index an array outside its bounds[4].
Let’s look at some examples. Maybe that will help.
We'll take some common features that you'll see in your day to day projects that will help you get an idea of how to think of a feature's testing from a security standpoint.
And what kind of questions you can ask?
Now let's look at some other examples
Keep the password obscure - Is it visible on the page? Will it be accessible anywhere? Logs? Database?
Are you giving the user too much information? Invalid username/password?
Password reset page?
Does the back button from a login page take you back to
How are you keeping a user logged in? How long is he logged in idle?