Security process should be integrated with SDLC well to be successful. While many companies have already moved from Waterfall to Agile methodologies security remains behind more often than not. We have demonstrated in our presentation how security can move to agile by utilizing open source tools, customizing them to meet our needs and to implement a continuos security testing using dynamic scanners as well as manual testing.
It’s very important also to assure that false positives are not fed to the developers bug tracking systems and to assign a severity for each finding correctly. To make it happen we import all our findings to a security dashboard and review them before exporting to a bug tracking system.
3. 3
DREAD
• Damage potential: How great is the
damage if the vulnerability is exploited?
• Reproducibility: How easy it is to reproduce
the attack?
• Exploitability: How easy it is to successfully
exploit this condition?
• Affected users: As a rough percentage,
how many users are affected?
• Discoverability: How easy it is to find the
vulnerability?
Making Security Agile
4. 4
STRIDE
• Spoofing
• Tampering
• Repudiation
• Information disclosure
• Denial of service
• Elevation of privileges
Making Security Agile
5. Old AppSec Summary
We’ve generated huge *doc and *xls with
Threat Models and pen test results.
We wrote nice exec summaries and we
presented details to Dev/OPS/DB teams
We had a cool heat map to visualize the
risk allocation for execs.
Then we worked on remediation with
developers – informal name “nagging”
We supported a security dashboard and
reported/escalated to CTO weekly
5Making Security Agile
6. DevOps + Agile = Big Bang for Security
6Making Security Agile
7. What is SAMI
Our core product is an API based
service running in a cloud that supports
IoT devices in real time
They also include web applications built
on top of that.
All of that runs in an agile mode,
because competition is too high and
losing is not an option
More details: www.samsungsami.io
7Making Security Agile
8. We had no choice
8
http://www.octopus.euhttp://brentmailphotography.com
Making Security Agile
9. New Agile Reality aka Reqs
No 2-4 weeks for security purposes
Nobody cares about (or understand)
formalities such as STRIDE, DREAD or
heat maps
Forget about 2 full days of interviews, not
possible
Nobody gives Environment for 2-3 weeks
solely for pen testing. Need to share.
Need to do security as Dev process goes and
in parallel
Need to be integrated better with the DevOps
and QA teams
9Making Security Agile
10. Other Challenges for API
products
You *can* pass an API URL to a scanner
and tell “do automatic scanning” , but it’s
not going to crawl. No Spidering is
available.
You can manually call each and every
API while on a scan proxy and then do
scanning, but if you have too many ever
changing API’s, you’ll die doing that
Sanjay will talk about our approach in
details
10Making Security Agile
11. Aggravating factor – security
testing is not exactly QA
QA test are deterministic, AppSec tests
are anything but
You can’t dump all app scanner’s
“findings” to a bug tracking system
You usually need to triage each finding
and verify a severity before submitting it
to a bug tracking system
In many cases you still need to explain
your finding and remediation to
developers
11Making Security Agile
12. Coming to a Solution
Since Security does require a manual
review, our hypothesis was that we need
a dashboard with a broad import/export
capabilities
Import from scanners, export to bug
tracking systems
All scanners should be automated to run
regression
Should also allow manual testing and
import for new features and products
12Making Security Agile
13. Making Security Agile 13
Scheduled Job
Scheduled Job
Manual testing
Custom Tools
Security Artists
DevOps Team
Notifications
14. Step-by-step instructions to
configure security automation
Demo of Security Automation
Install open source software
Configure scans for Web applications
Configure security testing using QA
regression
14Making Security Agile
16. A. Install ThreadFix (in AWS or
VM)
Download ThreadFix from
http://www.threadfix.org/download/
Follow instructions to start ThreadFix
https://github.com/denimgroup/threadfix/wi
ki/Getting-Started
Enable port 8080 for external access.
Once ThreadFix is up and running, go to
http://xxx.xxx.xxx.xxx:8080/threadfix in a
web browser and accept warning from
browser.
Username: user
Password: password
16Making Security Agile
17. B. Add users in Security team
into ThreadFix
17Making Security Agile
18. C: Add Team in ThreadFix
18Making Security Agile
19. D. Add application Payments
under Accounts team in
Threadfix
19Making Security Agile
22. G. Visit application in ThreadFix
and note application #
Note: Application Id is shown as 31 for Payments app in URL in
snapshot below. We will use this later when we upload findings
in XML format from command line.
22Making Security Agile
23. H. Set Defect tracker for
application in ThreadFix
23Making Security Agile
24. I. Enter login for Defect tracker
24Making Security Agile
25. J. Install ZAP
Download and Install open source ZAP
(Zed Attack Proxy) on AWS/VM using
following
https://github.com/zaproxy/zaproxy/wiki/
Downloads?tm=2
We will be using ZAP on the AWS or VM
in an headless mode (without UI).
Install ZAP on your laptop or Mac using
above link. We will be using the UI to
create sessions for web applications.
25Making Security Agile
27. L. Install web based job
scheduler e.g. Juicebox
Install any Open source Cron job
scheduler on AWS/VM.
Configure
scan
Start scan
27Making Security Agile
28. BScan – burp automation and more
Making Security Agile 28
• Automates burp using Ruby extensions
• Allows adding custom tests to the same scan
• Can be run as simple as:
jruby –S bscan –c config
• Will generate report automatically when completed
• Can work in auto proxy mode
• Configurable – can overwrite any Burp and custom settings
• Version 3.0 is coming, stay tuned !
30. Configure ThreadFix jar
Use following commands in terminal mode
on AWS/VM where ThreadFix is installed
(obtain your API key from Threadfix)
java -jar /home/user/tfcli.jar --set key
5fPO07kc8xyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxy
java -jar /home/user/tfcli.jar --set url
https://127.0.0.1/threadfix/rest
30Making Security Agile
31. Create Shell scripts to scan
each application
Note: Following is Payment.sh script to scan Payment web app. We use
app id # 31 found earlier.
31Making Security Agile
32. Configure JuiceBox to schedule
daily scan for Payment web app
32Making Security Agile
34. Why use QA regression
Over the years, QA team has built
hundreds or even thousands of
functional test scripts. They continue to
add new scripts everyday.
Security team can reuse the work done
by QA team to do security testing.
They can use a proxy such as BURP or
ZAP to analyze requests & responses to
find any security vulnerabilities.
34Making Security Agile
36. Start ZAP Proxy
Use command such as following to start
ZAP in headless mode on port 8085:
java -Xmx2048m -jar
/home/user/zap/ZAP_2.4.2/zap-2.4.2.jar
-cmd -daemon -installdir
/home/user/zap/ZAP_2.4.2 -config
api.key=12345 -port 8085
Making Security Agile 36
37. Configure and start QA
regression
Configure QA regression properties to
use proxy running on port 8085.
Use command such as following to start
QA regression (will vary depending on
scripts used for your QA regression)
mvn clean test -Dcategory=api -
Dtarget.environment=qa -DexcludedGroups=in-
progress,sdr,email,logs,cleanup -
DuseProxy=true -DlocalProxy=true
37Making Security Agile
38. Save security findings from
regression in XML form
At the end of regression, save findings in XML form:
curl
"http://localhost:8085/OTHER/core/other/xmlre
port/?apikey=12345" >
/home/user/temp/newpaymentregression.xml
Upload the findings to Threadfix using (app id for
regression # 32 found earlier)
java -jar /home/user/tfcli.jar --upload 32
/home/user/temp/newpaymentregression.xml
Note: Above commands are part of automated script.
38Making Security Agile
45. Security Transformation Cheat Sheet
45
From TO
Two full days TM meetings with many
people involved
Participating in arch/design
discussions through Confluence or
other collab tools
Very formal, “scientific” TM with
DREAD, STRIDE residual risks, etc.
Recommendations in JIRA coming
from arch/design discussions
DREAD, residual risks, etc. HML (High, Medium, Low)
STRIDE CVE (usually automatically assigned
by a tool)
Manual pen testing for everything Selective pen testing for new features
and products
Expensive commercial scanners To OWASP ZAP, Burp, custom tools
Docs and PDF reports, manual export
to Dev bug tracking systems
Exporting tickets directly to JIRA from
Threadfix
Running/uploading scans manually Automated batch jobs w auto-upload
to Threadfix
Making Security Agile
46. Conclusion
You cannot do security as you did it 5-10
years ago. Security must work in parallel w
Dev.
Automate security testing by using
scheduled jobs, auto-uploading “findings”
to a dashboard
Use Security professionals to review
findings
Make sure that findings are triaged before
they are submitted to a bug tracking
system
Don’t feed false positives to Dev or
you’ll completely lose your credibility
46Making Security Agile
We’ve just returned from an AppSecUSA conference where quite a few folks were talking about security automation tools, which is cool of course. We love tools, but it’s not about tools only, it’s about how well your security process is integrated to SDLC.
Going back 5 to 7 years our SDLC model was still very much Waterfall and our security process was adjusted well to it:
We had 2 full days for onsite interviews
Business owners, s/w and DB architects, Dev, OPS, QA were available to sit with us for those two days and talk about security and threats
We were buttoned up and very formal
We’ve created our own scoring system based on DREAD
We created Threat Models based on interviews, We categorized findings based on STRIDE, After TM was completed we would start our pen testing. We had 2-4 weeks to complete it!
We used heavy commercial tools to do static and dynamic analysis
I won’t talk much about DREAD, it’s very much dead and it’s dead because of a good reason. You were required to score each dread category, and it was a pure guestimate anyway. It looks scientific because it has categories, but is essence it’s worth than just assigning a score in simple HML terms, because HML is very intuitive and can be easily understood by everyone.
STRIDE – again, looks scientific and cool, but that categorization is very abstract and it’s difficult to practically apply it, because some vulnerabilities will fall into several categories, while for others will be difficult to find a match for. When you use CVE, it’s actually something that has a match in real world, easy to understand and most importantly it will be automatically created by many scanners and understood by dashboard such as Threadfix.
It was very much like a waterfall model, where we had a “security gateway” clearly defined. There was no way to by-pass
It was clearly “us” and “them” model, meaning that our involvement to Dev matters or Dev involvement to our matters was limited.
It required a significant time and money to support all that
There was an essential lag between a sec assessment and a release date, which could result in big differences between production and what we’ve tested and reviewed.
Fixing was not easy, “nagging” didn’t always work
And then Big Bang has happened. It happened because everybody has moved to Agile SDLC and new projects started flying over our head with a huge velocity. We’ve realized at this point that we could not do all those cool things that we did in the waterfall model.
It’s not only about agility and speed. There were other issues that were coming form the products that we need to support. It’s all about API’s today. Web applications become just a thin
wrapper built on the top of API’s.
Let us take a quick look at the product that we support.
We had no choice. We needed to move from a waterfall to agile quickly. We’ve looked around at what is available. We’ve realized quickly that some new buzz words such as “security as code” or “security as service”, do not mean much and do not allow creating a comprehensive E2E security process alone.
It usually goes like this:
We write code and build services and they will solve ALL our security problems.
No we won’t and I don’t even want to go to this discussion until
I see a comprehensive security scanner that doesn’t produce “false positives” and gets all severities right.
Or after someone figures out how to design apps with all necessary sec controls in place from day one.
“Us” and “them” model didn’t work anymore. We need to move with the team at their pace.
This is how we’d come to our solution. This is our reasoning.
Explain diagram.
Our security team is not big, it’s two permanent people. Security artists as we modestly call ourselves, 1-2 consultants and people in other teams who support our projects.
Our QA team is probably 4 times bigger and Sanjay will talk today about automating the process and utilizing QA unit tests, thousand of them to implement security.
What remained unchanged here is a necessity to review the results and start working with developers on remediation.
What we try to avoid are tools that generate a lot of false positives, because it creates road block for automation
I took it from a Lean Startup conference and I think it’s very appropriate for our sec transformation process.
Sec as code is overstatement. You can’t solve all your security problems just by writing code. You need to do more than that.
That sec automation job does include writing code, creating sec services and an agile security process, but don’t forget about “security artists” who can make sense of tons of data generated by scanners and other tools, because security is still very much like politics, which is an art of possibilities
I’m still missing the old process a bit and ask myself often: did we through out any babies with bath waters during this transformation? The answer is probably “yes”, but this is the nature of any agile approach, it usually bears bigger risks, not only in security domain and we’ll have to live with it.
Security artists are people who can make sense of ravings generated by security tools.