3. 2. SOFTWARE IS LEAVING SECURITY IN THE
DUST
SOFTWARE
SECURITY
2000 2010 2020
SASTDAST
WAF
•Typical
enterprise has
hundreds or
thousands of
applications
•Applications are
by far the
leading cause of
breaches
(Verizon DBIR)
4. 3. SOFTWARE SUPPLY CHAIN SECURITY IS
TOTALLY BROKEN
Jan Feb Mar Apr May Jun Jul Aug Sept Oct
March 7
CVE-2017-5638
Disclosed, Apache
releases fixed version
March 8
We observed
widespread
attack probes
Mid-May
Equifax
breach
occurs
July 29
Equifax
learns of
breach
Sept 7
Equifax discloses,
Four more Struts2
CVEs disclosed
Equifax ignores
Protected
DisasterLivin’ la vida loca
Prepared
Equifax unaware
5. DIAGNOSIS: GOALS UNCLEAR, TIME WASTED
What we are delivering: What we must deliver:
Right defenses in place
Defenses are effective
Attacks detected/blocked
“I ran a scanner”
Application/API portfolioApplication/API portfolio
7. SO WHAT IS DEVOPS?
https://itrevolution.com/the-three-ways-principles-underpinning-devops/
The “Three Ways”
1. Establish work flow
2. Ensure instant feedback
3. Culture of experimentation
9. QUESTION: CAN DEVOPS HELP SECURITY?
• Problem: software is poor
quality, late, slow, and doesn’t
provide business value.
• Approach: DevOps
• Outcomes:
• 5x lower change failure rate
• 96x faster MTTR service
• 2x likely to exceed bus. goal
• Problem: security is poor quality,
late, slow, and doesn’t provide
business value.
• Possible Approach: DevOps
• Required Outcomes:
• 10x increase in portfolio coverage?
• 80% reduction in vulns to prod?
• 0x increase in time to market?
11. The
“Three Ways”
of Security*
1. Establish security work flow
• Build a concrete security story over time
• Enable development to build security
• Rip, mix, and burn security work
2. Ensure instant security feedback
• Enable self-inventory
• Get real application threat intelligence
• Create security notification infrastructure
3. Build a security culture
• Migrate to “positive” security
• Accelerate evolution of your security story
• Promote “security in sunshine”
* Shamelessly adapted from The Phoenix Project, by Gene Kim
13. Business
Security
Projects
Building defenses, compliance,
reporting, etc…
1
Internal
Security
Work
Threat modeling, security
architecture, security research,
vulnerability assessment, tools
2
Operational
Security
Jobs
Remediation, updates,
analytics, alerts, tickets,
etc…
3
Unplanned
Security
Tasks
Security “firefighting,”
response, recovery, public
relations, etc…
4
UMM…. WHAT IS SECURITY “WORK”?
14. * Shamelessly lifted from the Rugged Software Project
Your security story maps
threat model ➡️
defense strategy ➡️
defenses ➡️ assurance
Making security concrete:
• Enables communication
• Aligns your team
• Expose gaps and priorities
• Creates line-of-sight
FIRST WAY –
BUILD A CONCRETE
SECURITY STORY OVER
TIME
15. Leverage existing DevOps
processes and tools
Refactor
monolithic security
tasks into small
batch sizes.
Deliver
security one
little piece at
a time
FIRST WAY –
ENABLE DEVELOPMENT TO BUILD SECURITY
16. FIRST WAY –
WORK ON BIGGEST THREATS, ONE AT A TIME
Add a single risk to
threat model
• Create JIRA ticket:
Prevent XXE
Create defense
strategy
• Update JIRA Ticket
• Standardize parser
config
• Log & block attacks
Implement defense
• XML library
• Update training
Establish continuous
assessment
• Research typical
failures
• Build custom test
cases
• Enable IAST XXE rule
Establish attack
protection
• Enable RASP XXE rule
Monitor DEV and OPS
• Vulns go to JIRA with
Slack alert
• Attacks go to Splunk
and VictorOps
Do you really need security experts for all these tasks?
XXE
Updated
Security
Story
18. SECOND WAY –
ENABLE SELF-INVENTORY
•You need to know
the exact version of
every app, api, and
library running on
every server in
every environments
•Not hard to fully
automate self-
inventory
DEV
Internal
APIs ContainersPrivate
Public Cloud
OPS
Automatic Application
Inventory
19. SECOND WAY –
GET REAL APPLICATION THREAT INTELLIGENCE
Establish the
infrastructure to…
• Know who is
attacking you
• Know what
techniques they’re
using
• Know what they’re
targeting
• … and protect within
hours
Equifax Attack
20. SECOND WAY –
ESTABLISH A REALTIME APPSEC CONTROL
PLANE
PRODDEV TEST
APIs ContainersPrivate
Public Cloud
APIs ContainersPrivate
Public Cloud
APIs
21. The Third
Security Way
Build Security Culture
A culture that constantly
advances security with the
threat through
experimentation and learning
22. THIRD WAY –
MIGRATE TO “POSITIVE” SECURITY
Testing for all the ways you
might introduce XSS
Testing to verify
your XSS defense
Measure positive security
directly from your running
application
23. THIRD WAY –
ACCELERATE THE EVOLUTION OF YOUR
SECURITY STORY
Celebrate new big
risks without
recrimination
Focus on strength
and simplicity
The faster you
cycle, the faster
you get secure
24. THIRD WAY –
PROMOTE SECURITY IN SUNSHINE
AppSec
Visibility
Cycle
Audit
Developers
Infosec
Legal
Architects
Users
Research
Business
Monitor
Threat
Create
Security
Story
Define
Security
Defenses
Implement
Security
Defenses
Share
Intelligence
Understand
Laws
Verify
Compliance
Understand
Stakeholders
We
Trust
We
Blame
We
Hide
27. The first rule of security is…
…You do not talk about security
HIDE
28. The
“Three Ways”
of Security*
* Shamelessly adapted from The Phoenix Project, by Gene Kim
1. Establish security work flow
• Build a concrete security story over time
• Enable development to build security
• Rip, mix, and burn security work
2. Ensure instant security feedback
• Enable self-inventory
• Get real application threat intelligence
• Create security notification infrastructure
3. Build security culture
• Migrate to “positive” security
• Accelerate evolution of your security story
• Promote “security in sunshine”
29. CLOSING THOUGHTS – TURNING SECURITY
INTO CODE
•Don’t focus on how
to build software
securely…
•Make software
security into
something you
build!
Hi – My name is Jeff.
For over 25 YEARS I’ve been trying to figure out software security.
I’ve tried Orange Book, firewalls, formal modeling, penetration testing, maturity models,
SDLC, policy, training, threat modeling, architecture, etc…
And none of it has worked.
And it ain’t gonna work unless we do some serious rethinking.
Here’s how I know.
Here's what we have..
Typical application totals over 2 million lines of code. 21% custom code, The rest is libraries
But THIS MIGHT SURPRISE – ONLY 8.5% active library code. The other 70.5% totally unused libraries (DEAD)
Library vulnerabilities are all in the news recently because of Equifax…. But there are usually only 1 or 2 vulnerable libraries in an app.
And they are often in the 70.5% that isn’t used.
But look at this…
We are averaging a terrifying 26.7 vulns per app. THINK ABOUT THAT!
If that was 1 or 2 per app it would be A DISASTER
The vast majority of these are well understood problems. We lack the will to eliminate them.
Reason #2 – SAST/DAST/WAF were designed for a different type of software and different process.
* These are tools for EXPERTS to tailor, train, and triage.
Since then, Software has exploded with frameworks, cloud, containers, services/APIs... and most of all DevOps.
Who here is building Single Page Apps with Angular or React in the browser, and APIs on the server?
These tools are not going to help you. Dynamic Scanners can’t understand the complex payloads in REST requests with serialized objects, XML, JSON, etc… And Code Scanners can’t follow the complex frameworks used to handle those payloads.
We are seeing massive adoption of DevOps across a wide range of companies.
Every company is SOMEWHERE on their DEVOPS “journey”
Equifax was breached because they weren’t prepared to respond a vulnerability was disclosed in one of their open source libraries.
Disclosure March 7.
Attacks started a few hours after disclosure
Equifax took months to respond --- but they only had hours.
This is a totally expected understood problem.
The typical organization has tens of thousands of libraries
and there are hundreds of new CVEs every week.
You must have the infrastructure in place to respond within hours.
Traditional AppSec wastes a lot of time and effort -- NO CLEAR GOAL!!
Testing for SQL injection on apps without SQL database
Making dumb decisions about how to prioritize
Only looking at the “critical” applications
Only seeing what tools are good at – NOT WHAT’S IMPORTANT
STORY: The Bookshelf of PDF reports
What we need to deliver is quite different!!!
I want to be crystal clear about what we need to deliver, or else DevOps can't help us.
This represents assurance!
Which brings us to DEV SEC OPS – the PuppyMonkeyBaby of IT.
This is a different talk. I strongly encourage you to read The Phoenix Project and really think about the “three ways” of DevOps.
Really consider what it takes to get work moving, to make sure defects don’t snowball, and you keep innovating.
I can tell you that almost every company I talk to is investing heavily in their DevOps initiative.
All are at some point on their journey.
And they all believe it is the future of their Digital Transformation.
I’m not going to try to defend this picture as a representation of DEVOPS.
But in my head, this is what it’s like.
I mean…this is not what my brain is like.
Well, actually….. We’re off track.
What I like about this is that are small batch sizes, lots of feedback, and no bottlenecks. Work is FLOWING.
* The metrics are proving that this works for software…
So, the question is….
There are striking similarities between the problems with Software Development that DEVOPS is solving and the problems SECURITY continues to have.
So the question people are investigating is -- CAN DEVOPS help SECURITY?
My thought is MAYBE... Just MAYBE.
But it's going to have to deliver radically better results
These are NOT A JOKE
But what would it look like?
I'm pretty clear on a few things that DONT WORK.
But it does a decent job of showing small batch sizes, continuous flow, tight feedback, etc…
BUT WHEN you shove giant legacy AppSec practices into DEVELOPMENT, it breaks. Big BATCH sizes, lots of BOTTLENECKS.
1. DevSecOps is NOT JUST automating the scan button
Doesn’t work because the hard part of scanning is triaging all the findings to see if they are real vulnerabilities.
You have to get experts completely out of the process – no setup, no triage
2. DevSecOps is NOT “PUSHING LEFT”
Yes left is cheapest, but legacy expert approach doesn’t work – no experts!
AND -- really you want to push everywhere – check early and verify at each stage
WE HAVE TO CHANGE THE WORK!!!
So instead of trying to shove legacy security into devops…
What if we “translate” the Three Ways for security work.
In "The Phoenix Project" Gene details four different types of IT WORK.
It's a useful framework to think about SECURITY WORK.
The most important type of security work is what the business cares about. Like building security defenses and compliance.
Then there's INTERNAL SECURITY WORK. Like security reviews, threat modeling, security research, etc...
Don't forget about the OPERATIONAL side of AppSec... Our visibility into who is attacking, what techniques they are using, targets is VERY WEAK.
Finally, there's always the UNPLANNED work -- incidents, zero-days, etc...
We have to get ALL of this work FLOWING
Most teams have security requirements, architecture, test cases, tools, threat models, etc… .ALL DISCONNECTED.
Actually, they’re all different views of the same thing.
At Contrast, we created a security story that captures:
The key threats to our business (and customers)
Our defense strategy for each threat
Specific defenses that we selected
Assurance that it’s all tested
By making the security story CONCRETE, it’s a living deliverable that’s always up to date.
If someone asks, how are we protected against SSRF, XXE, or Clickjacking…
Either you have an answer and proof, or you don’t
This is how you align your whole team on the same goals.
Remember all those types of work? Well we can refactor ALL of them into little pieces that mostly can be done by development teams.
You don’t have to do a HUGE security architecture for EVERY kind of risk all at once.
You don’t have to THREAT MODEL everything all at once.
You don’t have to TEST everything
1. break that work up and deliver the same (or better) results in little pieces.
2. repackage and reframe that work SO THAT DEVELOPERS CAN DO IT.
Without needing security experts. That’s how you SCALE.
Here’s how we achieve that at Contrast.
INSTEAD of one gigantic security test at the END…
We focus on our single biggest risk at a time.
We treat these risks like most other work – we evaluate it and spec it out in JIRA.
We implement it and create test cases
Every risk we add improves us. We’re adding to our Continuous AppSec!
Here’s an XXE example that basically we rely on STANDARD PARSER CONFIG. There’s nothing in this flow that requires security expertise.
But most organizations have to rely on security experts to deal with XXE.
How do we make sure security stays on track as we go?
Your testing is too slow, too inaccurate, and too wasteful.
The first thing you have to do if you want GREAT SECURITY FEEDBACK is to get your arms around
EXACTLY what code you have and where it’s running.
That’s developers machines, test machines, CI/CD machines, QA servers, and OF COURSE production. Data center, VM, Container, Cloud, etc…
This is a simple thing to do – make sure your applications report themselves somewhere.
Self-report all the libraries in use and exactly what version
This is MANDATORY if you want to respond in hours not months.
There’s no better threat intelligence than what you collect from your own applications
But almost every organization is totally blind to application attacks. A WAF is not a good sensor.
Did you know that there was a huge uptick in Padding Oracle attacks last month?
Many organizations focus entirely on code hygiene.
I spent 15 years focused entirely on getting the code RIGHT, so that it couldn’t be attacked.
I viewed runtime protection as a flawed ineffective approach.
But the problem is that CODE HYGIENE is a flawed approach too. We’ve been trying for 20 years and the OWASP T10 hasn’t changed.
Here’s a typical organization – might be hundreds or thousands of apps or apis across DEV, TEST, PROD
First, you enable your applications to Self-inventory, Self-assess, Self-protect
They feed real time security telemetry from DEV, TEST, and PROD into the control plane – so you are always up to date about
The second part is
Everything works in parallel across the entire portfolio, without changing development processes, without slowing innovation.
But JEFF – this isn’t an “SSDLC” – RIGHT. Security has no business telling development how to build their code. This is the security infrastructure you need to build it correctly however they work. Feed your security story into this and you are on your way.
Once you’ve got your inventory…
Moving to “Positive” security is powerful.
Let’s say that you’ve decided to use a particular escaping method for XSS.
Or even better, you use APIs with UI generation in the browser. And you’ve chosen safe ways to update the DOM.
It’s far easier to verify this approach than it is to verify ALL the possible ways this can go wrong. AND it’s WAY easier to AUTOMATE.
I’m not suggesting that negative testing is wrong or bad, BUT it’s ICING – you can’t do it on the whole cake.
I think security is actually just the byproduct of constant challenges by “breakers” and adaptation by “builders.”
This is how you build security OVER TIME. You don’t start from zero with every test.
Make it okay -- not a culture of blame, fear, and secrecy. SECURITY IN SUNSHINE.
At Contrast, we encourage everyone to challenge our security model.
They can challenge the threat model, defense strategy, or our implementation.
And if you want to get rugged faster…. INCREASE CYCLE SPEED!
Have you noticed that people inherently trust software?
They trust open source code written by god knows who with their entire business
They trust shit they downloaded off the internet because they used it before and everything was fine!
So when something goes wrong, people are outraged!!!
How could this happen. Someone must be fired.
Blame is like acid rain on security culture
And it leads to HIDING… People don’t disclose breaches (UBER)
We are ashamed of vulnerabilities
And the things we hide never get fixed! We are absolutely preventing progress!
A DevOps principle is ”if something is painful, do it more often” – so Publish your vulnerabilities! Disclose breaches! Build a healthy kind of trust.
Problem #1 – We trust
There is something about software that people are willing to trust until it is proven to be insecure. Even in the face of massive evidence that new technologies are very likely to have flaws, we instinctively move to new technologies.
In fact, the Internet is the most trusted software environment of all time. Finances, healthcare, privacy, government, defense, energy…. Even if you don’t want to, you are trusting your future to billions of lines of code that you know absolutely nothing about.
Which brings us to problem #2…
Problem #2: We blame
Have you noticed that developers are not exactly enamored of security people? It’s not that developers don’t want to write secure code. And it’s not even that they don’t want help.
No, it’s because many “security” people aren’t working towards improving security. It’s the focus on finding holes and parading them around that undermines our credibility. The idea that continually poking holes in things is likely to lead to the creation of secure things doesn’t pass the laugh test.
Right when we need to combine our skills with those of developers to create secure systems, some are calling to make developers liable for security mistakes. This is probably the most most divisive thing we could possibly do.
This blame is "acid rain" for security ecosystems. Which brings us to….
Problem #3: We hide security
The natural reaction of developers who are getting blamed for security problems is to hide details. They naturally do not want to participate in the security process or share the information that is needed to make security decisions.
As I mentioned there are virtually no applications in existence that provide a reasonable assurance case. That lack of information is a serious market failure that prevents buyers from using security as a buying criteria… and paradoxically makes it silly for vendors to sell secure code.
These are just EXAMPLES!
Don’t forget the goal….
Get security work flowing -- it’s way past time for security to actually deliver tangible assurance
Create tight feedback loops – security should be instant… no more RISK REPOSITORIES
Build security culture – kill blame and hiding!
It’s time we stopped fooling around.
Get organized – define security, build it, and deliver it like anything else.
Throw away your Security Maturity Model – it’s not helping because it’s measuring the wrong thing. Don’t measure your security processes against your peer’s security processes. They’re a different beast.
Measure the OUTPUT – MEASURE THE SECURITY YOU ACTUALLY PRODUCE.
Hopefully this was useful and gave you some ideas about how security has to change.
I would love to take your questions.