While vulnerability assessment tools can identify unpatched or misconfigured code bases, these tools overlook a large portion of an organization's attack surface: known vulnerabilities in applications that are built in-house.

1. PCI & Vulnerability
Assessments
What’s Missing?
Mike Pittenger
VP, Security Strategy

2. Increasing regulatory scrutiny
• Force of law and penalties
• Expanding and overlapping
Common Goals
• Focus on protecting sensitive
information
• Documented responsibilities and
processes
• Require visibility to risks (e.g.,
vulnerability assessments)
Regulatory Landscape is Expanding and Overlapping
GLBA Sarbanes - Oxley

3. Is This a Network Security Problem?
Perimeter Defense
• Since the PII is stored
in a database, initial
focus was on network
security
• Firewalls, virus scan
and good network
configuration keep us
secure
• Encryption of data at
rest

4. Approach Matches Industry ”Spend” v. Actual Risk
MANY CLIENTS DO NOT PRIORITIZE APPLICATION SECURITY IN THEIR ENVIRONMENTS
35%
30%
25%
20%
15%
10%
5%
APPLICATION
LAYER
DATA
LAYER
NETWORK
LAYER
HUMAN
LAYER
HOST
LAYER
PHYSICAL
LAYER
SECURITY RISK
SPENDING
SPENDING DOES
NOT EQUAL RISK
Source: The State of Risk-Based Security Management, Research Study by Ponemon Institute, 2013

5. PCI-DSS
Build and Maintain a Secure Network and Systems
• Requirement 1 – Configure firewalls and routers to protect against unauthorized access to cardholderdata
• Requirement 2 – Don’t use vendor supplied default passwords and configurations
Protect Cardholder Data
• Requirement 3: Protect stored cardholder data, and don’t storewhat you don’t need
• Requirement 4: Use strong crypto and protocols
Maintain a Vulnerability Management Program
• Requirement 5: Use anti-virus
• Requirement 6: Develop and maintain secure systems and applications
Implement StrongAccess Control Measures
• Requirement 7: Limit access on a need-to-know basis
• Requirement 8: Use and maintain identitymanagement tools correctly
• Requirement 9: Physical security
Regularly Monitor and Test Networks
• Requirement 10: Log and audit all activity
• Requirement 11: Regularly test security systems and processes.
Maintain an Information Security Policy
• Requirement 12: Train your people
Where Does Application Security Apply?

6. Requires independent risk
rating of vulnerabilities
Requires monitoring for new
vulnerabilities, and applying
patches as available
Maintain a Vulnerability Management Program
6.1 Establish a process to identify
security vulnerabilities, using
reputable outside sourcesfor security
vulnerability information, and assign a
risk ranking(for example, as “high,”
“medium,” or “low”) to newly
discovered security vulnerabilities.
<snip>
This is not achieved by an ASV scan or
internal vulnerability scan, rather this
requires a processto actively monitor
industry sources for vulnerability
information.

7. • Requires security patches for
critical vulnerabilities to be
implemented within 30 days of
release (disclosure)
• Requires identification of
vulnerabilities in custom code
and all installed software
• Requires review of custom
code to identify vulnerabilities
Maintain a Vulnerability Management Program
6.2 Ensure that all system components and
software are protected from known
vulnerabilities by installing applicable vendor-
supplied security patches. Install critical
security patches within one month of
release.
<snip>
This requirement applies to applicable
patches for all installed software.
6.3.2 Review custom code prior to release to
production or customers in order to identify
any potential coding vulnerability…
Without the inclusion of security during the
requirements definition, design, analysis, and
testing phases of software development,
security vulnerabilities can be inadvertently
or maliciously introduced into the
production environment.

8. Regularly Monitor and Test Networks
Vulnerability assessments
• Quarterly requirement
• Ad hoc internal assessments
• “Continuous monitoring” (daily scans)
Vulnerability assessment (VA) tools focus on:
• System configurations
• Operating systems (including Linux)
• Commercial applications (Office, Adobe,
Oracle, etc.)
PCI-DSS 11.2
“Run internal and external network
vulnerability scans at least
quarterly and after any significant
change in the network (such as new
system component installations,
changes in network topology, firewall
rule modifications, product
upgrades).”

9. What’s missing?

10. What’s Missing?
1 – VA tools provide no visibility to custom applications
• Focus on commercial products and OSS applications
• No visibility to open source components you use
• No visibility to vulnerabilities in those components
2 – VA tools are reactive
• You must scan your entire system and applications for each new
update or vulnerability
• Do not maintain an inventory of your applications
• Each new issue must be searched for independently

11. How Well Do VA Tools Cover Open Source?
2015 - 3,000 vulnerabilities disclosed in open source
Nessus 2015 - Roughly 500 plug-ins generated
Focus on major components and OS
• 34 rules for
Poodle
• 14 for Freak
• 205 for Linux
• 35 for Red Hat
• 42 for SuSE
• 25 for Ubuntu
• 33 for Fedora
• 28 for Debian
• 14 for CentOS
• 11 for Mandriva

12. Proactive Vulnerability
Management

13. What if the Automotive Market Treated Recalls
Like Open Source Users Treat Vulnerabilities?
Known and Quantified Known and Unquantified

14. Automotive
• Regimented processes (JIT)
• Specificity in roles
• QA at each step
Software
• Developer independence
• Broader functional roles
• QA for builds
How Is Software “Manufacturing” Different?
Open Source
Community
Internally
Developed
Code
Outsourc
ed Code
Legacy
Code
Reused
Code
Supply
Chain
Code
Third
Party
Code
Delivered
Code
Open source code
introduced in many
ways…
…and absorbed
into final code.

15. Automotive
• Faulty part is disclosed
• Recall is issued
• OEM notify specific vehicle owners
Software
• Vulnerable component is disclosed
• Some users learn of vulnerability
• Hilarity ensues
How Is “Incident Response” Different?

16. Who’s Responsible for Monitoring Component Security?
Commercial Code Open Source Code
• Dedicated security researchers
• Alerting and notification infrastructure
• Regular patch updates
• Dedicated support team with SLA
• “Community”-based code analysis
• Monitor newsfeeds yourself
• No standard patching mechanism
• Ultimately, you are responsible

17. Is This a Big Deal?
Over 7,000 new
vulnerabilities in open
source since 2014
Over 76,000 total
vulnerabilities in NVD,
only 63 reference
automated tools
• 50 of those are for vulnerabilities
reported in the tools
• 13 are for vulnerabilities that
could be identified by a fuzzer
0
200
400
600
800
1,000
1,200
2010-Apr
2010-Feb
2010-Jun
2010-Nov
2011-Apr
2011-Feb
2011-Jun
2011-Nov
2012-Apr
2012-Feb
2012-Jun
2012-Nov
2013-Apr
2013-Feb
2013-Jun
2013-Nov
2014-Apr
2014-Feb
2014-Jun
2014-Nov
2015-Apr
2015-Feb
2015-Jun
2015-Nov
2016-Apr
2016-Mar
NVD
Open Source Vulnerability Disclosures by
Month
Heartbleed
Disclosure

18. Open Source: Attackers Have Quotas Too
Easy access to code
Vulnerabilities are publicized Exploits readily available
Used everywhere

19. A Software Bill of Materials Solves the Problem
• Components and serial numbers
• Unique to each vehicle VIN
• Complete analysis of open source components*
• Unique to each project or application
• Security, license, and operational risk surfaced

20. Continuous Monitoring for New Issues
Monitoring of risk to in-scope, production systems
• New vulnerabilities not detected by VA tools
• Monitoring for risk in custom applications
• Does not require re-scanning

21. Key Takeaways
PCI (and most regulatory standards) covers “all installed software”
Vulnerability Assessment tools are valuable, but
• Don’t cover custom software
• Don’t maintain knowledge of components
You Bill of Materials solves the issue of visibility, but updating the
components remains a requirement

22. Questions?