Security Validation through Continuous Delivery at Verizon - DEV403 - re:Invent 2017

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Security Validation through
Continuous Delivery at Verizon
C h r i s D u r a n d , D i r e c t o r o f C l o u d S e c u r i t y I n t e g r a t i o n S e r v i c e s ,
V e r i z o n
C h u c k D u d l e y , V P o f S e r v i c e s , S t e l l i g e n t
M a t t h e w D w y e r , A W S P r o f e s s i o n a l S e r v i c e s
D e c e m b e r 1 , 2 0 1 7
D E V 4 0 3
AWS re:INVENT

What to Expect from This Session
• How AWS Professional Services approaches security engagements
• Verizon’s Journey to AWS and their strategic plan for scaling security
• Understanding Continuous Security and how it fits into a DevOps
approach
• Deep dive into security static analysis of CloudFormation templates and
developing custom rules
• Automated certification of CloudFormation stacks
• Security as an automated gate to production

The AWS Professional Services
Security Perspective

AWS Security Perspective
Every company is concerned with
protecting information and assets as
they grow the business. They also
want to ensure they are operating
within the legal boundaries and
standards set by and on the behalf of
governmental agencies and industry
associations.
Security Perspective components
provide guidance that enables a
comprehensive and rigorous method
of describing a structure and behavior
for an organization’s security and
compliance processes, systems and
personnel.
Security Perspective
Directive
Preventative Detective
Responsive

Outcome Focused
Build core foundations backed by continuous
improvement of operations
Identify and document use and misuse cases that will
drive implementation
Clear implementation plan through sprints
Implement AWS-native security services at an
accelerated pace

Security Approach
Devise an iterative plan to rapidly transition from building core
foundations to establishing maturity in the cloud
ü Core 5 Security Epics
• IAM
• Logging and Monitoring
• Infrastructure Security
• Data Protection
• Incident Response
ü Augmenting the Core 5
• Resilience
• Compliance validation
• DevSecOps
• Configuration and vulnerability
analysis
• Security big data and
predictive analytics

Sample Security Epics Team
Build one or more security
epics delivery team with AWS
security consultants
Run a series of security sprints
to to secure the customer’s
cloud journey
AWS Security
Consultant
Customer’s
IAM Engineer
Customer’s
Infra Security Engineer
Customer’s
Data Protection Engineer
Customer’s
IR Engineer
Customer’s
Logging and Monitoring Engineer

Delivery Team – Increased Velocity
Identity and
Access
Management
Logging and
Monitoring
Infrastructure
Security
Data Protection Incident
Response
Run multiple security delivery teams in parallel to increase agility and
velocity

Outcomes
Enabled cloud security teams with AWS-native security
operations and deployment skills
Core security and compliance control and capability
Culture of security ownership and continuous improvement of
security operations

Verizon’s Journey to the AWS Cloud

Who We Are
We are Verizon.
• Verizon delivers
the promise of
the digital world.
• Fortune 500
rank: #14
• $31.7 B in third-quarter
revenue (2017)
• 160,100 employees
Wireless leadership
LTE covers more than 98% of U.S. population
115.3 M retail connections
LTE Advanced covers 470 markets
Largest all-fiber Fios network
5.8 M Fios internet and 4.6 M Fios video connections
Fios Gigabit downloads as fast as 940 Mbps and uploads as
fast as 880 Mbps.
Global IP network
99% of Fortune 500 customers
Media and technology
Innovating in entertainment, digital
media, the Internet of Things and broadband service

Our Cloud Journey
Fundamentals:
Ø Migrate all applications into AWS securely
Ø Cloud deployment requires full automation
Ø Security will enable cloud adoption through automation
Ø Compliance monitoring is not enough

The Journey
Ø Beginning in December 2015
Ø Start with Non Production
Ø Baseline Structure:
Ø Line of Business Account
Approach
Ø Shared Service Accounts for
Support
Ø No Live Data!

The Approach
Logging
and
Monitoring
Identity and
Access Mgmt
Infra/VPC
Security
Data
Protection
Vulnerability
Analysis
Incident
Response
Compliance
Validation
CICD
Security
Ø Formalize a team
Ø Agile based
Ø Leverage key partnerships
Ø Use existing tools/processes where it makes sense
Ø Log everything
Ø Full automation
Ø Build DevSecOps

Requirements
ü Onboarding Guidelines
ü Risk Based
ü Preventative Controls
ü Continuous Monitoring
ü Auto-Remediation
ü Skillsets and Training
Consistent and Current Documentation
Usage policies and onboarding
documentation.
Prevent Compliance Issues
DevSecOps pipeline
Risk Based Approach
Prioritization remediation efforts based on
risk
Automated Remediation Capability
Enforce compliance to key controls
Comprehensive Monitoring
Identify compliance infractions in near real-
time
Team of Automation Engineers
Security turns to code and coding

Security through
Automation
Automated Security Solutions
ü Self-service IAM roles and AWS KMS keys
ü Static AWS CloudFormation scanning
ü Dynamic infrastructure compliance check
ü Auto-remediation:
§ Security groups rules;
§ Encryption (Amazon S3, Amazon RDS,
Amazon EBS, etc.)
ü Self-remediation encryption tools
ü Risk-assessment monitoring:
§ Logging-compliance monitoring
§ Bucket-policy compliance
§ IAM policy risks
Amazon CloudWatchAWS Lambda
Amazon EC2

Constraints and Challenges
Ø Scale
Ø Governance
Ø Diversity of Applications
Ø Culture
Ø Automation

Partnerships
AWS ProServ
resources assigned to
epics
Stelligent assigned to
DevOps and IAM

DevOps and Security
Security as a first-class citizen of the DevOps pipeline

Development Security Operations

What Is DevOps?
Development OperationsThrough
QA Security Governance

Release Process… The Old Way
Release

Release Process… With DevOps
and Continuous Delivery
Release? Release?

Release
Security… The Old Way

Release Process… With DevOps
and Continuous Security
Release? Release?

Security OF and IN the Pipeline
Security as a first class citizen of the DevOps pipeline

Security OF the Pipeline
Your code IS your AWS
environment
Therefore security starts
with source control
Access and authorization
and isolation is even
more important with
infrastructure code
Hardening the pipeline
is essential

Hardening the Pipeline
• Managed services
(AWS CodePipeline)
are preferred
• If your organization
requires, you may
need manage your
own pipeline
infrastructure
• Authorization and
access are critical
• Isolation to
prevent cross-
contamination
• Least privilege is
very important in
the pipeline

• Security as a gate function doesn’t scale
• Fast feedback loops provide speed and confidence
in software development, so use the same
techniques when developing infrastructure code
• Pulling the security validation forward in the
pipeline produces more secure code more quickly
AND more security-aware developers
Security IN the Pipeline

The Stelligent Pipeline
Commit Acceptance Capacity Pre-Prod Preproduction

The Commit Stage
GOAL:
Fast feedback for developers
Pipeline Actions:
1. Unit Tests
2. Static Code Analysis
Security Tests:
1. Security static analysis of application code
2. Security static analysis of infrastructure
code

Static Analysis (Commit Stage)
• Typically runs fast and can give developers good feedback long
before the security team can
• Makes predictions based upon understanding of code
before deploying resources
• Can't be the sole security analysis, because it's only predictive

• Scan CloudFormation template for obvious
security shortcomings:
• IAM wildcard actions, principals,
resources
• Security groups or port ranges open to
the world
• Encryption not enabled (EBS volumes)
• Access logging not enabled (ELBs,
buckets)
• Scan occurs BEFORE create-stack
• Stop the pipeline on failing violations, notify on
warnings
cfn_nag Usage

• Each rule has:
• A unique identifier
• A description for found violations
• An indicator of whether it is a failure
or a warning
• Typical logic for a rule:
• Select all AWS resources of a certain
type (e.g. AWS::IAM::ManagedPolicy)
• Interrogate each resource for a
security anti-pattern
• Record the logical resource identifiers
for violating resources
cfn-nag – Rule Development

cfn-nag – Rule Example
require 'cfn-nag/violation'
require_relative 'base'
class IamManagedPolicyWildcardActionRule < BaseRule
def rule_text
'IAM managed policy should not allow * action'
end
def rule_type
Violation::FAILING_VIOLATION
end
def rule_id
'F5'
end
def audit_impl(cfn_model)
violating_policies = cfn_model.resources_by_type('AWS::IAM::ManagedPolicy').select do
|policy|
!policy.policy_document.wildcard_allowed_actions.empty?
end
violating_policies.map { |policy| policy.logical_resource_id }
end
end

Demo

• As security tools are integrated into a delivery
pipeline, security/operations has to have enough
control to dictate that these tools be run before a
deployment to production
• If security/operations controls the last step of the
pipeline before production and runs these tools,
that is too late to receive feedback
• If security/operations controls the pipeline
outright, then that becomes a chokepoint for
developers being able to do their work quickly
• Can control pipeline structure with extension
points to allow customization, but if we can certify
security tools were invoked...
Problem – Certification of
Test Results

Question:
How to prove that proper tests have
been run against an object and the
results are acceptable?
Answer:
Need a few things together to certify:
• Actual object under test
• The test specification
• Test results
• Trust in the test administrator, and that he
actually ran the test
• In so many words, prove that: "The test
administrator applied a set of particular tests to
that particular object and obtained a set of
particular results"
Certification of Test Results –
in Abstract

Question:
How to prove that static analysis has
been run against an AWS
CloudFormation template and it
doesn't have known security anti-
patterns?
Answer:
Need a few things together to certify:
• AWS CloudFormation template, a.k.a. the object
under test
• cfn-nag list of rules applied, a.k.a. the
test specification
• cfn-nag results
• Aforementioned content digitally signed together
by a trusted test administrator
Certification of Test Results
– in Abstract

• cfn-nag fronted by a simple REST API
• Request: AWS CloudFormation template
• Response: single JSON document including:
• AWS CloudFormation template
• List of cfn-nag rule identifiers applied
• Result of cfn-nag analysis
• Digital signature of whole document
(Ed25519 - libsodium)
• Distribute public half of key pair to verify the
signature and cfn-nag service administered by
security operations holds the private half
Trusted Test Administrator –
cfn-nag service

cfn-nag Service Response Example
"signature":"Z1ZLw8nAjBB3nf8kEmT2ctqgotCZIAKd8jXHCWTIQq24Ngbikqnt+Mtj556mbEO7F2VmG
u8X7jPUIB93DA==",
"cfn_template_with_cfn_nag_results":"eyJjZm5fdGVtcGxhdGUiOiJ7XG4gIFwi...",
"cfn_nag_results": [
{
"id":"F2000",
"type":"FAIL",
"message":"User is not assigned to a group",
"logical_resource_ids":["iamUserWithAddition"]
}
]
}

• Base 64 encoding of a
digital signature of the
template alongside the
analysis results
• Nested JSON document that
includes the template and
results
• This is the content that is
actually signed
• JSON in a JSON field is a
nightmare so encoded with
Base 64
• Duplication of the signed cfn-
nag results, but human
readable for convenience
cfn-nag Service Response Anatomy
Signature
cfn_template_with_cfn_
nag_results
cfn_nag_results

• Security has control over a final automated
"deployment gate" to production
• Instead of a security team running scans before
deployment and becoming a choke point that
returns late feedback, scans are made public as
services for developers to run early and often
• The "deployment gate" verifies the digital
signatures to prove the scans have been done
instead of having to run them
• Can also verify the nature of the test (which rules
in case there are new ones)
Consuming Test Certification
Results

The Acceptance Stage
GOAL:
Comprehensive testing of the application and its infrastructure
Pipeline Actions:
1. Integration Tests
2. Acceptance Tests
Security Tests:
1. Infrastructure Analysis

Configuration Inspection
and Governance
• After infrastructure is
converged, configuration is
inspected for security anti-
patterns
• Inspections can overlap with
static analysis checks, but can
also check for things that are
harder to discover from just
the code, for example, a
security group open to the
world on port 80 is attached to
an EC2 instance versus a load
balancer
• These inspections should be in
code, and in an ideal world,
code for on-demand
inspections from a pipeline
should be the same code as is
used for ongoing governance
and compliance activities

Configuration Inspection
and Governance
• AWS Config Rules are used to
ensure resources are compliant
with standards
• Can be used on-demand, but a
little awkward...
• As an AWS CloudFormation
stack is converged, can request
noncompliant resources from
Config Rules and filter them
against resources in the stack
under test
• Config Rules are evaluating
constantly in background
against the whole of the AWS
account as events occur
• Any noncompliant resources in
the stack stop the pipeline

• AWS Config Rules fronted by a simple REST API
• Request: Arn of a converged AWS CloudFormation
stack
• Response: single JSON document including:
• AWS CloudFormation template scraped
from converged stack
• List of AWS Config Rules applied
• List of noncompliant resources from the
converged stack
• Digital signature of whole document
(Ed25519 - libsodium)
• Distribute public half of key pair to verify the
signature and Config Rule service administered by
security operations holds the private half
Trusted Test Administrator –
Config Service

Attack! (Acceptance or
Cross-Functional Stage)
Beyond configuration
inspection to
instigating and
measuring behavior
Scan system for
vulnerabilities and
hit running system
with attacks to
capture the
response

The Capacity Stage
GOAL:
Test the system under real world conditions
Pipeline Actions:
1. Performance Tests
2. Load Tests
Security Tests:
1. OWASP ZAP Pen Test
2. OpenSCAP Image Testing

Vulnerability Scanning
and Penetration Testing
• Examine instances for
weaknesses
• Examine system for fatal
flaws via automated pen
testing
• Similar to previous stages,
though more difficult to
represent as code due to the
nature of the tooling
• Same precepts: Collect data on
target, rule sets and results,
digitally sign for verification

• Vision – BYOS (Bring Your Own Security):
Security-as-a-service with digital signatures
enables distributed teams to invoke
analysis in their own custom pipelines
without interference from centralized team
• Given the infrastructure certification
pipeline structure is mostly identical across
applications...
• Pipelines can be generated from metadata
furnished by the application teams
Bringing It Together:
Infrastructure Certification Pipeline

Conclusion
• Infrastructure IS code… treat it as such. Applying modern development techniques
such as TDD and Continuous Delivery yields immense value.
• Infrastructure is part of the solution in application development now. Its
development should be integrated into the application development process,
treating the solution as an integrated entity.
• From within development team, CD reduces cycle time for releases and improves
confidence in released code (including infrastructure code).
• From outside, it allows security/governance/compliance to inject best practices as
automated gates in the delivery process without introducing delays for review and
approval.
• This allows for control at scale without grinding to a halt.

Resources
https://stelligent.com/dev403

Thank you!

Security Validation through Continuous Delivery at Verizon - DEV403 - re:Invent 2017

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Security Validation through Continuous Delivery at Verizon - DEV403 - re:Invent 2017

Semelhante a Security Validation through Continuous Delivery at Verizon - DEV403 - re:Invent 2017 (20)

Mais de Amazon Web Services

Mais de Amazon Web Services (20)

Security Validation through Continuous Delivery at Verizon - DEV403 - re:Invent 2017