In a rapidly changing IT environment, detecting and responding to new threats is more important than ever. This session shows you how to build a predictive analytics stack on AWS, which harnesses the power of Amazon Machine Learning in conjunction with Amazon Elasticsearch Service, AWS CloudTrail, and VPC Flow Logs to perform tasks such as anomaly detection and log analysis. We also demonstrate how you can use AWS Lambda to act on this information in an automated fashion, such as performing updates to AWS WAF and security groups, leading to an improved security posture and alleviating operational burden on your security teams.
2. Cybersecurity headlines from 2015…
...Over 169 million personal records were exposed, stemming from 781
publicized breaches across the financial, business, education,
government and healthcare sectors.
...There were 38 percent more security incidents detected than in 2014.
...The median number of days that attackers stay dormant within a
network before detection is over 200.
... 81 percent reported they had neither a system nor a managed security
service in place to ensure they could self-detect data breaches, relying
instead on notification from an external party.
... Only 38 percent of global organizations claim they are prepared to
handle a sophisticated cyberattack.
3. You will learn how to…
Build a log analytics stack with Amazon Elasticsearch
Service
Utilize Amazon Machine Learning to predict bad actors
Perform forensic analysis on your network paths
Implement advanced options in your continuous,
predictive security stack
10. Awesome, we can see stuff!
Now we have real-time visualization of all logs
Great for risk scenarios we
already know about!
Example – Single user logging in from
several IP addresses
Not so great for unknown
scenarios
There are many of these!
How do we protect against these risks?
12. Amazon Machine Learning
Easy to use,
managed machine
learning service built
for developers
Robust, powerful
machine learning
technology based on
Amazon’s internal
systems
One-click production
model deployment
Binary classification
Multiclass classification
Regression
13. Using Amazon Machine Learning’s real-time predictions, we
can drastically shorten how long it takes you to become aware
of a threat
14. Training your model (daily)
Amazon S3
Stores machine
learning dataset
AWS Lambda
Daily machine
learning model
training
Amazon Machine
Learning
Build model from
dataset
Log analytics
stack
AWS Lambda
Transform and
store logs in S3
15. Using Big Data – Example dataset
{
"datetime": "7/30/16 0:20",
"AWSregion": "aws-sa-east-1",
"IP": "69.90.60.155",
"protocol": "TCP",
"source": "6000",
"destination": "1433",
"country": ”BrVirginIslands",
"region": ”PricklyPear",
"postalcode": ”VG1120",
"Lat": ”18.5000",
"Long": ”64.3667”,
"Threat": 94
}
18. Security stack
Amazon Machine
Learning
Trained model and
endpoint for real-
time predictions
Log analytics
stack
AWS Lambda
Trigger on each
new log entry
Amazon SNS
notification
Amazon S3
Stores machine
learning dataset
AWS Lambda
Daily machine
learning model
training
AWS Lambda
Transform and
store logs in S3
19. Close, but not perfect!
We still wont catch every potential breach
Machine learning cannot predict every possible threat
Attackers are getting smarter and more sophisticated every day
When one does occur, we want to know why
This helps us prevent it from happening again!
26. Evolving the practice of security architecture
Security architecture as a separate function can no longer
exist
Static position papers,
architecture diagrams, and
documents
UI-dependent consoles and
technologies
Auditing, assurance, and
compliance are decoupled,
separate processes
Current security
architecture
practice
27. Evolving the practice of security architecture
Architecture artifacts
(design choices, narrative,
etc.) committed to common
repositories
Complete solutions account
for automation
Solution architectures are
living audit/compliance
artifacts and evidence in a
closed loop
Evolved security
architecture
practice
AWS
CodeCommit
AWS
CodePipeline Jenkins
Security architecture can now be part of the “maker” team
29. More sophisticated machine learning models
Train your model with your data
Real-world data specific to your application
Previous threats you have dealt with
Considering modeling threats by clusters of logs
Identify threats more accurately than just a single log entry
Build threat profiles that pattern typical attack stages
Reconnaissance, scanning, gaining access, maintaining access, and
covering tracks
30. Tying it all together
Amazon Machine
Learning
Trained model and
endpoint for real-
time predictions
Log analytics
stack
AWS Lambda
Trigger on each
new log entry
Amazon SNS
notification
Amazon S3
Stores machine
learning dataset
AWS Lambda
Daily machine
learning model
training
AWS Lambda
Transform and
store logs in S3
AWS Config Rules
Network
reasoning
VPC, security groups,
network ACLs
31. Next steps
Set up your log analytics stack: http://amzn.to/2dIZjIz
Blog post and AWS CloudFormation template
Build your first Amazon ML machine learning model:
http://amzn.to/1K8HfRu
Stay tuned on the AWS Security Blog for more on this
topic
We’re here all week! Come chat with us.