With AWS CloudTrail, you can get log files of AWS API calls for your account. CloudTrail enables you to perform security analysis, track resource changes, and aid in compliance reporting.
In this webinar you will learn how CloudTrail collects and stores your AWS log files so that software from AWS Technology Partner Splunk can be used as a Big Data Security Information and Event Management (SIEM) system. You will hear how AWS log files are made available for many security use cases, including incident investigations, security and compliance reporting, and threat detection/alerting. You will also hear from a joint Splunk/AWS customer, FINRA, who will explain how they leverage Splunk in AWS to support their cloud efforts.
What you'll learn:
• Why the machine data from AWS CloudTrail is relevant to security and compliance
• How to visualize data from AWS CloudTrail to monitor and audit security-related activity
• How AWS CloudTrail data can be combined with machine data from other sources in your IT infrastructure, including the OS and apps in your AWS images, for a wide range of operational and security use cases
• How the combination of AWS CloudTrail and Splunk Software improve your uptime, accelerate security and operational investigations, and simplify compliance.
3. Webinar Overview
Submit Your Questions using the Q&A tool.
A copy of today’s presentation will be made available on:
AWS SlideShare Channel@ http://www.slideshare.net/AmazonWebServices/
AWS Webinar Channel on YouTube@ http://www.youtube.com/channel/UCT-
nPlVzJI-ccQXlxjSvJmw
4. Sivakanth Mundru
Sr. Product Manager
Amazon Web Services
Gary Mikula
Sr. Dir. of Information Security
FINRA
Introducing
Praveen Rangnath
Director of Product Marketing
Splunk
Joe Goldberg
Security Product Marketing
Splunk
5. Overview of Amazon CloudTrail
How Splunk analyzes CloudTrail and other machine data to improve security
and compliance
Case study: How FINRA leverages Splunk Cloud and Splunk App for AWS
CloudTrail to support their cloud efforts
Demo: CloudTrail logs in Splunk App for Enterprise Security
Q&A
What We’ll Cover
7. Amazon Confidential
Introduction
Customers are
making API
calls...
On a growing set
of services around
the world…
CloudTrail is
continuously
recording API
calls…
And delivering
log files to
customers in
less than 15
min
7
9. AWS Services supported by CloudTrail
• CloudTrail supports 15 AWS services, including EC2, RDS, IAM, Redshift.
• Includes API calls made by higher-level AWS services such as AWS CloudFormation, AWS
Elastic Beanstalk and AWS OpsWorks to other AWS services (EC2,RDS etc..)
Amazon Confidential
9
Image credit: Jeff Barr
10. Information in a recorded API call
• Who made the API call?
• When was the API call made?
• What was the API call?
• What were the resources that were acted up on in the API call?
• Where was the API call made from?
Amazon Confidential 10
11. Who made the API call?
• Records detailed information for all AWS identity types
Root user
IAM user
Federated user
Role
• Information includes
Friendly user name
AWS AccessKeyId
12 digit AWS account number
Amazon Resource Name (ARN)
Session context and issuer information, if applicable
invokedBy section identifies the AWS service making request on behalf of
the user
Amazon Confidential 11
12. Who? Example 1: IAM user Bob making an API call
"userIdentity": {
"accessKeyId": "AKEXAMPLE123EJVA",
"accountId": “123456789012",
"arn": "arn:aws:iam::123456789012:user/Bob",
"principalId": "AIEXAMPLE987ZKLALD3HS",
"type": "IAMUser",
"userName": “Bob"
}
Amazon Confidential
12
13. Who? Example 2: Federated user Alice making an API
call
"userIdentity":{
"type":"FederatedUser",
"principalId":"123456789012:Alice",
"arn":"arn:aws:sts::123456789012:federated-user/Alice",
"accountId":"123456789012",
"accessKeyId":"ASEXAMPLE1234WTROX8F",
"sessionIssuer":{
"type":"IAMUser",
"accountId":"123456789012",
"userName":“Bob"
}
}
Amazon Confidential
13
14. When was the API call made?
• Time and Date of the event in ISO 8601 format
"eventTime": "2013-10-23T23:30:42Z“
• Event time is captured on the service host where the API call is executed
• Event time is NOT the time log file is written to S3
Amazon Confidential
14
15. What was the API call?
What resources were acted up on?
• API call and the service the API call belongs to.
"eventName": "RunInstances"
"eventSource": "EC2"
• Request parameters provided by the requester and Response elements
returned by the AWS service
• Response elements for read only API calls (Describe*, Get*, List*) are not
recorded to prevent event size inflation
Amazon Confidential
15
16. Where was the API call made from and to?
• Apparent IP address of the requester making the API call
• Records the apparent IP address of the requester when making API calls
from AWS Management Console
• AWS region to which the API call was made. Global services
( Examples: IAM/STS) will be recorded as us-east-1
"sourceIPAddress": "54.234.127.135",
"awsRegion": "us-east-1“
Amazon Confidential
16
17. Errors and Authorization Failures
• Detailed and Descriptive error codes and error messages, recorded only
when errors occur.
Examples
Client error code: TagLimitExceeded
Server error code: Internal Error
Authorization failure: UnauthorizedOperation
• Authorization Failure Example
“eventName": “TerminateInstances",
“errorCode": “UnauthorizedOperation”,
“errorMessage”:”You are not authorized to perform this operation”
Amazon Confidential
17
18. Use cases enabled by CloudTrail
• Security Analysis
Use log files as an input into log management and analysis solutions to perform
security analysis and to detect user behavior patterns.
• Track Changes to AWS Resources
Track creation, modification, and deletion of AWS resources such as Amazon EC2
instances, Amazon VPC security groups and Amazon EBS volumes.
• Troubleshoot Operational Issues
Quickly identify the most recent changes made to resources in your environment.
• Compliance Aid
Easier to demonstrate compliance with internal policies and regulatory standards.
Amazon Confidential
18
19. Splunk – Company Overview
Company (NASDAQ: SPLK)
Founded 2004, first software release in 2006
HQ: San Francisco / Regional HQ: London, Hong Kong
Over 1,000 employees, based in 12 countries
FY 2014 Revenue: $302M (YoY +52%)
Business Model / Products
Free download to massive scale
Splunk Enterprise, Splunk Cloud
Hunk: Splunk Analytics for Hadoop
7,000+ Customers
Customers in over 90 countries
More than 60 of the Fortune 100
Largest license: Over 100 Terabytes per day
Mission: Make machine data accessible, usable, and
valuable to everyone
19
20. What is Machine Data?
Volume | Velocity | Variety | Variability
GPS,
RFID,
Hypervisor,
Web Servers,
Email, Messaging,
Clickstreams, Mobile,
Telephony, IVR, Databases,
Sensors, Telematics, Storage,
Servers, Security Devices, AWS CloudTrail
Machine data is the fastest growing, most
complex, most valuable area of big data
22. Aug 08 06:09:13 acmesep01.acmetech.com Aug 09 06:17:24 SymantecServer acmesep01: Virus found,Computer
name: ACME-002,Source: Real Time Scan,Risk name: Hackerremotetool.rootkit,Occurrences: 1,C:/Documents
and Settings/smithe/Local Settings/Temp/evil.tmp,"""",Actual action: Quarantined,Requested action: Cleaned,
time: 2009-01-23 03:19:12,Inserted: 2009-01-23 03:20:12,End: 2009-01-23 03:19:12,Domain: Default,Group: My
CompanyACME Remote,Server: acmesep01,User: smithe,Source computer: ,Source IP: 10.11.36.20
Aug 08 08:26:54 snort.acmetech.com {TCP} 10.11.36.20:5072 -> 10.11.36.26:443 itsec snort[18774]:
[1:100000:3] [Classification: Potential Corporate Privacy Violation] Credit Card Number Detected in Clear Text
[Priority: 2]:
{"requestParameters": {"durationSeconds": 43200}, "responseElements": {"credentials": {"sessionToken":
"AQoDYXdzEPP///==", "accessKeyId": "ASIAJWQDLBKDOAKEWNIQ", "expiration": "Nov 13, 2013 5:22:32 AM"},
"eventSource": "sts.amazonaws.com", "sourceIPAddress": “10.11.36.1", "eventTime": "2013-11-12T17:22:32Z",
"userIdentity": {Administrator:root", "principalId": "930458123955", "accountId": "930458123955", "type":
"Root"}, "eventName": "GetSessionToken", "userAgent": "signin.amazonaws.com"}
22
Machine Data Contains Critical Insights
Sources
Time Range
Intrusion
Detection
Endpoint
Security
AWS
CloudTrail
All three occurring within a 24-hour period
Example Correlation – Data Loss
Source IP
Source IP
Source IP
Data Loss
Default Admin Account
Malware Found
23. Big Data SIEM – All Data is Security Relevant
OSes
Service
Desk
Storage
CloudTrailEmail Web
Call
Records
Network
Flows
DHCP/ DNS
Hypervisor
Custom
Apps
Industrial
Control
Badges
Databases
Mobile Intrusion
Detection
Firewall
Data Loss
Prevention
Anti-
Malware
Vulnerability
Scans
Traditional SIEM
Authentication
24. The Top Five Splunk Security Use Cases
a SIEM plus much more
Security &
Compliance
Reporting
Real-time
Monitoring of
Known
Threats
Real-time
Monitoring of
Unknown
Threats
Incident
Investigations
& Forensics
Splunk Can Complement OR Replace Existing SIEMs
Fraud detection
26. Leading Big Data SIEM (plusmore!)
26
Gartner SIEM MQ Best SIEM & Enterprise
Security Solution
Best SIEM
27. Splunk Offerings For AWS Security and Compliance
• App for AWS
CloudTrail - FREE
• Splunk App for
Enterprise Security
Applications
• Splunk Enterprise as
a service
• Full app, SDK, API,
platform support
SaaS
• Self-deploy in cloud
or on-premises
• Centralized view
across cloud and
on-premises
• Splunk Enterprise
and Hunk AMIs
• Accelerate
deployment in AWS
Amazon Machine
Images (AMI)
Software
28. FINRA’s use of AWS, Splunk Cloud &
Splunk App for AWS CloudTrail
followed by…
Demo of CloudTrail data in
Splunk App for Enterprise Security
29. FINRA – CloudTrail Copyright 2014 FINRA
Who We Are
FINRA—the Financial Industry Regulatory Authority—is an independent,
non-governmental regulator for all securities firms doing business with
the public in the United States.
FINRA protects investors by regulating brokers and brokerage firms and
by monitoring trading on U.S. stock markets.
FINRA watches over 6 billion shares traded on the stock market each
day
FINRA handles more ‘big data’ on a daily basis than the Library of
Congress or Visa—to build a holistic picture of the trading market
FINRA – Deter, Detect, Discipline
30. FINRA – CloudTrail Copyright 2014 FINRA
Where We Were
FINRA onPrem
Data Center
Location A
FINRA onPrem
Data Center
Location B
LOTS OF HARDWARE
DR REQUIRED CONFIG CHANGES
TRADITIONAL SIEMs ONLY KNOW
MESSAGES THAT THEY KNOW ABOUT
SIEMs THINK ONLY SECURITY WILL NEED LOGS
CANNED ALERTS – MORE MARKETING THAN REALITY
LACK OF USER COMMUNITY KNOWLEDGE BASE
31. FINRA – CloudTrail Copyright 2014 FINRA
Where We Are Today
Offload Hardware Worries
What DR?
Can Collect Anything
Widened Our User Base
Granular AC
Easily Duplicated All Reporting &
Alerting
Vendors Give Us Apps!!!
Great User Community
32. FINRA – CloudTrail Copyright 2014 FINRA
Why the AWS CloudTrail Application?
FINRA has a goal to be fully in the Cloud within 5 years
AWS is currently FINRA’s primary Cloud Provider
Data Collection via AWS s3 bucket objects not trivial
CloudTrail covers many ServicesAPIParameters
SQS messages are small pointers to the s3 objects
CloudTrail captures everything, but Splunk App allows for filtering
Fully extracted & tagged AWS CloudTrail records in an easy, flexible UI. Of
course, all Splunk S&R is available as well.
35. FINRA – CloudTrail Copyright 2014 FINRA
Use Cases
Operations
Who started that ec2 in development?
Who stopped that ec2 in production?
Security
Was that change to the security group authorized?
Why was that user added to the group?
Why is this ID generating so many AuthFailure/AccessDenied?
Application
My application worked yesterday, what changed?
Have I been added to the monitoring group yet?
36. FINRA – CloudTrail Copyright 2014 FINRA
Conclusion
True security requires collecting ALL data
AWS CloudTrail delivers valuable visibility into user
account activity
Splunk dashboards / reports coupled with search and
reporting is critical
AWS and Splunk Enable Secure Cloud Adoption