This technical session examines real world scenarios security professionals will encounter defending Cloud workloads running on Serverless Infrastructure. Attendees will see a series of hands-on attack techniques for extracting credentials from serverless functions, and how to leverage those credentials for data exfiltration. The session starts with insecure secrets management in Serverless. Live demonstrations will show how a vulnerability in a function can allow attackers to exfiltrate secrets from a configuration file inside the function’s execution environment. Attendees will then see how to extract credentials from a function’s execution environment, and use those credentials from a remote machine to gain unauthorized access to data. Next, the session explores the ephemeral execution environment that is supposed to live for a few hundred milliseconds and then disappear. In practice, does that hold true? Concluding the session, we discuss some defensive techniques for locking down serverless environments, controlling egress traffic, restricting credential access, and querying audit logs. Attendees will leave with an understanding of the common attacks and practical security controls for defending their Serverless Infrastructure.

1. © 2020 Puma Security, LLC | All Rights Reserved
FWD:CLOUDSEC 2020
Winning in the Dark:
Defending Serverless Infrastructure
Eric Johnson
Principal Security Engineer, Puma Security
Senior Instructor, SANS Institute
www.linkedin.com/in/eric-m-johnson
@emjohn20
June 29, 2020

2. © 2020 Puma Security, LLC | All Rights Reserved
SESSION GOALS
• Discover insecurely stored function secrets
• Exfiltrate authentication tokens from the serverless container
• Detect stolen authentication tokens accessing cloud resources
• Apply network controls to prevent command and control
• Leverage audit logging and monitoring to detect malicious
activity

3. © 2020 Puma Security, LLC | All Rights Reserved
ACKNOWLEDGEMENTS
Gal Bashan - @galbashan1
• https://github.com/epsagon/lambda-internals
OWASP Serverless Top 10 Project
• https://www.owasp.org/index.php/OWASP_Serverless_Top
_10_Project
• Major contributions from Puresec and Protego
Rhino Security
• AWS Penetration Testing with Kali Linux

4. © 2020 Puma Security, LLC | All Rights Reserved
PUMA SECURITY: SERVERLESS PREY
Serverless Prey is an open source repository
containing:
• Functions to establish a reverse shell in each
cloud
• Cheetah: Google Function
• Cougar: Azure Function
• Panther: AWS Lambda
• https://github.com/pumasecurity/serverless-prey
• https://www.youtube.com/watch?v=-
SSrAro8m3M

5. © 2020 Puma Security, LLC | All Rights Reserved
Function Execution Environment
Defending Serverless Infrastructure

6. © 2020 Puma Security, LLC | All Rights Reserved
SERVERLESS EXECUTION ENVIRONMENT
Reverse engineering each function's execution environment:
Function OS Directory User
NodeJS 12 Amazon Linux 2 /var/task sbx_user1051
.NET Core 3.1 Debian GNU/Linux 9 / app
Go 1.11 Ubuntu 18.04.2 LTS /srv/files root

7. © 2020 Puma Security, LLC | All Rights Reserved
SERVERLESS SECRETS: WHERE IS THE SOURCE CODE?
Start by looking for secrets in the function source code:
AWS Lambda
Azure Functions
GCP Functions
/var/task
/home/site/wwwroot/
/srv/files

8. © 2020 Puma Security, LLC | All Rights Reserved
SERVERLESS SECRETS : ENVIRONMENT VARIABLES
Secrets in environment variables can be accessed by remote attackers
using local file inclusion or command injection vulnerabilities:
1
2
3
4
5
6
7
8
9
$ cat /proc/self/environ
WEBSITE_AUTH_ENCRYPTION_KEY=BBDAD8269958635C8D4E3C713636D
APPSETTING_AzureWebJobsStorage=6BZ4kOCoSD7T1fc8v4h8JpRg==
APPSETTING_APPINSIGHTS_INSTRUMENTATIONKEY=5D17A234-6B81-
4777-8528-6814374E9BD3
MSI_SECRET=A788C6DE68224140A927BB412B4E24AB
AzureWebEncryptionKey=BBDAD8046F6B9F0E81A4B349
CONTAINER_ENCRYPTION_KEY=AYXxtNMabRpw2EIgoGpibUk= Azure
Functions

9. © 2020 Puma Security, LLC | All Rights Reserved
Defending Serverless Infrastructure
Function Execution Role

10. © 2020 Puma Security, LLC | All Rights Reserved
SERVERLESS EXECUTION ROLE
AWS Lambda
Azure Functions
GCP Functions
Functions gain access to other cloud resources (vault, secrets, storage,
database, etc.) by executing with predefined permissions:
Execution Role
Managed Identity
Service Account

11. © 2020 Puma Security, LLC | All Rights Reserved
GCP FUNCTION DEFAULT SERVICE ACCOUNT
• New functions inherit the Google managed "Editor" role by
default
• Editor role inherits read and modify state permissions for all
existing resources
• Function has full read and write access to storage buckets
• Payloads in the Secrets Manager require additional permissions
GCP Functions

12. © 2020 Puma Security, LLC | All Rights Reserved
SERVERLESS ACCOUNT CREDENTIAL STORAGE
Managed serverless platforms executing under a service account have
credentials stored in the following locations:
AWS Lambda
Azure Functions
GCP Functions
Environment Variables
Instance Metadata Service
Managed Identity

13. © 2020 Puma Security, LLC | All Rights Reserved
Function Data Persistence & Exfiltration
Defending Serverless Infrastructure

14. © 2020 Puma Security, LLC | All Rights Reserved
SERVERLESS FUNCTION CREDENTIAL LIFETIME
Comparing the credential expiration time (number of minutes) across
cloud providers:
AWS Lambda
Azure Functions
GCP Functions 0 100 200 300 400 500 600 700 800

15. © 2020 Puma Security, LLC | All Rights Reserved
Comparing the malware persistence lifetime (number of minutes)
across cloud providers:
AWS Lambda
Azure Functions
GCP Functions
FUNCTION MALWARE PERSISTENCE EXAMPLE
0 2 4 6 8 10 12

16. © 2020 Puma Security, LLC | All Rights Reserved
Detecting Compromised Function
Credentials
Defending Serverless Infrastructure

17. © 2020 Puma Security, LLC | All Rights Reserved
Analyzing service audit logs can identify credential usage outside
the function execution environment
AWS Lambda
Azure Functions
GCP Functions
Cloud Trail
FUNCTION CREDENTIAL AUDIT LOGGING
Azure Monitor (partial service support)
IAM Audit Logs

18. © 2020 Puma Security, LLC | All Rights Reserved
Function Network Access Controls
Defending Serverless Infrastructure

19. © 2020 Puma Security, LLC | All Rights Reserved
DEFAULT FUNCTION EXECUTION NETWORKING
• Configurable triggers from
HTTP or API Gateway events
• Routing allows egress traffic to
the Internet
• Routing allows egress traffic to
public cloud service APIs

20. © 2020 Puma Security, LLC | All Rights Reserved
Function execution environments can integrate with customer
managed virtual private cloud networks:
AWS Lambda
Azure Functions
GCP Functions
Virtual Private Cloud
FUNCTION NETWORK INTEGRATION OPTIONS
Virtual Network Integration
*premium plan only
Not Available

21. © 2020 Puma Security, LLC | All Rights Reserved
• Create egress traffic filters
• Enable flow logging / traffic
mirroring
• Create private endpoints for
managed cloud services
access
• Block access to cloud
resources from the public
API
FUNCTION PRIVATE ENDPOINT CONFIGURATION
Function
Private Subnet
Public Subnet
Private Cloud
Endpoints
Internet
Storage Secrets
NAT Gateway
Virtual Private Cloud
Database

22. © 2020 Puma Security, LLC | All Rights Reserved
FWD:CLOUDSEC 2020
THANK YOU FOR ATTENDING!
QUESTIONS?
Eric Johnson
Principal Security Engineer, Puma Security
Senior Instructor, SANS Institute
www.linkedin.com/in/eric-m-johnson
@emjohn20
June 29, 2020