Will St. Clair: AWS San Francisco Startup Day, 9/7/17 Operations: Security Crash Course & Best Practices! All companies should build with security and protection of customer data as the number one priority. This talk will cover a wide range of best practices from MFA, root accounts, encrypting laptops, inventory management, MDM, and incident response. You'll learn key principles of how to build a secure organization to protect your data. Don't wait until your first security incident before putting these best practices in place.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
September 2017
AWS Startup Day
Security Crash Course
#AWSstartupday
#AWSstartupday

2. Will St. Clair
Sr. Solutions Architect, Education Technology
wstclair@amazon.com

3. People will always choose the
path of least resistance.

4. Visibility

5. Visibility
Aggregated Logs
Systems Databases Access Logs
Log
Tool
Have one central place to aggregate and
review logs
Log access logs, application, db queries,
and more
Central location makes it easy to detect
suspicious behavior and set automated
alerts
Centralized logging speeds up response
time.

6. Visibility
Aggregated Logs: Filtering at the host
System
Log
Tool
Pros
• Limits network bandwidth utilization
• Limited bottleneck at log ingestion point
• Can leverage application specific log features
Cons
• Costs CPU cycles to process log data
• Difficult to manage log filters across large fleets
• Not good for high performance systems

7. Visibility
Aggregated Logs: Filtering at log ingestion
System
Log
Tool
Pros
• Easy to manage filters in one place
• Scale out log servers as demand increases
• Frees up CPU cycles on servers
Cons
• Introduces potential network & CPU limits
• Increased network traffic
• Potential SPOF (Single Point of Failure)

8. Visibility
Aggregated Logs: Aggregation Layer
System
Log
Tool
Log Aggregation
Pros
• Can handle queuing in case of log server failure
• Scale out aggregation nodes as needed
• Frees up CPU cycles on servers
• Easy to push out filter rules to set of servers
Cons
• More to manage
• Not needed in small environments
• Increased network traffic vs host level filters

9. Visibility
Logging: Write once, read many approach
System
Streaming Service
(Kinesis, Kafka, etc)
Log
Archives
Metrics
Write Read
Log tool

10. Least Permissions

11. Least Permissions
Processes and Files
• Run each process as it’s own user, not root
• Use file systems permissions to restrict or allow access to data
• Services like SELinux help
• Limit network access through security groups, and more

12. Least Permissions
Locking down SSH
• Ensure SSH is not open to the world
• Consider running SSH on an alternative port to reduce automated attacks
• Use key based SSH logins over passwords
• Trigger email notifications to distribution lists on system logins through PAM
• Look into third party services like DuoPush for added security

13. Least Permissions
Removing dependency on SSH
Interface to authenticate
and run commands.
Actions logged.
Output of commands run.
Production SystemAuthorized User
1
2
34

14. Least Permissions
Replacing SSH logins with EC2 Systems Manager
IAM control for who can run commands
Actions sent to CloudTrail for auditing
Can trigger automated responses
through SNS or Lambda

15. Keeping Systems Updated
Immutable Infrastructure
Users
Server
Server
Web Application Database
Server
Server
Users
version 1.1
minor modification
version 1.0
Each deployment builds on updated AMIs with updated packages
AMI 12.13 AMI 12.13
AMI 12.1 AMI 12.1

16. Encryption

17. Encryption
Avoid using IPSec everywhere
There are easier and more efficient ways to build transport encryption
When it fails, failures can be somewhat invisible. Things stop being able to
talk to other machines on the same network.
Hard to spot and hard to troubleshoot
The added overhead of management and potential risk of failure can
outweigh the benefits

18. Encryption
TLS is an easier approach
Most architectures in present day are API based making TLS an easy way to
manage encryption in transit
Many ways to wrap services with TLS including using lightweight TLS
proxies
Abstracts away from OS-level configuration
IPSec for legacy applications

19. Encryption
Encryption at rest: LUKS (non EBS)
System boots and attaches instance store volume
Key is generated on boot and stored in /dev/shm (in memory)
Luks is used to encrypt the volume by reading the key from /dev/shm
After encryption is done, key is deleted from /dev/shm
If instance terminates data vanishes
No way to steal a key and decrypt data; key has been thrown away
Ensure your systems replicate data across other AZs, Regions, and into
encrypted backups such as S3
Great for structured storage systems like Cassandra

20. Encryption
Encryption at rest: EBS w/ KMS
Simply check a box!

21. Encryption
Object Stores
Use server side encryption with object stores.

22. Encryption
Laptops & Workstations
MacOS Windows ChromeBooks
FileVault 2 BitLocker Built-in

23. Keeping Systems Updated
Alert on old servers
Server
12 days old
Server
18 days old
Server
32 days old
Automatically
rebuild server
Alert to manually
rebuild
or
- Old servers can accumulate and generate tech debt
- Outdated servers increase risk of security
vulnerabilities (outdated packages, manually tweaked
services, and more)
Start easily by manually refreshing
servers eventually automating it

24. Application Security

25. Application Security
Secure your build pipeline!
Source Code CI Server
Test &
Scan
Artifacts Deploy Monitoring
Each one of these steps can be exploited to push malicious code into your environment.
MFA No Shared Accounts Limited Access

26. Application Security
Building security testing into CI/CD
Source Code CI Server
Test &
Scan
Artifacts Deploy Monitoring
Nessus
TLS
ZAP
Owasp
Gauntlt

27. Application Security
Simple rules to keep you safe
Code review your application and infrastructure changes.
Don’t store credentials in your application
Avoid added unnecessary dependencies to your codebase
Always build for reliability and performance

28. Database & Analytics

29. Databases & Analytics
Code review process for migrations and prod changes
Code review application changes but few people review database changes!
Easy to build systems that commit your SQL into a repo, it gets
reviewed/approved, and then another system triggers running the actual
migration.
The automated piece can be handled by ChatOps, Jenkins, or others. This
piece can be manual to start with.

30. Databases & Analytics
Shift access control layer into an application
AbstractedReportingLayer
User
MySQL
Database
Postgres
Database
GroupUserPer
Role/Team
GroupUserPer
Role/Team
Easier to manage access through a unified application layer like Looker,
Metabase, Quicksight, and others than having to manage users in each
database.

31. Databases & Analytics
Audit all activity
Audit all you access, logs, and queries to your databases
Be able to answer who accessed what data when
You don’t need certain logs until the one day you do. Storage is cheap, store
the logs and have them available when needed. Easy option is to throw them
into S3 and use Athena to query.

32. Office IT

33. Office IT
Office VPN vs Individual VPN
Office VPN Individual VPN
If office network is compromised then access is gained to your
infrastructure (security cameras, network access ports, guest
networks, etc)
Laptops work with access to everything needed regardless of
location. Requires VPN connection while in the office.
Easier when no employees have laptops or are mobile. If mobile or laptops exist, you’ll need individual VPN anyways.
Managing just individual VPN is simpler and removes unnecessary
infrastructure.
Use multiple VPN servers to eliminate single point of failure at the
server (as well as location optimization)
Use multiple VPN servers to eliminate single point of failure at the
server (as well as location optimization)
If you have remote workers and laptops it’s easier to manage individual VPN. If you have dedicated workstations and no remote
workers, Office VPN may be a good fit.

34. Office IT
VPN vs oAuth
Consider using oAuth & whitelists (positive security) for enabling access to
resources
Handles encryption and authentication for you
Seeing an increase in adoption of this approach however it may not be
suitable for every situation

35. Office IT
Multi-Factor Authentication
There is no excuse to skip setting up MFA on your accounts. Audit users who
have access to your resources and make MFA a requirement.
Typically at least two of the following categories:
§ knowledge (something they know)
§ possession (something they have)
§ inherence (something they are)

36. Office IT
Multi-Factor Authentication
Authy
DuoMobile
YubiKey
SMS MFA
Gemalto Token

37. Office IT
Logon Notifications to Distribution Lists
Production
Server
User logs in
From: prod-18acf81-d1
To: engineers@mycompany.com
Cc: security@mycompany.com
Subject: Mackenzie logged into prod-18acf81-d1 from
54.239.17.7
Date: 2017-06-12 03:12:18am
Notification user Mackenzie logged into prod-18acf81-d1 from
54.239.17.7 at 2017-06-12 03:12:18am
Last login was on 2017-02-26 at 12:38:11pm
Production
Server
User logs in
Only alert between 7pm & 7am
AnyLoginTimeBased

38. Office IT
Wireless, WPA2, and 802.1x
Segment your guest network, and any external ports such as security cameras
or key access systems.
At the very least you should be using WPA2
You could go the extra step with 802.1x where each system has a certificate
issued which permits it onto the network regardless of wired or wireless.

39. Office IT
Web based interfaces
Thinclients
Reporting Tools
Call Center Systems
Email
Chat
Thin clients mean no data is
stored and at risk of being lost.
Web based makes managing
authentication easier (oAuth,
LDAP, etc)
No need to update applications on
the laptop itself. Everyone running
on the same versions.

40. Office IT
Phishing
One of the easiest ways people
gain access to unauthorized
systems.
Train all your employees from
day one about phishing attempts.
However, logon notifications and
monitoring are essential.

41. Office IT
Social Engineering
Understand how to mitigate common social engineering attacks:
• If someone calls saying they are from an organization, ask for a call back
number and search that number online. Ask the caller to verify information
about your account (recent transactions? number of registered users?
monthly spend? account reps name?)
• Do not plug free devices into your computer. USB drives, keyboards
received as gifts, USB plants, USB fans, and more.

42. Office IT
Security training on first day
Provide security training on day one. Topics to cover include:
- How to request access to systems
- Phishing attempts
- Data classification
- Identifying false browser messages and warnings
- Who and how to contact someone in the event of an emergency
- Most importantly: “Knowing it’s okay to ask others if you’re unsure if
something is safe.” Build that culture early on, and encourage people to
ask those around them for a second opinion.

43. Office IT
Access Requests
User requests
access to
system
Ticket created
in approval
queue
Manager signs
off and
approves
Process
triggers to
enable access
Ticketing systems make it easy to capture who is requesting access to
which systems.
Useful for understanding business justifications, auditing who gained access
to systems when, and what permissions similar new hires should have.

44. Trusted Advisor
Quickly identify security issues (and more!)

45. https://aws.amazon.com/security
https://d0.awsstatic.com/whitepapers/aws-security-whitepaper.pdf

46. Summary
1. Visibility
Aggregated Logs, Write Once Read Many
2. Least Permissions
Processes & Files, SSH Security, Immutable Design
3. Encryption
TLS vs IPsec, Luks, EBS, Object Stores, Laptops & Workstations
4. Keeping Systems Updated
Immutable Infrastructure, Alert on Old Systems
5. Application Security
Secure Build Pipeline, Security CI/CD, Layering
6. Office IT
VPN vs oAuth, MFA, Notifications, 802.1x, Web UIs, Phishing, Social
Engineering, Security Training, Access Requests
7. Database & Analytics
SQL Code Review, Application Layer Security, Audit Logs

47. Will St. Clair
wstclair@amazon.com