AWS Connectivity, VPC Design and Security Pro Tips

2. Agenda

3. Agenda
• AWS Connectivity
– Direct Connect

4. What isAWS Direct Connect…
• Dedicated, private pipes into AWS
• Create private (VPC) or public interfaces to AWS
• Cheaper data-out rates than Internet (data-in still
free)
• Consistent network performance compared to
Internet
• Multiple AWS accounts can share a connection

5. Why useAWS Direct Connect?
$0.000
$0.050
$0.100
$0.150
First 10TB
Next 40TB
Next
100TB Next
350TB
Direct Connect
Internet

6. Public Subnet
Availability Zone A
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
Instance A
10.1.1.11 /24
Instance C
10.1.3.33 /24
Instance B
10.1.2.22 /24
Instance D
10.1.4.44 /24
VPC CIDR: 10.1.0.0 /16
Virtual
Private
Gateway
(VGW)
Internet
Gateway
(IGW)
Only 1 IGW and 1 VGW
per VPC
VPN
connection
Customer
data center
Customer
data center
AWS Direct
Connect
Route Table
Destination Target
10.1.0.0/16 local
Internal CIDR VGW

7. Direct Connect – Single Link, Single CGW
VPC 1
10.1.0.0/16 AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
Service Provider
Network
` `
VGW
AWS
Network
DX POP
Location
Cross
Connect
Customer
Gateway
Router
Circuit to
Customers
Network
Customers
Network
Backbone
Circuit to
Customers
Site
Customer
Provider
Edge
Router Customers
Local
Network
Demarcation

8. Direct Connect – Single Link, Single CGW
VPC 1
10.1.0.0/16 AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
Service Provider
Network
` `
From - To
CGW to VGW
VGW to CGW
Route
172.16.0.0/16
10.1.0.0/16
Metric
-
-
eBGP
From - To
Customer to CGW
CGW to Customer
Route
172.16.0.0/16
10.1.0.0/16
Routing – Probably eBGP
Layer 2 VLAN Connectivity
BGP is a requirement for Direct Connect:
http://aws.amazon.com/directconnect/faqs/

9. VLAN Y
VLAN X
VIFs
virtual private cloud 1
virtual private cloud 2
virtual private cloud N
…
public endpoints
Region
Direct Connect Location
private VIF 1
public virtual interface (VIF)
private VIF 2
VLAN Z
VLAN N
AWS DX
Router
Customer
Router
Each interface can be
associated with a different
AWS Account. (Hosted
Virtual Interfaces)

10. Public Virtual Interfaces (VIFs)

12. Private Virtual Interfaces (VIFs)

14. Agenda
• AWS Connectivity
– VPN
– Design Patterns

15. Customer Network
Only 1 IGW and 1 VGW
per VPC

16. Customer Network

17. Customer Network

18. Agenda
• AWS Connectivity
– VPN
– Design Patterns

19. Direct Connect – Single Link, Single CGW
VPC 1
10.1.0.0/16 AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
Service Provider
Network
` `
With IPSEC Failover

20. Direct Connect – Single Link, Single CGW
VPC 1
10.1.0.0/16 AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
Service Provider
Network
``
IPSEC over
The Internet
From - To
CGW to VGW
VGW to CGW
Route
172.16.0.0/16
10.1.0.0/16
Metric
-
-
eBGP
With IPSEC Failover

21. Direct Connect – Dual Links, Single CGW
VPC 1
10.1.0.0/16 AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
Service Provider
Network
` `

22. Direct Connect – Dual Links, Single CGW
VPC 1
10.1.0.0/16
AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
Service Provider
Network
` `

23. Direct Connect – Dual Links, Single CGW
VPC 1
10.1.0.0/16
AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
Service Provider
Network
` `
From - To
CGW to VGW
VGW to CGW
Route
172.16.0.0/16
10.1.0.0/16
Metric
LP 150
eBGP
From - To
CGW to VGW
VGW to CGW
Route
172.16.0.0/17
10.1.0.0/16
Metric
LP 90
eBGP
- You can split your route advertisements to the VGW
- Instead of using AS Path Prepend
CGW to VGW172.16.128.0/1
7

24. Direct Connect – Dual Links, Dual CGW
VPC 1
10.1.0.0/16
AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
Service Provider
Network
` `

25. Direct Connect – Dual Links, Dual CGW
VPC 1
10.1.0.0/16
AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
`
Service Provider
Network
`

26. Direct Connect – Dual Links, Dual CGW
VPC 1
10.1.0.0/16
AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
``
`
Service Provider
Network

27. Direct Connect – Dual Links, Dual CGW
VPC 1
10.1.0.0/16
AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
`
`
Service Provider
Network
`

28. Direct Connect – Dual Links, Dual CGW
VPC 1
10.1.0.0/16
AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
`
`
Service Provider
Network
`
BGP AS -
Y
BGP AS -
X iBGP between RoutersiBGP between Routers

29. Direct Connect – Dual Links, Dual CGW
VPC 1
10.1.0.0/16
AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
`
`
Service Provider
Network
`
- So far so good?
- What’s wrong with this topology?
- SPoF!

30. Direct Connect – Dual Links, Dual CGW
VPC 1
10.1.0.0/16
AWS Direct
Connect
Customer
Gateway
Colocation
DX
Location
Customer
Edge Router
172.16.0.0/16
Customers DC
Customer
s Subnet
`
`
Service Provider
Network
`

31. Direct Connect – Dual Locations, Dual Links
VPC 1
10.1.0.0/16
AWS Direct
Connect
Customer
Gateway
Colocation
DX Location -
1
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
`
`
Service Provider
Network
`
AWS Direct
Connect
Customer
Gateway
Colocation
`
DX Location -
2

32. VPC 1
10.1.0.0/16
AWS Direct
Connect
Customer
Gateway
Colocation
DX Location -
1
Customer
Edge Router
172.16.0.0/16
Customers DC
Customers
Subnet
`
`
`
AWS Direct
Connect
Customer
Gateway
Colocation
`
DX Location -
2
DirectConnect – Dual Locations, Dual Links, Dual
Routers
Service Provider
Network

33. Multi Account DX
VPC 1
10.1.0.0/16
Customer
Gateway
VLAN320Private VI
AWS Direct
Connect
Ethernet
Trunk
SVI/Sub 320
IP 169.x.x.2IP 169.x.x.1
BGP AS 65xxxBGP AS 17493
VLAN 320
Colocation

34. Multi-Account Direct Connect
Customer
Gateway
VLAN320
Ethernet
Trunk
SVI/Sub 320
IP 169.x.x.2
BGP AS 65xxx
VPC 1
10.1.0.0/16
Private VI
IP 169.x.x.1
BGP AS 17493
VLAN 320
Colocation
VPC 2
10.2.0.0/16 IP 169.y.y.1
BGP AS 17493
VLAN 330
VLAN330
SVI/Sub 330
IP 169.y.y.2
BGP AS 65xxx
Private VI
AWS Direct
Connect
AWS Account 1

35. Multi-Account Direct Connect
Customer
Gateway
VLAN320
Ethernet
Trunk
SVI/Sub 320
IP 169.x.x.2
BGP AS 65xxx
VPC 1
10.1.0.0/16
Private VI
IP 169.x.x.1
BGP AS 17493
VLAN 320
Colocation
VPC 2
10.2.0.0/16 IP 169.y.y.1
BGP AS 17493
VLAN 330
VLAN330
SVI/Sub 330
IP 169.y.y.2
BGP AS 65xxx
Private VI
AWS Direct
Connect
AWS Account 1
AWS Account 2

36. How to Delegate VI to Another Account.
Step 1.

37. Delegate Virtual Interface to Another Account.
Step 2.

38. Agenda
– VPN
– Design Patterns
• VPC Design
– Concepts

39. AvailabilityZoneA
AvailabilityZoneB

40. AvailabilityZoneA
AvailabilityZoneB

41. VPC A - 10.0.0.0/16
AvailabilityZoneA
AvailabilityZoneB
Choose your VPC address
range
• Your own private, isolated
section of the AWS cloud
• Every VPC has a private IP
address space
• That maximum CIDR block
you can allocate is /16
• For example 10.0.0.0/16 –
this allows 256*256 =
65,536 IP addresses
Select IP addressing strategy
• You can’t change the VPC
address space once it’s
created
• Think about overlaps with
other VPCs or existing
corporate networks
• Don’t waste address space,
but don’t’ constrain your
growth either

42. VPC A - 10.0.0.0/16
AvailabilityZoneA

43. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
10.0.3.0/24 10.0.5.0/2410.0.4.0/24

44. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
10.0.3.0/24
EC
2
10.0.5.0/2410.0.4.0/24
EC
2
Ap
p
Log
EC
2
Web
Bastio
n

45. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
10.0.3.0/24
EC
2
10.0.5.0/2410.0.4.0/24
EC
2
App
“Web servers can connect to
app servers on port 8080”
Log
EC
2
Web
Bastion

46. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
10.0.3.0/24
EC
2
10.0.5.0/2410.0.4.0/24
EC
2
Ap
p
“Web servers can connect to
app servers on port 8080”
“Allow
outbound
connections to
the log server”
Log
EC
2
Web
Bastio
n

47. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
10.0.3.0/24
EC
2
10.0.5.0/24
Bastio
n
10.0.4.0/24
EC
2
Ap
p
“Web servers can connect to
app servers on port 8080”
“Allow
outbound
connections to
the log server”
“Allow SSH and
ICMP from
instances in the
Bastion security
group”
Log
EC
2
Web

48. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
10.0.3.0/24
EC
2
10.0.5.0/2410.0.4.0/24
EC
2
Ap
p
Log
EC
2
Web
Security groups
• Operate at the instance level
• Supports ALLOW rules only
• Are stateful
• Max 50 rules per security group
• Max 5 groups per instance
Bastio
n

49. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
10.0.3.0/24
EC
2
Router
10.0.5.0/2410.0.4.0/24
EC
2
Ap
p
Log
EC
2
Web
Bastio
n

50. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
10.0.3.0/24
EC
2
Router
10.0.5.0/2410.0.4.0/24
EC
2
Ap
p
Log
EC
2
Web
“Deny all traffic between the
web server subnet and the
database server subnet”
Bastio
n

51. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
10.0.3.0/24
EC
2
Router
10.0.5.0/2410.0.4.0/24
EC
2
Ap
p
Log
EC
2
Web
Bastio
n
NACLs are optional
• Applied at subnet level
• Stateless and permit all by default
• ALLOW and DENY
• Applies to all instances in the subnet
• Use as guard rails (port 21, 135,…)

52. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
10.0.3.0/24
EC
2
Router
10.0.5.0/2410.0.4.0/24
EC
2
Ap
p
Log
EC
2
WebEC
2
Web
Elastic Load
Balancer
Bastio
n

53. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
10.0.3.0/24
EC
2
Router
10.0.5.0/2410.0.4.0/24
EC
2
Ap
p
Log
EC
2
WebEC
2
Web
Elastic Load
Balancer
Bastio
n

54. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
EC
2
10.0.3.0/24
EC
2
Router
10.0.5.0/2410.0.4.0/24
EC
2
Ap
p
Log
EC
2
WebEC
2
WebEC
2
EC
2
Web
Elastic load balancers
• Instances can automatically be
added and removed from the
balancing pool using rules
• You can add instances into
security groups at launch time
Elastic Load
Balancer
Auto
scalin
g
Bastio
n

55. VPC A - 10.0.0.0/16
AvailabilityZoneA
EC
2
EC
2
EC
2
Ap
p
EC
2
WebEC
2
WebEC
2
EC
2
Web
Internet Gateway
VPC
Router

56. VPC A - 10.0.0.0/16
AvailabilityZoneA
EC
2
EC
2
EC
2
Ap
p
EC
2
WebEC
2
WebEC
2
EC
2
Web
Internet Gateway
VPC
Router
Internet routing
• Add route tables to subnets to
control Internet traffic flows –
these become Public subnets
• Internet Gateway routing
allows you to allocate a static
Elastic IP address or use
AWS-managed public IP
addresses to your instance

57. VPC A - 10.0.0.0/16
AvailabilityZoneA
EC
2
EC
2
EC
2
Ap
p
EC
2
WebEC
2
WebEC
2
EC
2
Web
Internet Gateway
VPC
Router
Internet routing
• Use a NAT instance to
provide Internet connectivity
for private subnets - required
to access AWS update
repositories
• This will also allow back-end
servers to route to AWS APIs
– for example storing logs on
S3, or using Dynamo, SQS,
SNS and SWS
NA
T

58. VPC A - 10.0.0.0/16
AvailabilityZoneA
10.0.1.0/24
10.0.2.0/24
EC
2
10.0.3.0/24
EC
2
NA
T
10.0.4.0/24
EC
2
Ap
p
EC
2
WebEC
2
WebEC
2
EC
2
Web VPC
Router
Amazon S3
DynamoDB
Amazon SNS
Amazon SQS
Internet Gateway
NA
T

59. Agenda
– VPN
– Design Patterns
• VPC Design
– Concepts
– Design Patterns

60. To NACL or not to NACL?
Pros
Another layer of defense
Can speed up deals
• Fits legacy IT models
• Network/FW Engineer’s friend
Can help with networking compliance
• Separate groups for SGs/NALCs
Explicit deny rules
Apply to an entire subnet
Cons
Adds complexity
Can slow down adoption
• Fits legacy IT processes
• DevOps Enemy
Potentially not necessary for
compliance
• Third-party proactive controls
• SG audits (programmable infra)
Stateless FW rules
Apply only to subnets/CIDR
addresses

61. NACL Best Practices

62. Routing Instances
Love Them
NAT instances
VPN tunnels (between VPCs)
Data loss prevention
Intrusion detection
Hate Them
Single point of failure
Extra costs (EC2, third-party licenses)
More for customer to manage
Potential network bottleneck

63. Routing Instance Best Practices

64. AWS region
Public-facing
web app
Internal
company
app
What’s next?
VPN
connection
Customer data center
Multiple VPCs

65. Multiple VPCs use case

66. Multiple VPCs tips and tricks

67. Public-facing
web app
Internal
company
app #2
HA pair VPN
endpointsCustomer data center
Internal
company
app #3
Internal
company
app #4
Internal
company
app #1
Internal
company
Dev
Internal
company
QA
AWS region
BackupAD, DNS Monitoring
Logging
Multiple VPCs over IPSEC VPN

68. About IPSEC and multiple VPCs

69. Public-facing
web app
Internal
company
app #2
HA pair VPN
endpointsCustomer data center
Internal
company
app #3
Internal
company
app #4
Internal
company
app #1
Internal
company
Dev
Internal
company
QA
AWS region
BackupAD, DNS Monitoring
Logging
Multiple VPCs over AWS Direct Connect
Direct
Connect
Facility
Customer
Data Center
Physical
Connection
Logical
Connections
VLANs
Logical
Connections
VLANs

70. About AWS Direct Connect and multiple VPCs

71. • Security groups and NACLs still
apply
AWS
region
Public-facing
web app
Internal
company
app #1
HA pair VPN
endpoints
company data center
Internal
company
app #2
Internal
company
app #3
Internal
company
app #4
Services
VPC
Internal
company
Dev
Internal
company
QA
AD, DNS
Monitoring
Logging
• Security groups still bound to
single VPC
Multiple VPCs over VPC Peering

72. VPC peering use cases

73. 10.1.0.0/16
10.0.0.0/16
• VPCs within same region
Peer
Request
Peer
Accept
• Same or different accounts
• IP space cannot overlap
• Only 1 between any 2 VPCs
VPC peering configuration

74. 10.1.0.0/16
10.0.0.0/16 10.0.0.0/16
✔
Overlapping IP is not a dead end

75. 10.0.0.0/16 10.0.0.0/16
PCX-1 PCX-2
Subnet 1
10.1.1.0/24
Subnet 2
10.1.2.0/24
10.1.0.0/16
Route Table Subnet 1
Destination Target
10.1.0.0/16 local
10.0.0.0/16 PCX-1
Route Table Subnet 2
Destination Target
10.1.0.0/16 local
10.0.0.0/16 PCX-2
A
B C

76. 10.1.0.0/16
10.4.0.0/16 10.0.0.0/16
10.3.0.0/16
172.16.0.0/16
192.168.0.0/16
10.2.0.0/16
172.17.0.0/16
CA

77. 10.1.0.0/16
10.4.0.0/16 10.0.0.0/16
10.3.0.0/16
172.16.0.0/16
192.168.0.0/16
10.2.0.0/16
172.17.0.0/16
company data center
10.10.0.0/16

78. 10.1.0.0/16
10.4.0.0/16 10.0.0.0/16
10.3.0.0/16
172.16.0.0/16
192.168.0.0/16
10.2.0.0/16
172.17.0.0/16
company data center
10.10.0.0/16

79. 10.4.0.0/16 10.0.0.0/16
172.16.0.0/16
192.168.0.0/16
172.17.0.0/16
10.1.0.0/16 10.2.0.0/1610.3.0.0/16

80. Peer review
• Shared infrastructure services
moved to VPC
• 1 to 1 peering = app isolation
• Security groups and NACLs still
apply
AWS
region
Public-facing
web app
Internal
company
app #1
HA pair VPN
endpoints
company data center
Internal
company
app #2
Internal
company
app #3
Internal
company
app #4
Services
VPC
Internal
company
Dev
Internal
company
QA
AD, DNS
Monitoring
Logging
• Security groups still bound to
single VPC
Multiple accounts

81. About VPC peering and multiple VPCs

82. Model 1: “Lollipop”

83. Model 2: “Shared Services Model”

84. Model 3: “HIPS Model”

85. Scenario #4 – “Threat Layer Model”

86. Model 5: “NIDS Model”

87. Model 6: “Hybrid Model”

88. S3 VPC endpoint

89. Agenda
– VPN
– Design Patterns
• VPC Design
– Concepts
– Design Patterns
• Security Pro Tips

90. MFA

91. IAM Best Practices

92. AWS Trusted Advisor

93. CloudTrail – Log & monitor these!
• API actions with potential impact
– Internet Gateway
– Routes and Route Tables
– Network ACLs
– EC2 instances (run/create/launch/terminate)
– Security Groups
– CloudTrail (stop/delete/update)
– Put[Group/Role/User]Policy
– ModifyAccount
– ModifyBilling, ModifyPaymentMethods
– "Type":"Root"
– Create[User/Role/Group]
– CreateAccessKey

94. Continuous Change
Recording
Changing
Resources
AWS Config
History
Stream
Snapshot (ex. 2014-11-05)
AWS Config

95. ENCRYPT YOUR SENSITIVE DATA

96. segregate duties
With
AWS
IAM
you
get
to
control
who
can
do
what
in
your
AWS
environment
and
from
where
Fine-­‐grained
control
of
your
AWS
cloud
with
two-­‐
factor
authen;ca;on
Integrated
with
your
exis;ng
corporate
directory
using
SAML
2.0
and
single
sign-­‐on
AWS account
owner
Network
management
Security
management
Server
management
Storage
management

97. DDoS Protection
Inbound HTTP
CloudFront
Amazon S3
WAFDynamic
App
App
AppPeering
DDoS
users

98. Instance patching and upgrades

99. VPC Flow Logs

100. Discussion…