In this session, learn how you evaluate, design, build, and manage distributed applications over hybrid infrastructures using Amazon Web Services. This session follows the evolution of a simple legacy data center expansion with
basic connectivity into managing complex hybrid applications. Along the way, we investigate best practice designs in use by AWS customers. Topics covered include: interconnectivity, availability, security, hybrid networks with Amazon VPC and AWS Direct Connect as well as automated provisioning with AWS CloudFormation, and configuration management with AWS OpsWorks.
Speakers:
Miha Kralj, AWS Solutions Architect
Amarpal S. Attwal, Senior Technical Lead, ICT Engineering, Just Eat
Koen van den Biggelaar, AWS Solutions Architect
2. Our journey today
VPC
VPN
Backup &
archive
Storage
expansion
AWS Direct
Connect
AuthenKcaKon
FederaKon
OperaKons Tools and
Monitoring
Start
What is
Hybrid
IntegraKon?
Integrated
Infrastructure
Integrated
Services
Integrated
PlaTorm
Integrated
SoluKon
CI/CD
Managed AWS
Services
MigraKon
Roadmap
3. “Consumption of Cloud Services and On-Premises IT into a combined pool of resources.”
Defining Hybrid Integration
On-
premises
IT Services
Platform
Solutions
Cloud
Services
Infrastructure
Benefits:
• Cost Efficiencies
• Scalability
• Flexibility
• Security
5. AWS Virtual Private Network (IPSec VPN)
o IPSec hardware VPN connection
Supported VPN appliances:
https://aws.amazon.com/vpc/faqs/#C9
o Encryption and Validation
o Private RFC 1918 Addressing
o Uses Border Gateway Protocol (BGP)
for routing and fail-over
o VPN Service provides managed
redundant end-points
http://docs.aws.amazon.com/AmazonVPC/latest/
UserGuide/VPC_VPN.html
Virtual
Gateway
Corporate
data center
Users
Data center router
Servers
Internet
IPSec VPN
VPC Subnet
Availability Zone
Security Group
VPC Subnet
Availability Zone
Security Group
6. AWS Direct Connect
o Requires Layer 2 single mode fiber
1000BASE-LX or 10GBASE-LR
o Requires 802.1Q VLANs across
connection.
Ø Tagging of IP traffic
o Routing uses BGP A/A or A/P
multipath.
o Each DX is mapped to a single AWS
Region
o Various Partners for every Region
http://aws.amazon.com/directconnect/
Virtual
Gateway
Corporate
data center
Users
Data center router
Servers
VPC Subnet
Availability Zone
Security Group
VPC Subnet
Availability Zone
Security Group
Customer
router
AWS Direct Connect
LocaKon
AWS Direct Connect
routers
7. Customer
router
AWS Direct Connect
LocaKon
AWS Direct Connect
routers
AWS Direct Connect + AWS VPN
o Dedicated network path with assured
bandwidth
o More secure than Internet-based IPSec
VPN – avoids internet traverse
o Reduced IPSec network transfer costs
o Additional Network Security
http://aws.amazon.com/directconnect/
Virtual
Gateway
Corporate
data center
Users
Data center router
Servers
VPC Subnet
Availability Zone
Security Group
VPC Subnet
Availability Zone
Security Group
IPSec VPN
9. AWS Direct Connect
LocaKon
AWS Direct Connect
routers
Active Directory and LDAP
o Reduced back-reach Traffic
o Reduced Latency for Authentication
o Additional Resiliency
o Enablement of both:
Ø Multi-Master Read/Write Domain
Controllers
Ø Read-only Domain Controllers (RODCs)
² Requires IPSec VPN or Direct Connect
connectivity
http://aws.amazon.com/microsoft/whitepapers/ad-reference-
architecture/
Virtual
Gateway
Corporate
data center
Users
Data center router
Servers
VPC Subnet
Availability Zone
Security Groups
VPC Subnet
Availability Zone
Security Groups
Type
Port Number
TCP
54, 88, 135, 137, 139, 389, 445, 464, 636, 3268,
3269, 5722, 49152-‐65535
UDP
53,67,123, 138, 389, 445, 464, 2535, 5355,
49152-‐65535
AD.Domain
Domain
controller
Domain
controller
Domain
controller
AcKve Directory
ReplicaKon
Customer
router
10. AWS Direct Connect
LocaKon
AWS Direct Connect
routers
AWS Directory Service
o Deploys in two modes
Ø Directory Service Connect
Ø Simple AD - built on Samba 4 Active
Directory compatible server
o Simplifies IAM Federation
Ø Avoids complexity and cost of hosting
SAML-based federation infrastructure
Ø Acts as a proxy - no data is stored on
AWS infrastructure
Ø Supports existing RADIUS-based MFA
² Requires IPSec VPN or Direct Connect
connectivity
http://aws.amazon.com/directoryservice/
Virtual
Gateway
data center
Users
Data center router
Servers
VPC Subnet
Availability Zone
Security Groups
VPC Subnet
Availability Zone
Security Groups
AD.Domain
Domain
controller
AD Connector
AD Connector
AD Connector
Customer
router
11. AWS Federation/Account Governance
Financial users,
controllers
SOC/Auditors
Global AWS admin
Billing account
Socware development
Non-‐prod
account #1
ProducKon
account #1
User management
account
Security / Audit
account
Non-‐prod
account. #2
App owners
DevOps teams
Security/audit
ProducKon
Dev/test/sandbox
Financial
Consolidated Billing,
Billing Alerts
Read-‐only access
for all accounts
12. AWS Direct Connect
LocaKon
AWS Direct Connect
routers
Operations Tools and Monitoring
o Security Monitoring integration
points with with CloudTrail and
SIEM Aggregator.
o Logging with CloudTrail and SNMP
MIBs to SIEM Aggregator.
o Platform and App Health to SIEM
Aggregator via agent on EC2 guest.
o Access to Patching and Updates for
AMI by on premises Update Server.
Virtual
Gateway
data center
Users
Data center router
VPC Subnet
Availability Zone
Security Group
VPC Subnet
Availability Zone
Security Group
Update
Servers
SIEM
Aggregator
CloudTrail
CloudWatch
CloudTrail S3
Bucket
Customer
router
14. Application Deployment Management
Apache
Tomcat
Struts
Your Code
Log4J
Spring
Hibernate
JEE
Linux
Java App Stack
Inventory of AMIs
Apache
Tomcat
Struts
Your Code
Log4J
Spring
Hibernate
JEE
Linux
Java AMI
Amazon EC2
Apache
Tomcat
Struts
Your Code
Log4J
Spring
Hibernate
JEE
Linux
Apache
Tomcat
Struts
Your Code
Log4J
Spring
Hibernate
JEE
Linux
Apache
Tomcat
Struts
Your Code
Log4J
Spring
Hibernate
JEE
Linux
Apache
Tomcat
Struts
Your Code
Log4J
Spring
Hibernate
JEE
Linux
Golden AMI +
Fetch binaries on boot
Apache
Tomcat
Hibernate
JEE
Linux
Java AMI
Amazon EC2
Struts
Spring
Log4J
Your Code Fetch on boot
Fetch on boot
From S3
Apache
Tomcat
Hibernate
JEE
Linux
Apache
Tomcat
Hibernate
JEE
Linux
Apache
Tomcat
Hibernate
JEE
Linux
JeOS AMI and Library of
recipes (install scripts)
JeOS AMI
Amazon EC2
JEE
Linux
CHEF
Struts
Spring
Log4J
Apache
Tomcat
Your Code Fetch on boot
CHEF
recipes
JEE
Linux
CHEF
JEE
Linux
CHEF
JEE
Linux
CHEF
JEE
Linux
CHEF
15. AWS
Elas)c
Beanstalk
Automated
resource
management
–
web
apps
made
easy
AWS
OpsWorks
DevOps
framework
for
applica;on
lifecycle
management
and
automa;on
DIY
/
On
Demand
DIY,
on
demand
resources:
EC2,
S3,
custom
AMI’s,
etc.
Convenience Control
AWS
CloudForma)on
Templates
to
deploy
&
update
infrastructure
as
code
Deployment and Management
16. Customer
router
AWS Direct Connect
LocaKon
AWS Direct Connect
routers
Continuous Integration and Deployment
o Automates application deployments
for both On-Premise and AWS EC2
instances with use of CodeDeploy
o Reuse existing scripts and tools
Ø Bash, PowerShell, Chef,
Puppet, anything…
o Integrate with developer tool chain
Ø GitHub, Jenkins, CloudBees,
TravisCI, Eclipse…
Virtual
Gateway
data center
Users
Data center router
VPC Subnet
Availability Zone
Security Group
VPC Subnet
Availability Zone
Security Group
AWS CodeDeploy
Servers
AWS CloudFormaKon
S3 bucket
Agent
Agent
Agent
Agent
Agent
Agent
17. Customer
router
AWS Direct Connect
LocaKon
AWS Direct Connect
routers
Managed AWS Services
o Managed Services Advantages
Ø Flexibility and Agility
Ø Scalability
Ø Security
Ø Automated Maintenance & Upgrade
Virtual
Gateway
data center
Users
Data center router
VPC Subnet
Availability Zone
Security Group
VPC Subnet
Availability Zone
Security Group
Servers
S3 bucket
MySQL MySQL
Apache
Kaga
Amazon Redshic
Amazon EMR
Amazon Redshic
Amazon EMR
19. Customer
router
AWS Direct Connect
LocaKon
AWS Direct Connect
routers
Storage expansion
o Virtual volumes presented to local
network iSCSI, NFS and CIFS
volumes
o Local disk cache to provide fast on-
premises access
o Gateway side encryption for security
Virtual
Gateway
Corporate
data center
Users
Data center router
VPC Subnet
Availability Zone
Security Group
VPC Subnet
Availability Zone
Security Group
Amazon S3
AWS Storage
Gateway
iSCSI
Storage
Appliance
AWS Storage
Gateway
iSCSI
Servers
AWS Storage
Gateway
Cloud ONTAP Secure Cloud-‐
Integrated Backup
Panzura Global NAS
TwinStrata CloudArray
AWS Marketplace Partners
20. Customer
router
AWS Direct Connect
LocaKon
AWS Direct Connect
routers
Backup and archiving
o Backup gateways integrated with
Amazon S3
o Leverage Amazon S3 archival
to Amazon Glacier
o Take advantage of current
investments and solutions for options
o De-duplication
o Compression
o WAN Acceleration
Virtual
Gateway
data center
Users
Data center router
VPC Subnet
Availability Zone
Security Group
VPC Subnet
Availability Zone
Security Group
Amazon S3
Amazon Glacier
VTL
AWS Storage
Gateway
iSCSI
Backup
System
VTL
AWS Storage
Gateway
iSCSI
Servers
VTL
AWS Storage
Gateway
Symantec Net Backup
Veeam Backup & ReplicaKon
Cloud ONTAP Secure Cloud-‐
Integrated Backup
AWS Marketplace Partners
23. Cloud Adoption Framework
The AWS CAF organizes and
describes the perspectives in
planning, creating, managing, and
supporting a modern IT service.
Offers practical guidance and
comprehensive guidelines for
establishing, developing and running
AWS cloud-enabled environments.
It provides a structure where
business and IT can work together
towards common strategy and vision,
supported by modern IT automation
and process optimization.
http://bit.ly/AWSCAF
People
Perspective
Process
Perspective
Security
Perspective
Maturity
Perspective
Operations
Perspective
Business
Perspective
Platform
Perspective
25. JUST EAT plc (incorporated in the UK) is proud to be the world’s
leading online takeaway ordering service.
We allow hungry local consumers to order in real-time from their
local independent takeaway restaurants via a single online
portal.
• Tech team is ~150 people, 3 sites.
• Windows+.NET platform, cloud native in AWS.
• Very predictable load, ~1200 orders/min peak in UK
• Recruiting!
JUST EAT
26. Our Journey and Challenges
Hybrid
plaTorm
TradiKonal
plaTorm and
infrastructure
Change our
approach
Architect and
build
Decommission
legacy
Enterprise
plaTorm v2.0
On premise
• Physical servers
• Hypervisors
• ConnecKvity
• SANs
• Backup and Tape
• Etc…
• Flexible
• AutomaKon
• Time to deploy
• Centralise
• OpKmise costs
• Fail fast!
• ConnecKvity
• Security
• Not lic and shic
• Decoupling
• Data is core
• Disposable
Infrastructure
• Throw it away!
27. Connectivity and traffic flow
Customer
router
AWS Direct Connect
LocaKon
AWS Direct Connect routers
Virtual
Gateway
Corporate
data center
Users
Data center router
Server
VPC Subnet
Availability Zone
Security Group
VPC Subnet
Availability Zone
Security Group
IPSec VPN
28. Example – Active Directory
AWS CloudFormaKon
Unajend
DCPromo
Build vanilla server
*Add in security
group for DC Ports
Domain
Prep
Manual – run
unajend file
DC Dies
Domain
Cleanup
Repeat
29. Example – Critical Application
Start
S3 bucket
AWS CloudFormaKon
S3 bucket
AWS CloudFormaKon
Script Library
Design – How to build
Push data – ref
CF
Build and store build config
Use build config to
rebuild in failure
30. Outcomes
• Core data stored securely and reliably
• Centralised connectivity
• Disposable infrastructure
• Built-in flexibility (Elasticity)
• Consistent and automated builds
• Library of reusable scripts
• Cross charging of services to business units
• Continuous BC & DR
• Less time maintaining – More time INNOVATING
31. JustEat - Lessons learnt
• Planning is everything
• Be prepared for a steep learning curve
• Give yourself plenty of time
• Simplicity is key
32. AWS Marketplace software
• Launch software on AWS with
1-click
• Pay-by-the-hour, monthly, or
annual
• Single invoice for AWS usage &
software
33. Takeaways
• Connectivity is a key to a successful hybrid integration between cloud and
corporate data center
• Authentication and Authorization is the corner stone of Enterprise Integration
• Hybrid infrastructure enables a variety of hybrid workload implementations
• Application migration is just a piece of large-scale Cloud Adoption
– The Cloud Adoption Framework whitepaper: http://bit.ly/AWSCAF