The document discusses Vanguard's move from a mainframe-based architecture to microservices in the cloud. It describes Vanguard's initial complex IT environment with monolithic applications and a mainframe. Vanguard's approach was to replicate data from the mainframe to the cloud, refactor applications to make API calls to microservices, and migrate batch processes. This "strangulation strategy" allowed the monolith to be gradually replaced. The document outlines Vanguard's cloud data architecture and how it leveraged AWS services like RDS, DynamoDB, Lambda and Kinesis while addressing compliance and operational requirements. Lessons learned included preparing for regulatory needs and pushback to cloud migration.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
From Mainframe to Microservices:
Vanguard’s Move to the Cloud
I l y a E p s h t e y n , A W S S o l u t i o n s A r c h i t e c t
B a r r y S h e w a r d , V a n g u a r d C h i e f E n t e r p r i s e A r c h i t e c t
E N T 3 3 1
N o v e m b e r 3 0 , 2 0 1 7

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What to expect from this session
Mainframe modernization approaches
Vanguard’s cloud data architecture and mainframe strangulation strategy
Benefits and lessons learned

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Characteristics of microservices architecture
Reusable modules of code that are built and deployed as
independent entities
Organized around business capabilities
Own their domain logic
Decentralized governance and data management
Automation and designed for failure

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Microservices benefits
Removes business and data logic from applications
Helps customers reduce technical debt
Eliminates monolithic bottlenecks
Improves developer velocity

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
monolithic application
+
monolithic teams
But how do you get there from here…

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Re-host—emulator-based approach
• Reengineer
• Refactor (automated)—automatic code conversion and refactoring
• Rewrite (manual)—re-developed as microservices
• Batch job migration—cloud-native optimized batch processing
• Data analytics enablement—cloud-native data lake and analytics
• Data-driven augmentation—cloud-native data-based services
• Re-platform—Java and Linux migration
• Repurchase—code replaced with new packaged solutions
• Retire—code abandoned while data is migrated or archived
• Combination of the above
Mainframe modernization approaches

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Modernization speed to business value
Cloud
business
value
6 months 1 year 2 years 3 years 4 years 5 years Time
Rewrite
Refactor
Re-host
Repurchase

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Martin Fowler’s Strangler Pattern
“…gradually create a new
system around the edges of
the old, letting it grow slowly
over several years until the old
system is strangled.”
Martin Fowler
June 29, 2004

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
No New Features on the Monolith
Creating a new system around the edges
Monolith
Browser
DB
Proxy
Browser
Service
DB
DB
Monolith
Proxy
Browser
Service
DB
Service
DB
Service
DB
DB
Monolith

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
API
gateway
Browser
Service
DB
Service
DB
Service
DB
Monolith
DB
New system may not always lead to
strangulation

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
API
gateway
Browser
Service
DB
Service
DB
Service
DB
Monolith
DB
A better approach: Focus on the data

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vanguard’s Journey from
Mainframe to Microservices

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vanguard is one of the world's largest investment
companies, offering a large selection of low-cost
mutual funds, ETFs, advice, and related services
Core purpose—To take a stand for all investors, to
treat them fairly, and to give them the best chance
for investment success
Oldest fund—Wellington Fund (inception 1929)
Began Operations—May 1, 1975 in Valley Forge, PA
Funds—Over 180 U.S. funds (including variable
annuity portfolios) and 190 additional funds in
markets outside the United States
Vanguard—Background

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vanguard’s IT Environment
• Multiple data centers plus disaster
recovery datacenters
o Huge reserves to handle spikes
• Custom web applications
o Monolithic Java applications
• Large and complex
o Stateful, requiring sticky sessions
o Use internal non-distributed cache
o Largest Lines of Business (LOBs) use
data mainly in DB2 on the mainframe
o Other LoBs use other databases
Complex and Inflexible
Mainframe
CacheCache
LoB#1LoB#2LoB#3
Web Tier App Tier DB Tier

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• z/OS (MVS)
• Build system for COBOL
• Data in DB2 supports online systems
o 3,000+ relational tables
o 6,000+ COBOL stored procedures
• Record-keeping systems
o VSAM files
o CICS interfaces
o COBOL batch processes
• Integration logic
o MQ
o COBOL
Vanguard’s Mainframe Environment
Complex and Inflexible
Mainframe
DB2
Record Keeping
Integration
LogicMQ
CICS
COBOL
SPs

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
From Legacy Web Applications to Microservices
• Microservices solve monolith problem
o Stateless—horizontally scalable
o Strict bounded context via network
o Simplifies continuous deployment (CD)
o Enables polyglot—multiple types of data
store or multiple programming languages.
To the Public Cloud
• Supports Infrastructure as Code (IaC)
• Enables managed services
• Makes elasticity viable
Off the Mainframe
• Mainframe prevents pure public cloud
• Cost (hardware and software)—horizontal scaling
• Desire for common programming model
• Skills availability
• Continuous deployment through to data layer
Why migrate?

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Next-Gen Applications (NGA)
MONOLITHIC
APPS
CLEAN,
MODULAR
CODE
APPS &
SERVICES
CLOUD
CONTINUOUS
DELIVERY /
DEVOPS
TRUE AGILE /
LEAN IT
NEXT GEN
APPS
Isolate
Data
Layer
Bounded
Contexts Elasticity
Simple and Flexible

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Low latency Compliant Cost optimized
Secure Multi-region geo distributed Autonomous within region
NGA in the Public Cloud
Simple and Flexible

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Initial Approach—Next-Gen Data
• Met requirements
• Hub and spoke
• Cloud-hosted spokes
• Incorporated past decisions
• Object relational mapping
• Business events
• Two-phase commit
• Asynchronous
• Proprietary hardware
• No managed platforms

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Simplified Cloud Data Architecture
• Supports:
o CDC to DB
o Read from DB
o Direct write
• Easy improvements:
o DB HA
• Multi-AZ
o DB scalability
Option 1
μservice
CDC
CDC
μservice
ExtendedHub

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CDA—RDS with Buffered Writes Option
Buffered Writes
+ User-perceived
latency
+ Availability
+ Disaster impact
+ Smoothens spikes
- No single integration
point
- Asynchronous writes
Option 3
μservice
CDC
CDC
μservice
Replicator
Dispatcher
DB Writers
ExtendedHub

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• Three separate AWS Kinesis streams:
o To-Do
o Done
o Error
• Helps support synchronous-like operations
• Provides full situational awareness
• Kitty Hawk
o Deals with in-flight messages during
write operations!
Buffered Write Deep Dive
μservice
μservice
Replicator
Dispatcher
DB Writers
Replicators
Kitty Hawk
μservice
Kitty Hawk
μservice
To-Do
Stream
Error
Stream
Done
Stream

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
CDA—NoSQL Support Option
Features:
+ NoSQL data store
+ Relational to NoSQL
mapping
+ Single point of
integration
- Slightly higher latency
- Complexity re-
introduced
Option 6
μservice
CDC
CDC
μservice
Replicator
Dispatcher
DB Writers
Event Writers
ExtendedHub

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cloud Data Architecture—Final
Features:
+ RDBMS databases
+ NoSQL DynamoDB
databases
+ Single point of
integration
+ Meets availability
requirements
- Complexity re-
introduced
μservice
CDC
CDC
μservice
Replicator
Dispatcher
DB Writers
Event Writers
Big
Data
ExtendedHub

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Mainframe Strangulation Strategy
• Replicate data to the cloud
• Refactor the monolithic apps so web pages make
AJAX calls to microservices
• Migrate batch processes to the cloud, using data
in the cloud
• Gold copy in cloud—Reverse replication
• Treat mainframe record keeping systems as
bounded contexts
o Integrate with them from the cloud

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Relational Database Service (Amazon RDS)
SOC2
Compliance ✓
Data at Rest
Encryption ✓
User Access
Management ✗ Home-grown IDM
interface
Data Activity
Monitoring ✗ Home-grown DB log
monitoring

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon DynamoDB
SOC2
Compliance ✓
Data at Rest
Encryption ✗ Client-side encryption
User Access
Management ✓
Data Activity
Monitoring
✗
(Write-
Only)
DynamoDB Streams
Client-side encryption
+ KMS & CloudTrail

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Lambda
SOC2
Compliance ✓ App selection
AWS engagement
Data at Rest
Encryption n/a
User Access
Management ✓
Data Activity
Monitoring n/a

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Kinesis
SOC2
Compliance ✓ App selection
AWS engagement
Data at Rest
Encryption ✓
User Access
Management ✓
Data Activity
Monitoring ✗ Client-side encryption
+ KMS & CloudTrail

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Microservices
+ Continuous deployment (CD) pipeline means:
o Pull request model forces peer review
o Quality gates
o Rapid feedback
+ Microservices principles means:
o Strictly enforced bounded contexts
o Stateless
- Eventual consistency
Other
+ Single development model
+ Polyglot—data stores, language
• Compliance
Cloud
+ Enables infrastructure as code (IaC)
+ Continuous deployment pipeline means:
+ Pull request model forces peer review
+ Quality gates
+ Rapid feedback
+ Managed services allow focus on the business
problem, not the infrastructure
- Eventual consistency / latency
Impacts of Migration

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Regulatory
• Be aware of regulatory compliance needs,
particularly scope, and process for risk acceptance
rather than risk avoidance.
• Understand data classifications for different
bounded contexts. Data with certain needs can be
treated differently, for example integrity rather than
confidentiality.
• Have a backup plan in case a primary choice isn’t
available due to compliance or functionality delivery
issues. Often a self-managed solution is available.
Acceptance
• Publicize the architecture to managers and
engineers. Make them part of the solution. Prepare
for pushback.
• Work with smaller teams to understand problems
and gain consensus. Larger teams for approvals.
Cloud Specific
• Be prepared to re-architect as AWS releases new
features and defer decisions whenever possible.
• Build a relationship with the AWS team. Your
Technical Account Manager and Solutions
Architects.
o Let AWS know if there is a gap in their
feature set that is needed. You may find out
that the capability is under development and
you can beta-test it.
Lessons Learned

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!