This document discusses best practices for migrating data from on-premises systems to the cloud. It recommends developing a multi-phase migration strategy, considering different approaches to migration, putting the right skills in place, and using data integration tools to improve data quality and analytics capabilities rather than just moving data. The case study of Crowley Maritime is presented, which faced challenges integrating legacy mainframe data into its cloud data warehouse until it used Precisely Connect to transform and replicate the data in real-time.

1. Fern Halper Ph.D.
VP and Senior Director, TDWI
Advanced Analytics
November 18, 2020
Don’t Bring Old Problems to Your
New Cloud Data Warehouse

2. SPONSORS
2

3. FERN HALPER
VP, Senior Research Director for
Advanced Analytics
TDWI

4. What we will talk
about today
• How and why organizations are
moving to the cloud
• Best practices for your cloud data
migration/integration strategy
• Data integration solutions to avoid
common problems of integrating
legacy data to cloud data
warehouses

5. THE WHY AND HOW OF CLOUD
ADOPTION

6. 10%
12%
18%
18%
19%
23%
27%
30%
36%
38%
44%
49%
68%
86%
95%
21%
18%
17%
25%
31%
35%
25%
21%
36%
21%
27%
22%
13%
6%
3%
0% 20% 40% 60% 80% 100% 120%
Audio data
Video data
Image still data
Clickstream data
Machine generated data (e.g., from sensors,…
Real-time event streams
Geospatial data
External text data
Semi-structured data
Internal text data
Time series data
Log data
Demographic data
Transactional data
Structured data
WHAT KIND OF DATA ARE YOU
CURRENTLY MANAGING? LOOKING
TO MANAGE IN THE NEXT YEAR?
Manage now Manage in next year
(Source: TDWI 2020 survey, n=~100)
• Data becoming more complex
• Analytics is becoming more
advanced
• Past reports and dashboards
• More self-service
• Iterative, compute intensive

7. More organizations are
looking to move to the
cloud for analytics and
other use cases
Source: 2020 TDWI survey, n=~100
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Object storage in the cloud
NoSQL DBMSs
Spark
Analytic DBMSs
Data integration platforms in the cloud
Data lake on premises
Open source software (except ubiquitous…
Containers (e.g., Docker)
Non-relational DBMS
Data lake in the cloud
Data warehouse in the cloud
Hadoop
Analytics tools in the cloud
Data integration platforms on premises
Data warehouse on premises
Analytics tools on premises
Relational DBMS
What data management and analytics tools does
your organization currently use? Please select all
that apply.

8. CLOUD BENEFITS
Scalability/
Elasticity
• Scales up/scale down
New Data • Capture and analyze diverse
cloud generated data
Agility • Separates compute from
storage
• New innovations
• Enables focus on value added
activities
New Analytics • Traditional DW often has
trouble supporting AA
• Cloud good for compute
intensive analytics
• Numerous benefits to
cloud
• COVID is accelerating
change

9. What we see at
TDWI
• Different reasons for moving to the cloud
• Move to a multi-platform environment
• Different broad strategies for cloud
adoption
– Many adopting a cloud first strategy and for new
projects
– Others moving all but sensitive data workloads
to the cloud
– Still others utilizing cloud for advanced analytics
workloads

10. Challenges
• Talent
• Data migration
• Data governance
• Building cloud strategies
• Dealing with legacy systems

11. BEST PRACTICES FOR CLOUD
MIGRATION

12. Develop a multi-
phase strategy
Consider
approaches and
plan
Put talent in place
Don’t just move:
improve

13. Develop a multi-
phase strategy
• Develop the strategy
– Segment the work into manageable pieces that
minimizes risk
– Involve different parts of the organization in the
plan
– Each piece has a technical goal that adds value
– Devise a technology roadmap
• Additionally
– Avoid “big bang” projects
– Align on platform

14. Consider
approaches and
plan
• Approaches
– Lift and shift
– Minimal changes
– Modernize?
• Plan
– Steps include assess, prioritize,
architecture design, development,
migration, test (iterate), parallel run,
acceptance

15. Put skills in place
• Talent requirements
– Architects
– Engineers
– Cloud expertise
• Additionally
– Legacy systems
• Solutions that can natively understand and access legacy
data
– Apps expertise for apps that are migrated
– Consider consultants but make sure there is
training before they leave

16. Don’t just
move: improve!
• Move
– Consider data loss
– Ensure integrity
– Apps, processes, people will move too
• Improve
– Enhance data quality/trust
– Enhance analytics
– The experience
– Architecture
– Improve governance with a holistic plan

17. Summary
Trends Best Practices
• Develop a multi-phase
strategy
• Consider migration
approaches and plan
• Put the right talent in place
• Don’t just move: improve!
• Organizations using new
data types for new analytics
• More often, they are moving
to cloud for
scalability/flexibility to
support new use cases
• There are different use
cases and broad strategic
principles employed

18. THANK YOU!

19. Ashwin Ramachandran
Senior Product Manager
Precisely

20. Crowley Maritime
Attaining enterprise data access
with Precisely Connect
2020
Ashwin Ramachandran | Senior Product Manager

21. The global leader in data integrity
Trust your data. Build your possibilities.
Our data integrity software and data enrichment products
deliver accuracy and consistency to power confident
business decisions.
Brands you trust, trust us
Data leaders partner with us
of the Fortune 100
90
Customers in more than
100
2,000
employees
customers
12,000
countries
Precisely + TDWI: Don’t Bring Old Problems to Your New Cloud Data Warehouse

22. Connecting today’s infrastructure with tomorrow’s technology to
unlock the potential of all your enterprise data.
Extract, Transform, Load Data Replication via CDC
High-performance ETL
for Apache Spark, Cloud,
Windows, Linux, Unix
and Hadoop MapReduce
Real-time database replication
to streaming platforms, Cloud,
databases and data warehouses
Connect
Precisely + TDWI: Don’t Bring Old Problems to Your New Cloud Data Warehouse

23. Precisely + TDWI: Don’t Bring Old Problems to Your New Cloud Data Warehouse
• Founded in 1892
• Global leader in Marine
Solutions, Energy and
Logistics Services
• Over 5,000 employees
worldwide
• Provides services and
staffing solutions for
Shipping, LNG
Distribution and Port
Escort operations
• Leading provider of
logistics for U.S. Federal
and NATO agencies

24. Business Mandate
1. Connect customer and
shipment information that
resides on multiple systems
of record
2. Improve integration and
replication of changes to
meet real-time demand
Precisely + TDWI: Don’t Bring Old Problems to Your New Cloud Data Warehouse

25. IT Mandate
1. Modernize IT environments
by moving data from on-prem
systems to the cloud
2. Lower costs and dependency
on mainframe system
Precisely + TDWI: Don’t Bring Old Problems to Your New Cloud Data Warehouse

26. Challenges
• Legacy data formats (e.g., mainframe and
IBM i) are not readily readable and
consumable within cloud data warehouses
– Binary data formats
– REDEFINEs in data fields
– Complex record structures
• Making data movement between on-
premises and the cloud data warehouse was
slow and delayed
• Complex integration architectures, made
it harder to connect to the cloud data
warehouse
Precisely + TDWI: Don’t Bring Old Problems to Your New Cloud Data Warehouse

27. Precisely + TDWI: Don’t Bring Old Problems to Your New Cloud Data Warehouse
+Connect

28. Precisely + TDWI: Don’t Bring Old Problems to Your New Cloud Data Warehouse
Results
1
Integrate and replicate
hundreds of z/OS
tables to Snowflake
2
Power customer
dashboards for “near
real-time” reporting
3
Improve productivity
with hourly
dashboard updates
for employees
4
Provide shipment
information in a
way that no other
competitor can

30. Questions and Answers
tdwi.org

31. CONTACT INFORMATION
If you have further questions or comments:
Fern Halper, TDWI
fhalper@tdwi.org @fhalper
Ashwin Ramachandran
aramachandran@precisely.com
tdwi.org

32. Thanks to Our Sponsors
3