Auckland SQLSaturday 2018 - Building a Modern Analytics Solution in the cloud - Sabrina and Sergio Zenatti

1. Building a Modern Analytics
Solution in the Cloud
Sergio Zenatti Filho &
Sabrina da Motta Zenatti

2. Thanks to our Gold Sponsors: SQL Sat Auckland
Place your
photo here
Place your
photo here
Sergio Zenatti Filho
Associate Director Data & Analytics at Satalyst
• Over 16 years experience in Data and System Solutions
• MCSE: Business Intelligence and Data Management and Analytics
• MCSA: Cloud Platform
• Perth SQL Server User Group Co-Leader
/sergiozenatti @SergioZenatti https://zenatti.net/
Sabrina da Motta Zenatti
Data & Analytics Consultant at Satalyst
• Over 4 years experience in Data and Analytics Solutions
• Over 15 years experience in Finance/Accounting
• Power BI Certified
• Perth SQL Server User Group Leader
/sabrinamottazenatti @sabmotta https://zenatti.net/

3. Thanks to our Gold Sponsors: SQL Sat Auckland
Thanks to all sponsors

4. Thanks to our Gold Sponsors: SQL Sat Auckland
Session
• Traditional Analytics Solution
• Modern Analytics Solution in the Cloud
• Components: Azure Data Factory, Azure Data Lake,
Azure Analysis Services, Azure SQL DB and Power BI
• Building Modern Analytics Solution in the Cloud
• Demo

5. Thanks to our Gold Sponsors: SQL Sat Auckland
Traditional Analytics
• Time consuming;
• High cost infrastructure and development time;
• Tools:
• SQL Server 2012, 2014, 2016 and 2017
• SQL Server Integration Services
• SQL Server Analysis Services
• SQL Server Reporting Services
• Power BI (On-premises data gateway)

6. Thanks to our Gold Sponsors: SQL Sat Auckland
Modern Analytics - Cloud
• Quick turnaround on
Provision
• 100% PaaS (Low
maintenance)
• Flexible to scale up and
down as per business
requirements
• Small and big data
sources
• Structure, Semi-
Structure and
Unstructured data
Hot/Stream Path
Cold/Batch Path

7. Thanks to our Gold Sponsors: SQL Sat Auckland
New York Taxi
The requirements are:
• Every Day we need to process the current month data
extracted from the company system;
• Company system extract the current month data in CSV
format;
• CSV file is available at 6am and the average file size for a full
month is around 800 MB;
• Data needs to be available at 7am;
• Before start the day the management team discuss previous
days/months performance;
• The company would like to leverage cloud platform;
Data Available at: http://www.nyc.gov/html/tlc/html/about/trip_record_data.shtml

8. Thanks to our Gold Sponsors: SQL Sat Auckland
Design
Process Store Model
Storage blob
Azure Data Lake
Store
A hyper-scale repository for Big Data
analytics workloads with limitless storage for
analytics data.
Object storage solution for the cloud
optimized for storing massive amounts of
unstructured data, such as text or binary
data.
Azure Data Lake
Store

9. Thanks to our Gold Sponsors: SQL Sat Auckland
Process
Process Store Model
Azure SQL DW
Azure SQL
database
Azure Data Lake
Analytics
Data Bricks
HDInsight
A managed cloud database providing
transactional consistency for app
developers.
Fast, flexible and secure analytics platform
for the enterprise. A massively parallel
processing architecture designed for the
cloud.
On-demand analytics job service
powering intelligent action across a
hyper-scale repository for Big Data
analytics workloads.
Fully managed cloud service that makes it easy, fast
and cost-effective to process massive amounts of
data. Use popular open-source frameworks such as
Hadoop, Spark, Hive, LLAP, Kafka, Storm, R and more.
Fast, easy and collaborative Apache Spark-based
analytics platform Accelerate innovation by enabling
data science with a high-performance analytics
platform that’s optimised for Azure.
Azure Data Lake
Store
Azure Data Lake
Analytics

10. Thanks to our Gold Sponsors: SQL Sat Auckland
Store
Store Model
Storage blob
Azure Data Lake
Store
Azure SQL
database Azure SQL DW
Azure Data Lake
Analytics
A hyper-scale repository for Big Data
analytics workloads with limitless storage for
analytics data.
Fast, flexible and secure analytics platform
for the enterprise. A massively parallel
processing architecture designed for the
cloud.
A managed cloud database providing
transactional consistency for app
developers.
Object storage solution for the cloud
optimized for storing massive amounts of
unstructured data, such as text or binary
data.
Azure Data Lake
Store
Azure SQL
database

11. Thanks to our Gold Sponsors: SQL Sat Auckland
Model
Model
Analysis Service
Azure Data Lake
Analytics
Azure SQL
database
Business analytics solution that lets
you visualize your data and share insights
across your organization, or embed them
in your app or website. Connect to
hundreds of data sources and bring your
data to life with live dashboards and
reports.
Analysis Services is an analytical data
engine used in decision support and
business analytics. It provides enterprise-
grade semantic data models for business
reports and client applications such as
Power BI, Excel, Reporting Services
reports, and other data visualization tools.
Azure Data Lake
Store
Analysis Service

12. Thanks to our Gold Sponsors: SQL Sat Auckland
Design
Analysis Service
Azure Data Lake
Analytics
Azure SQL
database
How to process in sequence ??
Fully-managed data
integration service in the
cloud.
Azure Data Lake
Store

13. Thanks to our Gold Sponsors: SQL Sat Auckland
SSRS in Power BI

14. Thanks to our Gold Sponsors: SQL Sat Auckland
What next?
• https://azure.microsoft.com/en-us/documentation/learning-paths/data-factory/
• https://www.edx.org/course/processing-big-data-azure-data-lake-analytics
• https://www.edx.org/course/processing-big-data-hadoop-azure-hdinsight
• https://www.edx.org/course/analyzing-visualizing-data-power-bi
• https://www.edx.org/course/databases-in-azure-0
• https://www.edx.org/course/developing-sql-server-analysis-services-microsoft-dat225x
• https://docs.microsoft.com/en-us/azure/analysis-services/tutorials/aas-lesson-10-create-partitions

15. Thanks to our Gold Sponsors: SQL Sat Auckland
Q&A

16. Thank you
for your time!
Sergio Zenatti Filho
Associate Director Data & Analytics,
Satalyst
sergiozenatti @SergioZenatti
/sabrinamottazenatti @sabmotta https://zenatti.net/
https://zenatti.net/
Sabrina da Motta Zenatti
Data & Analytics Consultant,
Satalyst