This document discusses data quality patterns when using Azure Data Factory (ADF). It presents two modern data warehouse patterns that use ADF for orchestration: one using traditional ADF activities and another leveraging ADF mapping data flows. It also provides links to additional resources on ADF data flows, data quality patterns, expressions, performance, and connectors.

1. Data Quality Patterns in the Cloud
with Azure Data Factory
Azure Data Week

2. ADF: Simple and Productive ETL in the Cloud at any Scale

6. Modern Data Warehouse Pattern Today
Applications
Dashboards
Business/custom apps
(structured)
Logs, files, and media
(unstructured)
r
Ingest storage
Azure Storage/
Data Lake Store
Data
Loading
Azure Data
Factory
Load flat files
into data lake on a
schedule
Data processing
Read data from
files using DBFS
Orchestration
Azure Data
Factory
Extract and
transform
relational data
Azure Databricks
Serving storage
Azure SQL DW
Load processed
data into tables
optimized for
analytics
Clean and
join with
stored data
Load to SQL DW
Databases

7. Modern Data Warehouse Pattern with ADF Mapping Data Flows
Applications
Dashboards
Business/custom apps
(structured)
Logs, files, and media
(unstructured)
r Azure Storage/
Data Lake Store
Azure Data
Factory
Load files into data
lake on a schedule
Azure Data
Factory
Extract and
transform
relational data
Azure SQL DW
Load processed
data into tables
optimized for
analytics
Clean and
join disparate
data
Databases
Azure Databricks

13. list of connectors
full list of compute environments data

16. Data Flow

17. Data Quality Patterns

20. http://aka.ms/dataflowvideos
http://aka.ms/dataflowpatterns
https://docs.microsoft.com/en-us/azure/data-
factory/concepts-data-flow-overview
http://aka.ms/dataflowexpressions
https://aka.ms/dfperf
https://aka.ms/dflinks