Azure Data Factory can be used to build modern data warehouse patterns with Azure SQL Data Warehouse. It allows extracting and transforming relational data from databases and loading it into Azure SQL Data Warehouse tables optimized for analytics. Data flows in Azure Data Factory can also clean and join disparate data from Azure Storage, Data Lake Store, and other data sources for loading into the data warehouse. This provides simple and productive ETL capabilities in the cloud at any scale.
3. Modern Data Warehouse Pattern with ADF Mapping Data Flows
Applications
Dashboards
Business/custom apps
(structured)
Logs, files, and media
(unstructured)
r Azure Storage/
Data Lake Store
Azure Data
Factory
Load files into data
lake on a schedule
Azure Data
Factory
Extract and
transform
relational data
Azure SQL DW
Load processed
data into tables
optimized for
analytics
Clean and
join disparate
data
Databases
Azure Databricks
Let’s take a look at the process through the visual user interface
Orchestration. Directs other services to execute actions as part of the transformation process.
Mapping Data Flows. Develop graphical data transformation logic at scale without writing code using Mapping Data Flows (preview).
Mapping Data Flows. Develop graphical data transformation logic at scale without writing code using Mapping Data Flows (preview).
Monitoring: Monitor pipeline and activity runs with a simple list view interface