This document discusses Landsbankinn's implementation of a logical data warehouse and data mesh architecture using Denodo over five years. It summarizes that:
1) Landsbankinn initially implemented a logical data warehouse with Denodo to create a single point of access, business logic, and control for reporting across heterogeneous data sources.
2) Over the next two years, they expanded this to include additional data consumers and sources without requiring ETL.
3) They then realized a data mesh approach where domains owned and published their data was better, reducing complexity and meetings for mapping changes.
4) The current approach has domains developing and sharing views via a data mesh while the logical data warehouse combines them, improving governance and flexibility
Sonagachi * best call girls in Kolkata | ₹,9500 Pay Cash 8005736733 Free Home...
Denodo: Enabling a Data Mesh Architecture and Data Sharing Culture at Landsbankinn
1. Denodo :
Enabling a Data Mesh
Architecture and Data
Sharing Culture at
Landsbankinn
Sylvain Dutilh
Landsbankinn Iceland
Denodo Technologies
Booth 504 - Exhibit hall
2. Landsbankinn
2
• Leading financial institution in Iceland
• 40% Market share Individual Banking
• 33% Market share Corporate Banking
• Best ESG risk ratings amongst European
banks (Sustainalytics 2021)
• Best bank at the Icelandic consumer
satisfaction ratings (Ánægjuvogin / Stjórnvísi 2021)
3. Data Virtualization and the Logical Data Warehouse
3
“Data virtualization can be used to create
virtualized and integrated views of data in-
memory, rather than executing data
movement and physically storing integrated
views in a target data structure.
- Gartner, November 2018
(Ehtisham Zaidi, Mark Beyer,
Ankush Jain, Sharat Menon)
“The logical data warehouse — a data
consolidation and virtualization architecture
of multiple analytic systems”
- Gartner, December 2020
(Henry Cook)
4. The Logical Data Warehouse
4
• Sits atop the “Physical” data warehouse
• Connects with other sources
• Transforms data for processing
• Exposes and abstracts data
6. SAS Macros
Year Zero - Before Data Virtualization
6
▪ Too many query points
▪ Heterogenous technologies
▪ Complex source systems
▪ Scattered business rules
▪ Semantic layers in BI
▪ Business logics in DB views
▪ Many points of access control
▪ Audit points all over the place
▪ Each system has its own access control
KPI DB Source DBs New DWH Old DWH Markets DB
Views
WebI
Lumira
Crystal
reports
Live Office
Views
Views
Views
General Reporting
KPI
SAS EG SAS VA
VA Server
Risk Reporting
Monitoring / Audit Business security
Business rules
Board
Other DBs
BO Semantic Layer (Universes / DF)
Data
Sources
Semantic
Layer
7. Year 1 - The Logical Data Warehouse
▪ Unique point of query
▪ “Need data? LDW has the answer!”
▪ For reporting, analytics, APIs, …
▪ Unique point of truth
▪ Business logic repository
▪ Lineage available
▪ Unique point of access control
▪ Unified access to the data
▪ Unique point of auditing
KPI DB Source DBs New DWH Old DWH Markets DB
WebI
Lumira
Live Office
General Reporting
KPI SAS EG SAS VA
VA Server
Risk Reporting
Board
Other DBs
Data
Sources
Logical Data Warehouse w/ Denodo
Monitoring / Audit Business security
Business rules
Crystal
reports
8. Years 2 and 3 - Expansion and Modernization
8
▪ Addition of data consumers
▪ Tableau
▪ REST / Restful APIs
▪ Addition of more data sources
▪ Where ETL is not required
▪ When history is provided in source
▪ Logical data pipelines
▪ Reduces the number of ETL jobs
▪ EDW gets data from LDW
WebI Tableau
Denodo to
Excel
General Reporting
KPI SAS EG SAS VA
VA Server
Risk Reporting
Board
Data
Sources
Logical Data Warehouse w/ Denodo
KPI DB
Source
DBs
New
DWH
Old
DWH
Markets
DB
Other
DBs
Flat files
Excel
SaaS
REST
SOAP
WWW
Customers
Domains
Operational
systems
Monitoring / Audit Business security
Business rules
Customer
statements
9. Year 4 - A flawed model
9
▪ Source data is cryptic
▪ Data comes from software vendors
▪ Lots of meetings needed to establish the data mapping
▪ Domains know their source
▪ How to find data in the source
▪ When source changes
▪ Domains must create views in the source
▪ Loss of lineage and governance
▪ What we wanted to get rid of in the first place
LDW
Source
DBs
Domains
Operational
systems
Views
10. Year 4 - Implementing a Data Mesh model
10
▪ A simplified process
1. Domains provided with a development space
2. LDW developers combine views
3. Domains publish data
4. Operational systems access the data
▪ Top-down modelling
▪ Using interface views (data contracts)
Source
system
Base
Data Mesh
Domain A
developer
Business
systems
LDW
developer
LDW
Requests
(interface contracts)
Shares Combines
LDW
Source
DBs
Operational
systems
Domains
Domain B
developer
Requests
(interface contracts)
Publication
Data Mesh
Publishes
LDW
11. Year 4 - Benefits of the Data Mesh w/ Denodo
11
▪ Delegate the ownership of data to the domains
▪ Data is in the hands of its creator
▪ Give better overview of the pipeline
▪ Views lifecycle managed by the source developer
▪ Reduce data pipelines
▪ Fewer ETL jobs when available
LDW
Source
DBs
Operational
systems
Domains
Savings
domain
Loans
domain
Cards
domain
Claims
domain
EDW
domain
LDW
developer
CRM Loan Online bank
12. Year 5 - What lies ahead
▪ Data Mesh conquers the bank
▪ Self-service for business
▪ Data API
12
KPI DB
New
DWH
Markets
DB
Other
DBs
Flat files SaaS
REST
SOAP
WWW
Source
DBs
Customers Risk
Reporting
Business
Board
Logical Data Warehouse w/ Denodo
API
Self-
Service
Self-
Service
Self-
Service
Data
Mesh
Data
Mesh
Data
Mesh
Data
Mesh
Data
Mesh
Data
Mesh
Data
Mesh
Data
Mesh
Data
Mesh
Op.
Systems
Data
Mesh