SlideShare uma empresa Scribd logo
1 de 45
Baixar para ler offline
Copyright © 2014 Pearson Education, Inc.
1
CHAPTER 9:
DATA WAREHOUSING
Essentials of Database Management
Jeffrey A. Hoffer, Heikki Topi, V. Ramesh
Chapter 9 2
Copyright © 2014 Pearson Education, Inc.
OBJECTIVES
 Define terms
 Explore reasons for information gap
between information needs and availability
 Understand reasons for need of data
warehousing
 Describe three levels of data warehouse
architectures
 Describe two components of star schema
 Estimate fact table size
 Design a data mart
 Develop requirements for a data mart
2
Chapter 9 3
Copyright © 2014 Pearson Education, Inc.
DEFINITIONS
 Data Warehouse
 A subject-oriented, integrated, time-variant, non-
updatable collection of data used in support of
management decision-making processes
 Subject-oriented: e.g. customers, patients,
students, products
 Integrated: consistent naming conventions,
formats, encoding structures; from multiple data
sources
 Time-variant: can study trends and changes
 Non-updatable: read-only, periodically refreshed
 Data Mart
 A data warehouse that is limited in scope
3
Chapter 9 4
Copyright © 2014 Pearson Education, Inc.
HISTORY LEADING TO DATA
WAREHOUSING
 Improvement in database technologies,
especially relational DBMSs
 Advances in computer hardware, including
mass storage and parallel architectures
 Emergence of end-user computing with
powerful interfaces and tools
 Advances in middleware, enabling
heterogeneous database connectivity
 Recognition of difference between
operational and informational systems
4
Chapter 9 5
Copyright © 2014 Pearson Education, Inc.
NEED FOR DATA WAREHOUSING
 Integrated, company-wide view of high-
quality information (from disparate
databases)
 Separation of operational and
informational systems and data (for
improved performance)
5
Chapter 9 6
Copyright © 2014 Pearson Education, Inc.
ISSUES WITH COMPANY-WIDE VIEW
Inconsistent key structures
Synonyms
Free-form vs. structured fields
Inconsistent data values
Missing data
See figure 9-1 for example
6
7
Figure 9-1
Examples of
heterogeneous
data
Chapter 9 7
Copyright © 2014 Pearson Education, Inc.
Chapter 9 8
Copyright © 2014 Pearson Education, Inc.
ORGANIZATIONAL TRENDS
MOTIVATING DATA WAREHOUSES
 No single system of records
 Multiple systems not synchronized
 Organizational need to analyze
activities in a balanced way
 Customer relationship management
 Supplier relationship management
8
Chapter 9 9
Copyright © 2014 Pearson Education, Inc.
SEPARATING OPERATIONAL AND
INFORMATIONAL SYSTEMS
 Operational system – a system that is used
to run a business in real time, based on
current data; also called a system of record
 Informational system – a system designed
to support decision making based on
historical point-in-time and prediction data for
complex queries or data-mining applications
9
Chapter 9 10
Copyright © 2014 Pearson Education, Inc.
10
Chapter 9 11
Copyright © 2014 Pearson Education, Inc.
DATA WAREHOUSE ARCHITECTURES
Independent Data Mart
Dependent Data Mart and
Operational Data Store
Logical Data Mart and Real-Time
Data Warehouse
Three-Layer architecture
11
All involve some form of extract, transform and load (ETL)
12
Figure 9-2 Independent data mart
data warehousing architecture
Data marts:
Mini-warehouses, limited in scope
E
T
L
Separate ETL for each
independent data mart
Data access complexity
due to multiple data marts
Chapter 9 12
Copyright © 2014 Pearson Education, Inc.
13
Figure 9-3 Dependent data mart with
operational data store: a three-level architecture
E
T
L
Single ETL for
enterprise data warehouse (EDW)
Simpler data access
ODS provides option for
obtaining current data
Dependent data marts
loaded from EDW
Chapter 9 13
Copyright © 2014 Pearson Education, Inc.
14
E
T
L
Near real-time ETL for
Data Warehouse
ODS and data warehouse
are one and the same
Data marts are NOT separate databases,
but logical views of the data warehouse
 Easier to create new data marts
Figure 9-4 Logical data mart and real
time warehouse architecture
Chapter 9 14
Copyright © 2014 Pearson Education, Inc.
15
Chapter 9 15
Copyright © 2014 Pearson Education, Inc.
16
Figure 9-5 Three-layer data architecture for a data warehouse
Chapter 9 16
Copyright © 2014 Pearson Education, Inc.
Chapter 9 17
Copyright © 2014 Pearson Education, Inc.
DATA CHARACTERISTICS
STATUS VS. EVENT DATA
17
Status
Status
Event = a
database action
(create/ update/
delete) that
results from a
transaction
Figure 9-6
Example of DBMS
log entry
Chapter 9 18
Copyright © 2014 Pearson Education, Inc.
With transient
data, changes
to existing
records are
written over
previous
records, thus
destroying the
previous data
content
Figure 9-7
Transient
operational data
DATA CHARACTERISTICS
STATUS VS. EVENT DATA
Chapter 9 19
Copyright © 2014 Pearson Education, Inc.
19
Periodic data
are never
physically
altered or
deleted once
they have been
added to the
store
Figure 9-8 Periodic
warehouse data
DATA CHARACTERISTICS
STATUS VS. EVENT DATA
Chapter 9 20
Copyright © 2014 Pearson Education, Inc.
OTHER DATA WAREHOUSE CHANGES
 New descriptive attributes
 New business activity attributes
 New classes of descriptive attributes
 Descriptive attributes become more
refined
 Descriptive data are related to one
another
 New source of data
20
Chapter 9 21
Copyright © 2014 Pearson Education, Inc.
DERIVED DATA
 Objectives
 Ease of use for decision support applications
 Fast response to predefined user queries
 Customized data for particular target audiences
 Ad-hoc query support
 Data mining capabilities
 Characteristics
 Detailed (mostly periodic) data
 Aggregate (for summary)
 Distributed (to departmental servers)
21
Most common data model = dimensional model
(usually implemented as a star schema)
22
Figure 9-9 Components of a star schema
Fact tables contain factual
or quantitative data
Dimension tables contain descriptions
about the subjects of the business
1:N relationship between
dimension tables and fact tables
Excellent for ad-hoc queries, but bad for online transaction processing
Dimension tables are denormalized to
maximize performance
Chapter 9 22
Copyright © 2014 Pearson Education, Inc.
23
Figure 9-10 Star schema example
Fact table provides statistics for sales
broken down by product, period and
store dimensions
Chapter 9 23
Copyright © 2014 Pearson Education, Inc.
24
Figure 9-11 Star schema with sample data
Chapter 9 24
Copyright © 2014 Pearson Education, Inc.
Chapter 9 25
Copyright © 2014 Pearson Education, Inc.
SURROGATE KEYS
 Dimension table keys should be surrogate
(non-intelligent and non-business related),
because:
Business keys may change over time
Helps keep track of nonkey attribute
values for a given production key
Surrogate keys are simpler and shorter
Surrogate keys can be same length and
format for all key
25
Chapter 9 26
Copyright © 2014 Pearson Education, Inc.
GRAIN OF THE FACT TABLE
 Granularity of Fact Table–what level of
detail do you want?
Transactional grain–finest level
Aggregated grain–more summarized
Finer grains  better market basket
analysis capability
Finer grain  more dimension tables,
more rows in fact table
In Web-based commerce, finest
granularity is a click
26
Chapter 9 27
Copyright © 2014 Pearson Education, Inc.
DURATION OF THE DATABASE
 Natural duration–13 months or 5 quarters
 Financial institutions may need longer duration
 Older data is more difficult to source and cleanse
27
Chapter 9 28
Copyright © 2014 Pearson Education, Inc.
SIZE OF FACT TABLE
 Depends on the number of dimensions and the grain
of the fact table
 Number of rows = product of number of possible
values for each dimension associated with the fact
table
 Example: assume the following for Figure 9-11:
 Total rows calculated as follows (assuming only half
the products record sales for a given month):
28
Chapter 9 29
Copyright © 2014 Pearson Education, Inc.
29
Figure 9-12 Modeling dates
Fact tables contain time-period data
 Date dimensions are important
Chapter 9 30
Copyright © 2014 Pearson Education, Inc.
VARIATIONS OF THE STAR SCHEMA
 Multiple Facts Tables
 Can improve performance
 Often used to store facts for different combinations of
dimensions
 Conformed dimensions
 Hierarchies
 Sometimes a dimension forms a natural, fixed depth
hierarchy
 Design options
 Include all information for each level in a single denormalized
table
 Normalize the dimension into a nested set of 1:M table
relationships 30
Chapter 9 31
Copyright © 2014 Pearson Education, Inc.
31
Figure 9-13 Conformed dimensions
Conformed
dimension
Associated with
multiple fact
tables
Two fact tables  two (connected) start schemas.
Chapter 9 32
Copyright © 2014 Pearson Education, Inc.
32
Figure 9-14 Fixed product hierarchy
Dimension hierarchies help to provide levels of
aggregation for users wanting summary information
in a data warehouse
Chapter 9 33
Copyright © 2014 Pearson Education, Inc.
SLOWLY CHANGING DIMENSIONS (SCD)
 How to maintain knowledge of the past
 Kimble’s approaches:
 Type 1: just replace old data with new (lose
historical data)
 Type 3: for each changing attribute, create a
current value field and several old-valued fields
(multivalued)
 Type 2: create a new dimension table row each
time the dimension object changes, with all
dimension characteristics at the time of change.
Most common approach.
33
Chapter 9 34
Copyright © 2014 Pearson Education, Inc.
34
Figure 9-16 Example of Type 2 SCD Customer dimension table
The dimension table contains several records for the same
customer. The specific customer record to use depends on the
key and the date of the fact, which should be between start
and end dates of the SCD customer record.
Chapter 9 35
Copyright © 2014 Pearson Education, Inc.
10 ESSENTIAL RULES FOR
DIMENSIONAL MODELING
 Use atomic facts
 Create single-process
fact tables
 Include a date
dimension for each fact
table
 Enforce consistent
grain
 Disallow null keys in
fact tables
 Honor hierarchies
 Decode dimension tables
 Use surrogate keys
 Conform dimensions
 Balance requirements
with actual data
35
Chapter 9 36
Copyright © 2014 Pearson Education, Inc.
 Big Data = databases whose volume, velocity, and variety
strain the ability of relational DBMSs to capture, manage,
and process data in a timely fashion
 Issue of Big Data
 huge volume
 often unstructured (text, images, RFID, etc.)
 Columnar databases
 Column-based (rather than relational DB, which are row-based)
 optimize storage for summary data of few columns (different need than
OLTP)
 Data compression
 Sybase, Vertica, Infobright
36
BIG DATA AND COLUMNAR DATABASES
Chapter 9 37
Copyright © 2014 Pearson Education, Inc.
 “Not only SQL”
 a class of database technology used to store and
access textual and other unstructured data, using
more flexible structures than the rows and columns
format of relational databases
 Example NoSql languages: Unstructured Query
Language (UQL), XQuery (for XML data)
 Example NoSql engines: Cassandra (used by
Facebook), Hadoop and MapReduce
37
NOSQL
Chapter 9 38
Copyright © 2014 Pearson Education, Inc.
THE USER INTERFACE
METADATA (DATA CATALOG)
 Identify subjects of the data mart
 Identify dimensions and facts
 Indicate how data is derived from enterprise data
warehouses, including derivation rules
 Indicate how data is derived from operational data
store, including derivation rules
 Identify available reports and predefined queries
 Identify data analysis techniques (e.g. drill-down)
 Identify responsible people
38
Chapter 9 39
Copyright © 2014 Pearson Education, Inc.
ONLINE ANALYTICAL PROCESSING
(OLAP) TOOLS
 The use of a set of graphical tools that provides users with
multidimensional views of their data and allows them to
analyze the data using simple windowing techniques
 Relational OLAP (ROLAP)
 Traditional relational representation
 Multidimensional OLAP (MOLAP)
 Cube structure
 OLAP Operations
 Cube slicing–come up with 2-D view of data
 Drill-down–going from summary to more detailed
views
39
40
Figure 9-18 Slicing a data cube
Chapter 9 40
Copyright © 2014 Pearson Education, Inc.
Chapter 9 41
Copyright © 2014 Pearson Education, Inc.
41
Figure 9-19
Example of drill-down
a) Summary report
b) Drill-down with color attribute added
Starting with summary
data, users can obtain
details for particular
cells
Chapter 9 42
Copyright © 2014 Pearson Education, Inc.
BUSINESS PERFORMANCE MGMT (BPM)
42
Figure 9-22
Sample Dashboard
BPM systems allow
managers to measure,
monitor, and manage
key activities and
processes to achieve
organizational goals.
Dashboards are often
used to provide an
information system in
support of BPM.
Charts like these are examples of data visualization, the representation
of data in graphical and multimedia formats for human analysis.
Chapter 9 43
Copyright © 2014 Pearson Education, Inc.
DATA MINING
 Knowledge discovery using a blend
of statistical, AI, and computer
graphics techniques
Goals:
Explain observed events or conditions
Confirm hypotheses
Explore data for new or unexpected
relationships
43
44
Chapter 9 44
Copyright © 2014 Pearson Education, Inc.
45
All rights reserved. No part of this publication may be reproduced, stored in a
retrieval system, or transmitted, in any form or by any means, electronic,
mechanical, photocopying, recording, or otherwise, without the prior written
permission of the publisher. Printed in the United States of America.
Copyright © 2014 Pearson Education, Inc.

Mais conteúdo relacionado

Semelhante a hoffer_edm_pp_ch09.ppt

Dataware house multidimensionalmodelling
Dataware house multidimensionalmodellingDataware house multidimensionalmodelling
Dataware house multidimensionalmodellingmeghu123
 
Data Mining Concept & Technique-ch04.ppt
Data Mining Concept & Technique-ch04.pptData Mining Concept & Technique-ch04.ppt
Data Mining Concept & Technique-ch04.pptMutiaSari53
 
introduction to datawarehouse
introduction to datawarehouseintroduction to datawarehouse
introduction to datawarehousekiran14360
 
1Copyright © 2014 Pearson Education, Inc. Publishing as Pren.docx
1Copyright © 2014 Pearson Education, Inc. Publishing as Pren.docx1Copyright © 2014 Pearson Education, Inc. Publishing as Pren.docx
1Copyright © 2014 Pearson Education, Inc. Publishing as Pren.docxhyacinthshackley2629
 
Laudon_MIS13_ch06.ppt
Laudon_MIS13_ch06.pptLaudon_MIS13_ch06.ppt
Laudon_MIS13_ch06.pptanisur_rehman
 
Data Warehousing and Data Mining
Data Warehousing and Data MiningData Warehousing and Data Mining
Data Warehousing and Data Miningidnats
 
Chapter 4. Data Warehousing and On-Line Analytical Processing.ppt
Chapter 4. Data Warehousing and On-Line Analytical Processing.pptChapter 4. Data Warehousing and On-Line Analytical Processing.ppt
Chapter 4. Data Warehousing and On-Line Analytical Processing.pptSubrata Kumer Paul
 
009445185.pdf
009445185.pdf009445185.pdf
009445185.pdfEidTahir
 
Dimensional Modeling
Dimensional ModelingDimensional Modeling
Dimensional ModelingSunita Sahu
 
Day 02 sap_bi_overview_and_terminology
Day 02 sap_bi_overview_and_terminologyDay 02 sap_bi_overview_and_terminology
Day 02 sap_bi_overview_and_terminologytovetrivel
 
Partitioning your Oracle Data Warehouse - Just a simple task?
Partitioning your Oracle Data Warehouse - Just a simple task?Partitioning your Oracle Data Warehouse - Just a simple task?
Partitioning your Oracle Data Warehouse - Just a simple task?Trivadis
 
OLAP Cubes in Datawarehousing
OLAP Cubes in DatawarehousingOLAP Cubes in Datawarehousing
OLAP Cubes in DatawarehousingPrithwis Mukerjee
 
Data Mining: Concepts and Techniques (3rd ed.) — Chapter _04 olap
Data Mining:  Concepts and Techniques (3rd ed.)— Chapter _04 olapData Mining:  Concepts and Techniques (3rd ed.)— Chapter _04 olap
Data Mining: Concepts and Techniques (3rd ed.) — Chapter _04 olapSalah Amean
 
MIS-CH6: Foundation of BUsiness Intelligence: Databases & IS
MIS-CH6: Foundation of BUsiness Intelligence: Databases & ISMIS-CH6: Foundation of BUsiness Intelligence: Databases & IS
MIS-CH6: Foundation of BUsiness Intelligence: Databases & ISSukanya Ben
 
Chapter 6 foundations of business intelligence
Chapter 6 foundations of business intelligenceChapter 6 foundations of business intelligence
Chapter 6 foundations of business intelligenceVan Chau
 

Semelhante a hoffer_edm_pp_ch09.ppt (20)

Dataware house multidimensionalmodelling
Dataware house multidimensionalmodellingDataware house multidimensionalmodelling
Dataware house multidimensionalmodelling
 
Data Mining Concept & Technique-ch04.ppt
Data Mining Concept & Technique-ch04.pptData Mining Concept & Technique-ch04.ppt
Data Mining Concept & Technique-ch04.ppt
 
introduction to datawarehouse
introduction to datawarehouseintroduction to datawarehouse
introduction to datawarehouse
 
1Copyright © 2014 Pearson Education, Inc. Publishing as Pren.docx
1Copyright © 2014 Pearson Education, Inc. Publishing as Pren.docx1Copyright © 2014 Pearson Education, Inc. Publishing as Pren.docx
1Copyright © 2014 Pearson Education, Inc. Publishing as Pren.docx
 
Laudon_MIS13_ch06.ppt
Laudon_MIS13_ch06.pptLaudon_MIS13_ch06.ppt
Laudon_MIS13_ch06.ppt
 
ITReady DW Day2
ITReady DW Day2ITReady DW Day2
ITReady DW Day2
 
Data Warehousing and Data Mining
Data Warehousing and Data MiningData Warehousing and Data Mining
Data Warehousing and Data Mining
 
Chapter 4. Data Warehousing and On-Line Analytical Processing.ppt
Chapter 4. Data Warehousing and On-Line Analytical Processing.pptChapter 4. Data Warehousing and On-Line Analytical Processing.ppt
Chapter 4. Data Warehousing and On-Line Analytical Processing.ppt
 
Database 1 Introduction
Database 1   IntroductionDatabase 1   Introduction
Database 1 Introduction
 
009445185.pdf
009445185.pdf009445185.pdf
009445185.pdf
 
Dimensional Modeling
Dimensional ModelingDimensional Modeling
Dimensional Modeling
 
Chap09
Chap09Chap09
Chap09
 
Day 02 sap_bi_overview_and_terminology
Day 02 sap_bi_overview_and_terminologyDay 02 sap_bi_overview_and_terminology
Day 02 sap_bi_overview_and_terminology
 
Partitioning your Oracle Data Warehouse - Just a simple task?
Partitioning your Oracle Data Warehouse - Just a simple task?Partitioning your Oracle Data Warehouse - Just a simple task?
Partitioning your Oracle Data Warehouse - Just a simple task?
 
OLAP Cubes in Datawarehousing
OLAP Cubes in DatawarehousingOLAP Cubes in Datawarehousing
OLAP Cubes in Datawarehousing
 
Data Mining: Concepts and Techniques (3rd ed.) — Chapter _04 olap
Data Mining:  Concepts and Techniques (3rd ed.)— Chapter _04 olapData Mining:  Concepts and Techniques (3rd ed.)— Chapter _04 olap
Data Mining: Concepts and Techniques (3rd ed.) — Chapter _04 olap
 
MIS-CH6: Foundation of BUsiness Intelligence: Databases & IS
MIS-CH6: Foundation of BUsiness Intelligence: Databases & ISMIS-CH6: Foundation of BUsiness Intelligence: Databases & IS
MIS-CH6: Foundation of BUsiness Intelligence: Databases & IS
 
Data wirehouse
Data wirehouseData wirehouse
Data wirehouse
 
Layout.ppt
Layout.pptLayout.ppt
Layout.ppt
 
Chapter 6 foundations of business intelligence
Chapter 6 foundations of business intelligenceChapter 6 foundations of business intelligence
Chapter 6 foundations of business intelligence
 

Último

Microeconomic Group Presentation Apple.pdf
Microeconomic Group Presentation Apple.pdfMicroeconomic Group Presentation Apple.pdf
Microeconomic Group Presentation Apple.pdfmxlos0
 
Brain Tumor Detection with Machine Learning.pptx
Brain Tumor Detection with Machine Learning.pptxBrain Tumor Detection with Machine Learning.pptx
Brain Tumor Detection with Machine Learning.pptxShammiRai3
 
Empowering Decisions A Guide to Embedded Analytics
Empowering Decisions A Guide to Embedded AnalyticsEmpowering Decisions A Guide to Embedded Analytics
Empowering Decisions A Guide to Embedded AnalyticsGain Insights
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptxFurkanTasci3
 
Enabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsEnabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsNeo4j
 
Báo cáo Social Media Benchmark 2024 cho dân Marketing
Báo cáo Social Media Benchmark 2024 cho dân MarketingBáo cáo Social Media Benchmark 2024 cho dân Marketing
Báo cáo Social Media Benchmark 2024 cho dân MarketingMarketingTrips
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptxFurkanTasci3
 
The market for cross-border mortgages in Europe
The market for cross-border mortgages in EuropeThe market for cross-border mortgages in Europe
The market for cross-border mortgages in Europe321k
 
Air Con Energy Rating Info411 Presentation.pdf
Air Con Energy Rating Info411 Presentation.pdfAir Con Energy Rating Info411 Presentation.pdf
Air Con Energy Rating Info411 Presentation.pdfJasonBoboKyaw
 
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j
 
Paul Martin (Gartner) - Show Me the AI Money.pdf
Paul Martin (Gartner) - Show Me the AI Money.pdfPaul Martin (Gartner) - Show Me the AI Money.pdf
Paul Martin (Gartner) - Show Me the AI Money.pdfdcphostmaster
 
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...ferisulianta.com
 
Unleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IMUnleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IMMarco Wobben
 
Stochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptxStochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptxjkmrshll88
 
Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1bengalurutug
 
How to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product DevelopmentHow to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product DevelopmentAggregage
 
Data Collection from Social Media Platforms
Data Collection from Social Media PlatformsData Collection from Social Media Platforms
Data Collection from Social Media PlatformsMahmoud Yasser
 
Understanding the Impact of video length on student performance
Understanding the Impact of video length on student performanceUnderstanding the Impact of video length on student performance
Understanding the Impact of video length on student performancePrithaVashisht1
 
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j
 

Último (20)

Microeconomic Group Presentation Apple.pdf
Microeconomic Group Presentation Apple.pdfMicroeconomic Group Presentation Apple.pdf
Microeconomic Group Presentation Apple.pdf
 
Brain Tumor Detection with Machine Learning.pptx
Brain Tumor Detection with Machine Learning.pptxBrain Tumor Detection with Machine Learning.pptx
Brain Tumor Detection with Machine Learning.pptx
 
Empowering Decisions A Guide to Embedded Analytics
Empowering Decisions A Guide to Embedded AnalyticsEmpowering Decisions A Guide to Embedded Analytics
Empowering Decisions A Guide to Embedded Analytics
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
 
Enabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsEnabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge Graphs
 
Báo cáo Social Media Benchmark 2024 cho dân Marketing
Báo cáo Social Media Benchmark 2024 cho dân MarketingBáo cáo Social Media Benchmark 2024 cho dân Marketing
Báo cáo Social Media Benchmark 2024 cho dân Marketing
 
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptxSTOCK PRICE ANALYSIS  Furkan Ali TASCI --.pptx
STOCK PRICE ANALYSIS Furkan Ali TASCI --.pptx
 
The market for cross-border mortgages in Europe
The market for cross-border mortgages in EuropeThe market for cross-border mortgages in Europe
The market for cross-border mortgages in Europe
 
Air Con Energy Rating Info411 Presentation.pdf
Air Con Energy Rating Info411 Presentation.pdfAir Con Energy Rating Info411 Presentation.pdf
Air Con Energy Rating Info411 Presentation.pdf
 
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
 
Paul Martin (Gartner) - Show Me the AI Money.pdf
Paul Martin (Gartner) - Show Me the AI Money.pdfPaul Martin (Gartner) - Show Me the AI Money.pdf
Paul Martin (Gartner) - Show Me the AI Money.pdf
 
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...
Prediction Of Cryptocurrency Prices Using Lstm, Svm And Polynomial Regression...
 
Unleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IMUnleashing Datas Potential - Mastering Precision with FCO-IM
Unleashing Datas Potential - Mastering Precision with FCO-IM
 
Stochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptxStochastic Dynamic Programming and You.pptx
Stochastic Dynamic Programming and You.pptx
 
Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1Bengaluru Tableau UG event- 2nd March 2024 Q1
Bengaluru Tableau UG event- 2nd March 2024 Q1
 
How to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product DevelopmentHow to Build an Experimentation Culture for Data-Driven Product Development
How to Build an Experimentation Culture for Data-Driven Product Development
 
Data Collection from Social Media Platforms
Data Collection from Social Media PlatformsData Collection from Social Media Platforms
Data Collection from Social Media Platforms
 
Understanding the Impact of video length on student performance
Understanding the Impact of video length on student performanceUnderstanding the Impact of video length on student performance
Understanding the Impact of video length on student performance
 
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
 
Target_Company_Data_breach_2013_110million
Target_Company_Data_breach_2013_110millionTarget_Company_Data_breach_2013_110million
Target_Company_Data_breach_2013_110million
 

hoffer_edm_pp_ch09.ppt

  • 1. Copyright © 2014 Pearson Education, Inc. 1 CHAPTER 9: DATA WAREHOUSING Essentials of Database Management Jeffrey A. Hoffer, Heikki Topi, V. Ramesh
  • 2. Chapter 9 2 Copyright © 2014 Pearson Education, Inc. OBJECTIVES  Define terms  Explore reasons for information gap between information needs and availability  Understand reasons for need of data warehousing  Describe three levels of data warehouse architectures  Describe two components of star schema  Estimate fact table size  Design a data mart  Develop requirements for a data mart 2
  • 3. Chapter 9 3 Copyright © 2014 Pearson Education, Inc. DEFINITIONS  Data Warehouse  A subject-oriented, integrated, time-variant, non- updatable collection of data used in support of management decision-making processes  Subject-oriented: e.g. customers, patients, students, products  Integrated: consistent naming conventions, formats, encoding structures; from multiple data sources  Time-variant: can study trends and changes  Non-updatable: read-only, periodically refreshed  Data Mart  A data warehouse that is limited in scope 3
  • 4. Chapter 9 4 Copyright © 2014 Pearson Education, Inc. HISTORY LEADING TO DATA WAREHOUSING  Improvement in database technologies, especially relational DBMSs  Advances in computer hardware, including mass storage and parallel architectures  Emergence of end-user computing with powerful interfaces and tools  Advances in middleware, enabling heterogeneous database connectivity  Recognition of difference between operational and informational systems 4
  • 5. Chapter 9 5 Copyright © 2014 Pearson Education, Inc. NEED FOR DATA WAREHOUSING  Integrated, company-wide view of high- quality information (from disparate databases)  Separation of operational and informational systems and data (for improved performance) 5
  • 6. Chapter 9 6 Copyright © 2014 Pearson Education, Inc. ISSUES WITH COMPANY-WIDE VIEW Inconsistent key structures Synonyms Free-form vs. structured fields Inconsistent data values Missing data See figure 9-1 for example 6
  • 7. 7 Figure 9-1 Examples of heterogeneous data Chapter 9 7 Copyright © 2014 Pearson Education, Inc.
  • 8. Chapter 9 8 Copyright © 2014 Pearson Education, Inc. ORGANIZATIONAL TRENDS MOTIVATING DATA WAREHOUSES  No single system of records  Multiple systems not synchronized  Organizational need to analyze activities in a balanced way  Customer relationship management  Supplier relationship management 8
  • 9. Chapter 9 9 Copyright © 2014 Pearson Education, Inc. SEPARATING OPERATIONAL AND INFORMATIONAL SYSTEMS  Operational system – a system that is used to run a business in real time, based on current data; also called a system of record  Informational system – a system designed to support decision making based on historical point-in-time and prediction data for complex queries or data-mining applications 9
  • 10. Chapter 9 10 Copyright © 2014 Pearson Education, Inc. 10
  • 11. Chapter 9 11 Copyright © 2014 Pearson Education, Inc. DATA WAREHOUSE ARCHITECTURES Independent Data Mart Dependent Data Mart and Operational Data Store Logical Data Mart and Real-Time Data Warehouse Three-Layer architecture 11 All involve some form of extract, transform and load (ETL)
  • 12. 12 Figure 9-2 Independent data mart data warehousing architecture Data marts: Mini-warehouses, limited in scope E T L Separate ETL for each independent data mart Data access complexity due to multiple data marts Chapter 9 12 Copyright © 2014 Pearson Education, Inc.
  • 13. 13 Figure 9-3 Dependent data mart with operational data store: a three-level architecture E T L Single ETL for enterprise data warehouse (EDW) Simpler data access ODS provides option for obtaining current data Dependent data marts loaded from EDW Chapter 9 13 Copyright © 2014 Pearson Education, Inc.
  • 14. 14 E T L Near real-time ETL for Data Warehouse ODS and data warehouse are one and the same Data marts are NOT separate databases, but logical views of the data warehouse  Easier to create new data marts Figure 9-4 Logical data mart and real time warehouse architecture Chapter 9 14 Copyright © 2014 Pearson Education, Inc.
  • 15. 15 Chapter 9 15 Copyright © 2014 Pearson Education, Inc.
  • 16. 16 Figure 9-5 Three-layer data architecture for a data warehouse Chapter 9 16 Copyright © 2014 Pearson Education, Inc.
  • 17. Chapter 9 17 Copyright © 2014 Pearson Education, Inc. DATA CHARACTERISTICS STATUS VS. EVENT DATA 17 Status Status Event = a database action (create/ update/ delete) that results from a transaction Figure 9-6 Example of DBMS log entry
  • 18. Chapter 9 18 Copyright © 2014 Pearson Education, Inc. With transient data, changes to existing records are written over previous records, thus destroying the previous data content Figure 9-7 Transient operational data DATA CHARACTERISTICS STATUS VS. EVENT DATA
  • 19. Chapter 9 19 Copyright © 2014 Pearson Education, Inc. 19 Periodic data are never physically altered or deleted once they have been added to the store Figure 9-8 Periodic warehouse data DATA CHARACTERISTICS STATUS VS. EVENT DATA
  • 20. Chapter 9 20 Copyright © 2014 Pearson Education, Inc. OTHER DATA WAREHOUSE CHANGES  New descriptive attributes  New business activity attributes  New classes of descriptive attributes  Descriptive attributes become more refined  Descriptive data are related to one another  New source of data 20
  • 21. Chapter 9 21 Copyright © 2014 Pearson Education, Inc. DERIVED DATA  Objectives  Ease of use for decision support applications  Fast response to predefined user queries  Customized data for particular target audiences  Ad-hoc query support  Data mining capabilities  Characteristics  Detailed (mostly periodic) data  Aggregate (for summary)  Distributed (to departmental servers) 21 Most common data model = dimensional model (usually implemented as a star schema)
  • 22. 22 Figure 9-9 Components of a star schema Fact tables contain factual or quantitative data Dimension tables contain descriptions about the subjects of the business 1:N relationship between dimension tables and fact tables Excellent for ad-hoc queries, but bad for online transaction processing Dimension tables are denormalized to maximize performance Chapter 9 22 Copyright © 2014 Pearson Education, Inc.
  • 23. 23 Figure 9-10 Star schema example Fact table provides statistics for sales broken down by product, period and store dimensions Chapter 9 23 Copyright © 2014 Pearson Education, Inc.
  • 24. 24 Figure 9-11 Star schema with sample data Chapter 9 24 Copyright © 2014 Pearson Education, Inc.
  • 25. Chapter 9 25 Copyright © 2014 Pearson Education, Inc. SURROGATE KEYS  Dimension table keys should be surrogate (non-intelligent and non-business related), because: Business keys may change over time Helps keep track of nonkey attribute values for a given production key Surrogate keys are simpler and shorter Surrogate keys can be same length and format for all key 25
  • 26. Chapter 9 26 Copyright © 2014 Pearson Education, Inc. GRAIN OF THE FACT TABLE  Granularity of Fact Table–what level of detail do you want? Transactional grain–finest level Aggregated grain–more summarized Finer grains  better market basket analysis capability Finer grain  more dimension tables, more rows in fact table In Web-based commerce, finest granularity is a click 26
  • 27. Chapter 9 27 Copyright © 2014 Pearson Education, Inc. DURATION OF THE DATABASE  Natural duration–13 months or 5 quarters  Financial institutions may need longer duration  Older data is more difficult to source and cleanse 27
  • 28. Chapter 9 28 Copyright © 2014 Pearson Education, Inc. SIZE OF FACT TABLE  Depends on the number of dimensions and the grain of the fact table  Number of rows = product of number of possible values for each dimension associated with the fact table  Example: assume the following for Figure 9-11:  Total rows calculated as follows (assuming only half the products record sales for a given month): 28
  • 29. Chapter 9 29 Copyright © 2014 Pearson Education, Inc. 29 Figure 9-12 Modeling dates Fact tables contain time-period data  Date dimensions are important
  • 30. Chapter 9 30 Copyright © 2014 Pearson Education, Inc. VARIATIONS OF THE STAR SCHEMA  Multiple Facts Tables  Can improve performance  Often used to store facts for different combinations of dimensions  Conformed dimensions  Hierarchies  Sometimes a dimension forms a natural, fixed depth hierarchy  Design options  Include all information for each level in a single denormalized table  Normalize the dimension into a nested set of 1:M table relationships 30
  • 31. Chapter 9 31 Copyright © 2014 Pearson Education, Inc. 31 Figure 9-13 Conformed dimensions Conformed dimension Associated with multiple fact tables Two fact tables  two (connected) start schemas.
  • 32. Chapter 9 32 Copyright © 2014 Pearson Education, Inc. 32 Figure 9-14 Fixed product hierarchy Dimension hierarchies help to provide levels of aggregation for users wanting summary information in a data warehouse
  • 33. Chapter 9 33 Copyright © 2014 Pearson Education, Inc. SLOWLY CHANGING DIMENSIONS (SCD)  How to maintain knowledge of the past  Kimble’s approaches:  Type 1: just replace old data with new (lose historical data)  Type 3: for each changing attribute, create a current value field and several old-valued fields (multivalued)  Type 2: create a new dimension table row each time the dimension object changes, with all dimension characteristics at the time of change. Most common approach. 33
  • 34. Chapter 9 34 Copyright © 2014 Pearson Education, Inc. 34 Figure 9-16 Example of Type 2 SCD Customer dimension table The dimension table contains several records for the same customer. The specific customer record to use depends on the key and the date of the fact, which should be between start and end dates of the SCD customer record.
  • 35. Chapter 9 35 Copyright © 2014 Pearson Education, Inc. 10 ESSENTIAL RULES FOR DIMENSIONAL MODELING  Use atomic facts  Create single-process fact tables  Include a date dimension for each fact table  Enforce consistent grain  Disallow null keys in fact tables  Honor hierarchies  Decode dimension tables  Use surrogate keys  Conform dimensions  Balance requirements with actual data 35
  • 36. Chapter 9 36 Copyright © 2014 Pearson Education, Inc.  Big Data = databases whose volume, velocity, and variety strain the ability of relational DBMSs to capture, manage, and process data in a timely fashion  Issue of Big Data  huge volume  often unstructured (text, images, RFID, etc.)  Columnar databases  Column-based (rather than relational DB, which are row-based)  optimize storage for summary data of few columns (different need than OLTP)  Data compression  Sybase, Vertica, Infobright 36 BIG DATA AND COLUMNAR DATABASES
  • 37. Chapter 9 37 Copyright © 2014 Pearson Education, Inc.  “Not only SQL”  a class of database technology used to store and access textual and other unstructured data, using more flexible structures than the rows and columns format of relational databases  Example NoSql languages: Unstructured Query Language (UQL), XQuery (for XML data)  Example NoSql engines: Cassandra (used by Facebook), Hadoop and MapReduce 37 NOSQL
  • 38. Chapter 9 38 Copyright © 2014 Pearson Education, Inc. THE USER INTERFACE METADATA (DATA CATALOG)  Identify subjects of the data mart  Identify dimensions and facts  Indicate how data is derived from enterprise data warehouses, including derivation rules  Indicate how data is derived from operational data store, including derivation rules  Identify available reports and predefined queries  Identify data analysis techniques (e.g. drill-down)  Identify responsible people 38
  • 39. Chapter 9 39 Copyright © 2014 Pearson Education, Inc. ONLINE ANALYTICAL PROCESSING (OLAP) TOOLS  The use of a set of graphical tools that provides users with multidimensional views of their data and allows them to analyze the data using simple windowing techniques  Relational OLAP (ROLAP)  Traditional relational representation  Multidimensional OLAP (MOLAP)  Cube structure  OLAP Operations  Cube slicing–come up with 2-D view of data  Drill-down–going from summary to more detailed views 39
  • 40. 40 Figure 9-18 Slicing a data cube Chapter 9 40 Copyright © 2014 Pearson Education, Inc.
  • 41. Chapter 9 41 Copyright © 2014 Pearson Education, Inc. 41 Figure 9-19 Example of drill-down a) Summary report b) Drill-down with color attribute added Starting with summary data, users can obtain details for particular cells
  • 42. Chapter 9 42 Copyright © 2014 Pearson Education, Inc. BUSINESS PERFORMANCE MGMT (BPM) 42 Figure 9-22 Sample Dashboard BPM systems allow managers to measure, monitor, and manage key activities and processes to achieve organizational goals. Dashboards are often used to provide an information system in support of BPM. Charts like these are examples of data visualization, the representation of data in graphical and multimedia formats for human analysis.
  • 43. Chapter 9 43 Copyright © 2014 Pearson Education, Inc. DATA MINING  Knowledge discovery using a blend of statistical, AI, and computer graphics techniques Goals: Explain observed events or conditions Confirm hypotheses Explore data for new or unexpected relationships 43
  • 44. 44 Chapter 9 44 Copyright © 2014 Pearson Education, Inc.
  • 45. 45 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. Printed in the United States of America. Copyright © 2014 Pearson Education, Inc.