SlideShare a Scribd company logo
1 of 33
Not quite what I was expecting
I went ahead and built my own report instead
Missed the mark
PAINFULLY SLOWDifficult to navigate
I’ll never use those features
Something like beforeI didn’t want you to change the way it’s always been done
IT’S NOT YOU.
IT’S YOUR DATA MODEL.
ALEX M POWERS
Microsoft Certified Solutions Associate: BI Reporting (Excel, Power BI)
Microsoft Technology Associate: Python (Introduction to Programming)
Microsoft Office Expert: Excel 2013, Excel 2010
Microsoft Office Specialist: Access 2013, Excel 2010
Microsoft #HowWeExcel Contest Winner
Co-Organizer of the St Louis Power BI User Group #STLPBIUG
Millennial Whisperer and Trainer | It’s Not About The Cell LLC: itsnotaboutthecell.com
Confused by the term “Pair of Socks”
Bad at Fortnite
USER EXPERIENCE (UX)
Refers to a person's emotions and attitudes about using a
product, system or service. It includes the practical,
experiential, affective, meaningful and valuable aspects
of human–computer interaction and product ownership.
Additionally, it includes a person’s perception of system aspects
such as ease of use, efficiency and utility.
Source (Wikipedia)
SESSION OBJECTIVES
1. Readability allows for discovery. (Ease of Use)
2. Performance considerations. (Efficiency)
3. Trust the abilities of your end users to learn. (Utility)
CLEAN THAT
_______ UP.
FIELD NAMES
Type Example
Camel Case orderDate
Pascal Case OrderDate
Snake Case ORDER_DATE
let
Source = #table(
type table [CUSTOMER_ID = Int64.Type, CUSTOMER_NAME = text, CUSTOMER_CITY = text, CUSTOMER_STATE = text],
{
{1, "Bob Smith", "Rockford", "IL"},
{2, "Randy Savage", "St Louis", "MO"},
{3, "Lucy Davis", "Lansing", "MI"}
}
),
// Replaces Underscore Field Name Values and Converts to Proper Casing
ReplaceUnderscores = Table.TransformColumnNames(Source,
each Text.Proper(
Replacer.ReplaceText( _ , "_", " "
)))
in
ReplaceUnderscores
In the above let expression we transform our column names replacing underscores with a space and proper
casing with an each loop. The underscore character is used to iterate thru each item in our column headers
collection.
Full Solution: Replaces Underscores
let
Source = #table(
type table [CustomerID = Int64.Type, CustomerName = text, CustomerCity = text, CustomerState = text],
{
{1, "Bob Smith", "Rockford", "IL"},
…
}
),
/*
Splits Column Header Characters When Transitioning From Lower Case to Upper Case to a list of Values
Wrapped in a Text Combine to Combine all List Values to a Scalar Text Value
*/
SplitByCharacter = Table.TransformColumnNames(Source,
each Text.Combine(
Splitter.SplitTextByCharacterTransition({"a".."z"}, {"A".."Z"})(_)
, " ")
)
in
SplitByCharacter
In the above let expression we transform our column names splitting each text into a list by character by
transition whenever lower-cased characters in a list are followed by upper-cased characters in a list. The
split lists of text are combined including a space between each list item.
Full Solution: Split Text By Character Position
PERFORMANCE
BEATS PRETTY
100% OF THE TIME
50% OF THE TIME
BEHIND THE CURTAIN
1. xVelocity Engine, Run Length Encoding, Auto Sort.
2. You Can Get Kicked Out of Your Own Party.
3. Segmentation. Compression. Memory Consumption.
xVELOCITY ENGINE
DICTIONARY
ENCODING
COLUMN
ENCODING
TABLEColor Value
Red 10
Blue 15
Green 13
Green 25
Yellow 31
Yellow 41
Red 11
Blue 14
Purple 21
Orange 6
Red 15
Follow Along: Colors
DICTIONARYID Color Value
0 Red 10
1 Blue 15
2 Green 13
2 Green 25
3 Yellow 31
3 Yellow 41
0 Red 11
1 Blue 14
4 Purple 21
5 Orange 6
0 Red 15
Dictionary ID Color
0 Red
1 Blue
2 Green
3 Yellow
4 Purple
5 Orange
DICTIONARYID Color Value
0 Red 10
1 Blue 15
2 Green 13
2 Green 25
3 Yellow 31
3 Yellow 41
0 Red 11
1 Blue 14
4 Purple 21
5 Orange 6
0 Red 15
Dictionary ID Color
0 Red
1 Blue
2 Green
3 Yellow
4 Purple
5 Orange
COLUMN ENCODING
ID Value
0 10
1 15
2 13
2 25
3 31
3 41
0 11
1 14
4 21
5 6
0 15
Dictionary ID Color
0 Red
1 Blue
2 Green
3 Yellow
4 Purple
5 Orange
EVALUATION
ID Value
0 10
1 15
2 13
2 25
3 31
3 41
0 11
1 14
4 21
5 6
0 15
Dictionary ID Color Value (SUM)
0 Red 36
1 Blue 29
2 Green 38
3 Yellow 72
4 Purple 21
5 Orange 6
OPTIMAL SORT ORDER
ID Value
0 10
0 11
0 15
1 15
1 14
2 13
2 25
3 31
3 41
4 21
5 6
Dictionary ID Color
0 Red
1 Blue
2 Green
3 Yellow
4 Purple
5 Orange
RUN LENGTH ENCODING
ID Value
0 10
0 11
0 15
1 15
1 14
2 13
2 25
3 31
3 41
4 21
5 6
Dictionary ID Color Start Count
0 Red 1 3
1 Blue 4 2
2 Green 6 2
3 Yellow 8 2
4 Purple 10 1
5 Orange 11 1
Dictionary ID Color
0 Red
1 Blue
2 Green
3 Yellow
4 Purple
5 Orange
ID Value
0 10
0 11
0 15
1 15
1 14
2 13
2 25
3 31
3 41
4 21
5 6
EVALUATION
Dictionary ID Color Value (SUM)
0 Red 36
1 Blue 29
2 Green 38
3 Yellow 72
4 Purple 21
5 Orange 6
BEHIND THE CURTAIN
1. xVelocity Engine, Run Length Encoding, Auto Sort.
2. You Can Get Kicked Out of Your Own Party.
3. Segmentation. Compression. Memory Consumption.
CARDINALITY
1. High-cardinality: columns with values that are very
uncommon or unique.
• DateTime
• Unique IDs (Not Needed For Relationships)
• Text (Freeform Fields)
• Numbers (Floating Point Precision)
2. Low-cardinality: columns with relatively few unique values.
• Date (365 Days a Year)
• Time (86,400 Seconds)
• True / False
• Currency (4 Digits after Decimal)
BEHIND THE CURTAIN
1. xVelocity Engine, Run Length Encoding, Auto Sort.
2. You Can Get Kicked Out of Your Own Party.
3. Segmentation. Compression.
RESOURCE CONSUMPTION
1M 1M 1M … Model
Compressed
Segment
Compressed
Segment
Data Model
(Hierarchies,
Calculated Columns,
Relationships)
Compressed
Segment
Compressed
Segment
Compressed
Segment
…
CPU
MEMORY
CPU
SEGMENTATION
In the above Vertipaq Analyzer we review a DateTime field with high-cardinality. The Rows count is equal to
the Cardinality count and is loaded in an uncompressed state to the xVelocity Engine. At the rate of 1M rows
per segment, 9 uncompressed segments are loaded as a single partition into memory.
Full Solution: DateTime Series Analyzer
SEGMENTATION
In the above Vertipaq Analyzer we review a Time and Date field with low-cardinality. The Order Time field
contains 86,400 unique values (equal to the number of seconds in a day) and the Order Date 103 unique
values. At the rate of 1M rows per segment, 85 compressed segments are loaded as a single partition into
memory.
Full Solution: Date and Time Series Analyzer
COMPRESSION
8,900,608 ROWS 88,743,740 ROWS
TAKEAWAYS
1. Only bring in what you need.
2. DateTime = BAD; unless necessary.
3. Calculated Column = POST compression.
• Perform Calculations Close to the Source (SQL or within Power Query)
SWITCH ( TRUE(),
YOUR , BATTLES
)
END USE
1. Be cautious with preview features.
2. Don’t assume your audience can’t learn.
3. Consistency. Consistency. Consistency.
4. Don’t risk your brand over someone else’s need for a pie chart.
5. Champion modern analytics within your organization.
• By 2020, 50% of analytical queries either will be generated via search, natural language processing or voice, or will be
automatically generated. (Gartner)
CLOSING REMARKS
DELIVER BETTER THAN THE ASK.
QUESTIONS?
SUGGESTIONS?
YOU THINK I’M CRAZY?!
Connect Online
@notaboutthecell
/in/alexmpowers
It’s Not About The Cell

More Related Content

What's hot

Sql practise for beginners
Sql practise for beginnersSql practise for beginners
Sql practise for beginners
ISsoft
 
Structure query language (sql)
Structure query language (sql)Structure query language (sql)
Structure query language (sql)
Nalina Kumari
 
BP208 Fabulous Feats with @Formula
BP208 Fabulous Feats with @FormulaBP208 Fabulous Feats with @Formula
BP208 Fabulous Feats with @Formula
Kathy Brown
 
Intro to tsql unit 7
Intro to tsql   unit 7Intro to tsql   unit 7
Intro to tsql unit 7
Syed Asrarali
 

What's hot (20)

Introduction to SQL
Introduction to SQLIntroduction to SQL
Introduction to SQL
 
Sql ch 5
Sql ch 5Sql ch 5
Sql ch 5
 
Introduction to SQL (for Chicago Booth MBA technology club)
Introduction to SQL (for Chicago Booth MBA technology club)Introduction to SQL (for Chicago Booth MBA technology club)
Introduction to SQL (for Chicago Booth MBA technology club)
 
Sql wksht-2
Sql wksht-2Sql wksht-2
Sql wksht-2
 
Sql basics
Sql  basicsSql  basics
Sql basics
 
Sql practise for beginners
Sql practise for beginnersSql practise for beginners
Sql practise for beginners
 
Sql wksht-6
Sql wksht-6Sql wksht-6
Sql wksht-6
 
Introduction to SQL
Introduction to SQLIntroduction to SQL
Introduction to SQL
 
Structure query language (sql)
Structure query language (sql)Structure query language (sql)
Structure query language (sql)
 
BIS05 Introduction to SQL
BIS05 Introduction to SQLBIS05 Introduction to SQL
BIS05 Introduction to SQL
 
Sql Basics And Advanced
Sql Basics And AdvancedSql Basics And Advanced
Sql Basics And Advanced
 
Extensible Data Modeling
Extensible Data ModelingExtensible Data Modeling
Extensible Data Modeling
 
BP208 Fabulous Feats with @Formula
BP208 Fabulous Feats with @FormulaBP208 Fabulous Feats with @Formula
BP208 Fabulous Feats with @Formula
 
Sql
SqlSql
Sql
 
Sql
SqlSql
Sql
 
Sql ch 4
Sql ch 4Sql ch 4
Sql ch 4
 
Intro to tsql unit 7
Intro to tsql   unit 7Intro to tsql   unit 7
Intro to tsql unit 7
 
Sql wksht-1
Sql wksht-1Sql wksht-1
Sql wksht-1
 
PostgreSQL Tutorial for Beginners | Edureka
PostgreSQL Tutorial for Beginners | EdurekaPostgreSQL Tutorial for Beginners | Edureka
PostgreSQL Tutorial for Beginners | Edureka
 
Sql Basics | Edureka
Sql Basics | EdurekaSql Basics | Edureka
Sql Basics | Edureka
 

Similar to It's Not You. It's Your Data Model.

Physical elements of data
Physical elements of dataPhysical elements of data
Physical elements of data
Dimara Hakim
 
Sap abap interview questions
Sap abap interview questionsSap abap interview questions
Sap abap interview questions
kssr99
 
Statements,joins and operators in sql by thanveer danish melayi(1)
Statements,joins and operators in sql by thanveer danish melayi(1)Statements,joins and operators in sql by thanveer danish melayi(1)
Statements,joins and operators in sql by thanveer danish melayi(1)
Muhammed Thanveer M
 
Ms sql server ii
Ms sql server  iiMs sql server  ii
Ms sql server ii
Iblesoft
 
Microsoft excel training
Microsoft excel trainingMicrosoft excel training
Microsoft excel training
EmilyE120
 

Similar to It's Not You. It's Your Data Model. (20)

Maryna Popova "Deep dive AWS Redshift"
Maryna Popova "Deep dive AWS Redshift"Maryna Popova "Deep dive AWS Redshift"
Maryna Popova "Deep dive AWS Redshift"
 
Advanced MySQL Query Optimizations
Advanced MySQL Query OptimizationsAdvanced MySQL Query Optimizations
Advanced MySQL Query Optimizations
 
Model Assistant Suite
Model Assistant SuiteModel Assistant Suite
Model Assistant Suite
 
GDS2015 spreadsheet_design
GDS2015 spreadsheet_designGDS2015 spreadsheet_design
GDS2015 spreadsheet_design
 
Physical elements of data
Physical elements of dataPhysical elements of data
Physical elements of data
 
Introduction to DAX Language
Introduction to DAX LanguageIntroduction to DAX Language
Introduction to DAX Language
 
Sap abap interview questions
Sap abap interview questionsSap abap interview questions
Sap abap interview questions
 
Application sql issues_and_tuning
Application sql issues_and_tuningApplication sql issues_and_tuning
Application sql issues_and_tuning
 
Statements,joins and operators in sql by thanveer danish melayi(1)
Statements,joins and operators in sql by thanveer danish melayi(1)Statements,joins and operators in sql by thanveer danish melayi(1)
Statements,joins and operators in sql by thanveer danish melayi(1)
 
Ms sql server ii
Ms sql server  iiMs sql server  ii
Ms sql server ii
 
esProc introduction
esProc introductionesProc introduction
esProc introduction
 
Lect11
Lect11Lect11
Lect11
 
Physical Design and Development
Physical Design and DevelopmentPhysical Design and Development
Physical Design and Development
 
Excel.useful fns
Excel.useful fnsExcel.useful fns
Excel.useful fns
 
Steps towards of sql server developer
Steps towards of sql server developerSteps towards of sql server developer
Steps towards of sql server developer
 
Microsoft excel training
Microsoft excel trainingMicrosoft excel training
Microsoft excel training
 
Alteryx Presentation
Alteryx PresentationAlteryx Presentation
Alteryx Presentation
 
Lecture 13
Lecture 13Lecture 13
Lecture 13
 
SQL – The Natural Language for Analysis - Oracle - Whitepaper - 2431343
SQL – The Natural Language for Analysis - Oracle - Whitepaper - 2431343SQL – The Natural Language for Analysis - Oracle - Whitepaper - 2431343
SQL – The Natural Language for Analysis - Oracle - Whitepaper - 2431343
 
C#
C#C#
C#
 

Recently uploaded

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Recently uploaded (20)

[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptx
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 

It's Not You. It's Your Data Model.

  • 1. Not quite what I was expecting I went ahead and built my own report instead Missed the mark PAINFULLY SLOWDifficult to navigate I’ll never use those features Something like beforeI didn’t want you to change the way it’s always been done
  • 2. IT’S NOT YOU. IT’S YOUR DATA MODEL.
  • 3. ALEX M POWERS Microsoft Certified Solutions Associate: BI Reporting (Excel, Power BI) Microsoft Technology Associate: Python (Introduction to Programming) Microsoft Office Expert: Excel 2013, Excel 2010 Microsoft Office Specialist: Access 2013, Excel 2010 Microsoft #HowWeExcel Contest Winner Co-Organizer of the St Louis Power BI User Group #STLPBIUG Millennial Whisperer and Trainer | It’s Not About The Cell LLC: itsnotaboutthecell.com Confused by the term “Pair of Socks” Bad at Fortnite
  • 4. USER EXPERIENCE (UX) Refers to a person's emotions and attitudes about using a product, system or service. It includes the practical, experiential, affective, meaningful and valuable aspects of human–computer interaction and product ownership. Additionally, it includes a person’s perception of system aspects such as ease of use, efficiency and utility. Source (Wikipedia)
  • 5. SESSION OBJECTIVES 1. Readability allows for discovery. (Ease of Use) 2. Performance considerations. (Efficiency) 3. Trust the abilities of your end users to learn. (Utility)
  • 7. FIELD NAMES Type Example Camel Case orderDate Pascal Case OrderDate Snake Case ORDER_DATE
  • 8. let Source = #table( type table [CUSTOMER_ID = Int64.Type, CUSTOMER_NAME = text, CUSTOMER_CITY = text, CUSTOMER_STATE = text], { {1, "Bob Smith", "Rockford", "IL"}, {2, "Randy Savage", "St Louis", "MO"}, {3, "Lucy Davis", "Lansing", "MI"} } ), // Replaces Underscore Field Name Values and Converts to Proper Casing ReplaceUnderscores = Table.TransformColumnNames(Source, each Text.Proper( Replacer.ReplaceText( _ , "_", " " ))) in ReplaceUnderscores In the above let expression we transform our column names replacing underscores with a space and proper casing with an each loop. The underscore character is used to iterate thru each item in our column headers collection. Full Solution: Replaces Underscores
  • 9. let Source = #table( type table [CustomerID = Int64.Type, CustomerName = text, CustomerCity = text, CustomerState = text], { {1, "Bob Smith", "Rockford", "IL"}, … } ), /* Splits Column Header Characters When Transitioning From Lower Case to Upper Case to a list of Values Wrapped in a Text Combine to Combine all List Values to a Scalar Text Value */ SplitByCharacter = Table.TransformColumnNames(Source, each Text.Combine( Splitter.SplitTextByCharacterTransition({"a".."z"}, {"A".."Z"})(_) , " ") ) in SplitByCharacter In the above let expression we transform our column names splitting each text into a list by character by transition whenever lower-cased characters in a list are followed by upper-cased characters in a list. The split lists of text are combined including a space between each list item. Full Solution: Split Text By Character Position
  • 10. PERFORMANCE BEATS PRETTY 100% OF THE TIME 50% OF THE TIME
  • 11. BEHIND THE CURTAIN 1. xVelocity Engine, Run Length Encoding, Auto Sort. 2. You Can Get Kicked Out of Your Own Party. 3. Segmentation. Compression. Memory Consumption.
  • 13. TABLEColor Value Red 10 Blue 15 Green 13 Green 25 Yellow 31 Yellow 41 Red 11 Blue 14 Purple 21 Orange 6 Red 15 Follow Along: Colors
  • 14. DICTIONARYID Color Value 0 Red 10 1 Blue 15 2 Green 13 2 Green 25 3 Yellow 31 3 Yellow 41 0 Red 11 1 Blue 14 4 Purple 21 5 Orange 6 0 Red 15 Dictionary ID Color 0 Red 1 Blue 2 Green 3 Yellow 4 Purple 5 Orange
  • 15. DICTIONARYID Color Value 0 Red 10 1 Blue 15 2 Green 13 2 Green 25 3 Yellow 31 3 Yellow 41 0 Red 11 1 Blue 14 4 Purple 21 5 Orange 6 0 Red 15 Dictionary ID Color 0 Red 1 Blue 2 Green 3 Yellow 4 Purple 5 Orange
  • 16. COLUMN ENCODING ID Value 0 10 1 15 2 13 2 25 3 31 3 41 0 11 1 14 4 21 5 6 0 15 Dictionary ID Color 0 Red 1 Blue 2 Green 3 Yellow 4 Purple 5 Orange
  • 17. EVALUATION ID Value 0 10 1 15 2 13 2 25 3 31 3 41 0 11 1 14 4 21 5 6 0 15 Dictionary ID Color Value (SUM) 0 Red 36 1 Blue 29 2 Green 38 3 Yellow 72 4 Purple 21 5 Orange 6
  • 18. OPTIMAL SORT ORDER ID Value 0 10 0 11 0 15 1 15 1 14 2 13 2 25 3 31 3 41 4 21 5 6 Dictionary ID Color 0 Red 1 Blue 2 Green 3 Yellow 4 Purple 5 Orange
  • 19. RUN LENGTH ENCODING ID Value 0 10 0 11 0 15 1 15 1 14 2 13 2 25 3 31 3 41 4 21 5 6 Dictionary ID Color Start Count 0 Red 1 3 1 Blue 4 2 2 Green 6 2 3 Yellow 8 2 4 Purple 10 1 5 Orange 11 1 Dictionary ID Color 0 Red 1 Blue 2 Green 3 Yellow 4 Purple 5 Orange
  • 20. ID Value 0 10 0 11 0 15 1 15 1 14 2 13 2 25 3 31 3 41 4 21 5 6 EVALUATION Dictionary ID Color Value (SUM) 0 Red 36 1 Blue 29 2 Green 38 3 Yellow 72 4 Purple 21 5 Orange 6
  • 21. BEHIND THE CURTAIN 1. xVelocity Engine, Run Length Encoding, Auto Sort. 2. You Can Get Kicked Out of Your Own Party. 3. Segmentation. Compression. Memory Consumption.
  • 22. CARDINALITY 1. High-cardinality: columns with values that are very uncommon or unique. • DateTime • Unique IDs (Not Needed For Relationships) • Text (Freeform Fields) • Numbers (Floating Point Precision) 2. Low-cardinality: columns with relatively few unique values. • Date (365 Days a Year) • Time (86,400 Seconds) • True / False • Currency (4 Digits after Decimal)
  • 23. BEHIND THE CURTAIN 1. xVelocity Engine, Run Length Encoding, Auto Sort. 2. You Can Get Kicked Out of Your Own Party. 3. Segmentation. Compression.
  • 24. RESOURCE CONSUMPTION 1M 1M 1M … Model Compressed Segment Compressed Segment Data Model (Hierarchies, Calculated Columns, Relationships) Compressed Segment Compressed Segment Compressed Segment … CPU MEMORY CPU
  • 25. SEGMENTATION In the above Vertipaq Analyzer we review a DateTime field with high-cardinality. The Rows count is equal to the Cardinality count and is loaded in an uncompressed state to the xVelocity Engine. At the rate of 1M rows per segment, 9 uncompressed segments are loaded as a single partition into memory. Full Solution: DateTime Series Analyzer
  • 26. SEGMENTATION In the above Vertipaq Analyzer we review a Time and Date field with low-cardinality. The Order Time field contains 86,400 unique values (equal to the number of seconds in a day) and the Order Date 103 unique values. At the rate of 1M rows per segment, 85 compressed segments are loaded as a single partition into memory. Full Solution: Date and Time Series Analyzer
  • 28. TAKEAWAYS 1. Only bring in what you need. 2. DateTime = BAD; unless necessary. 3. Calculated Column = POST compression. • Perform Calculations Close to the Source (SQL or within Power Query)
  • 29. SWITCH ( TRUE(), YOUR , BATTLES )
  • 30. END USE 1. Be cautious with preview features. 2. Don’t assume your audience can’t learn. 3. Consistency. Consistency. Consistency. 4. Don’t risk your brand over someone else’s need for a pie chart. 5. Champion modern analytics within your organization. • By 2020, 50% of analytical queries either will be generated via search, natural language processing or voice, or will be automatically generated. (Gartner)