http://www.kpipartners.com/webinar-Performance-Tuning-Oracle-BI-Applications/ ... From a virtual event that discusses techniques that can be used to optimize performance of the Oracle BI Apps.
The BI Apps from Oracle present customers with a nice head start to getting their BI environment up and running. But for many customers, their user community demands lighting-fast speeds while running dashboards, reports and ad-hoc queries. Learn about some of the key techniques you can use to take the BI Apps to performance levels you didn’t think were possible.
The discussion begins with a conceptual understanding of why performance problems can exist and the counteracting design considerations. Special attention will be paid to the concept of a Performance Layer, describing what it is, what it is comprised of and how to build it. The presentation includes several real world examples of the significant performance gains that can be had from a Performance Layer.
Objective 1: Learn about the concept of a performance layer and what is involved with building one.
Objective 2: Understand the most important steps to improve the performance of your system.
3. ①Introducing The Performance Layer
②Building The Performance Layer
③Mapping Into Oracle BI
④Implementation Considerations
⑤Q&A
Agenda
Performance Tuning Oracle’s BI Applications 3
5. Targeted At Organizations
Who Have:
Large Data Volumes
Custom Tables & Data Sets
Aggressive Performance Targets
Questionable Design Extensions
Slow Hardware
5
Introducing The Performance Layer
Performance Tuning Oracle’s BI Applications
6. These performance concepts
are applicable to any BI system
Custom Built Systems
Custom Stars in OBIA
SQL Server
6
Introducing The Performance Layer
Performance Tuning Oracle’s BI Applications
7. Wider tables slow you down
Dashboards only need a few tables
Smaller is faster
7
Introducing The Performance Layer
Performance Tuning Oracle’s BI Applications
8. Eliminate conflicting priorities
Singular focus on performance
Peak performance starts with design
8
Introducing The Performance Layer
Performance Tuning Oracle’s BI Applications
9. Great performance requires perfect design for how it is used
Mandate a top-down approach to tuning your BI application
Specialized design for specialized usage
9
Introducing The Performance Layer
Performance Tuning Oracle’s BI Applications
10. Pre-built logic
Clean star models
Reduced data weight
Tables which match usage by Oracle BI
10
Introducing The Performance Layer
Performance Tuning Oracle’s BI Applications
Top-Down design yields…
11. 11Performance Tuning Oracle’s BI Applications
Introducing The Performance Layer >> Oracle’s View On Data Warehouse Architecture
12. Keep the BI Apps model
mostly as-is & add a
performance layer…
① Source data from BI Apps tables
② Bring only what you need
③ Denormalizations & pre-calculations
④ Use all database performance tools
*Optimize To Usage*
12
Performance
Mini-Fridge
BIAppsDataFridge
ETL
Introducing The Performance Layer
Performance Tuning Oracle’s BI Applications
13. 13Performance Tuning Oracle’s BI Applications
① The Performance Layer is industry
standard architecture
② Design is driven only by report
performance improvement
③ Travel light
④ No need to alter BI Apps or DW
Introducing The Performance Layer - Takeaways
15. ① Start with priority areas (select a Fact table)
② Identify the use cases (reports w/ prompts & data security)
③ Analyze resulting physical SQL
④ Try to tune the BI Apps model first!
15
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
16. ⑤ Prototype a new data model to match those
needs
⑥ Adjust SQL & benchmark (SQL handcrafting needed)
⑦ Map into Oracle BI & test (Unit & Regression)
⑧ Benchmark the Oracle BI report using
prototyped tables
16
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
17. 9. Build the tables using INFA & DAC - Complete
Oracle BI RPD mapping
10. Formal Regression Test
11. Deploy
12. Enjoy praise from users
17
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
18. Reduce I/O with extreme prejudice
• Tune the BI Apps model first! It may work for you with low effort
• Employ techniques to eliminate I/O wherever possible
• Partition Elimination, Compression, Indexes, Aggregates, Star Transformations
• Let the Performance Layer do the work, not the report query
• Follow the KISS principle: Use a simple and clean Star. No
Snowflakes!
• Ensure OBI is mapped properly and uses correct tables with perfect
SQL
• Favor a general approach as opposed to a case-by-case approach
• A rising tide lifts all boats
18
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
19. There are 4 kinds of tables in the Performance Layer:
1. Skinny Dimension and Fact tables
2. New Dimension tables
3. Mini-Dimension tables
4. Fact Aggregate tables
Built directly from the base BI Apps or DW tables
Goal: use these tables in as many reports as possible (80/20 rule)
Guiding principles and performance influences:
1. Application use cases drive the layer’s design
2. Use minimal data for the job at hand
3. Aggregate Fact data when needed
4. Denormalize dimensions to eliminate extra joins
5. Pre-Build calculations to eliminate extra joins
6. Pre-Split data sets based on logical usage
19
BI Apps or
DW Perf. Layer
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
20. 20
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
① Single column, local bitmap
indexes on all Fact table FKs
(_WIDs) and filter fields
(DELETE_FLG)
② Single column bitmap indexes
on all dimensional fields used in
any sort of prompt or report filter
③ Special composite B-Tree
indexes to assist Snowflaked
areas
④ Composite B-Tree indexes on
large dimensions for join backs
(for list reports)
Query Indexing (4 Types)
For all Fact tables of a reasonable
size (e.g., > 5M rows)
Usually partition on Month (Range
or Interval)
The Database can easily
eliminate the majority of the table
Allows for smaller, local indexes
Table Partitioning
Before Beginning:
Tune the OOTB Model
21. Skinny Tables are highly selective versions of the BI Apps or DW
tables
• “Horizontal Aggregation” – use only 10 columns vs. 100 from the base table
Both Dimensions and Facts
Very easy to build and use
Goal: Reduce Avg. Row Length to 1/5th - 1/20th original size
Include only the columns you will need for top-down reporting
analysis
• If you don’t need Customer Address, don’t include it
• Ignore Meta Data columns (e.g., INTEGRATION_ID, etc.)
Row sets are identical (1:1) with the base tables
• For Dimensions use the same ROW_WIDs - can be used with existing fact tables easily
21
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
22. Build using:
1. Create Table as Select (CTAS)
2. Insert /*+ APPEND */
3. Materialized Views
Compress the table
Use Parallel hints & options
Don’t forget partitions
Enhance the tables with
calculation logic
Database is very fast at these
operations – expect only a few
minutes for 100M rows
22
CREATE TABLE WC_ACCT_BUDGET_SF
COMPRESS NOLOGGING PARALLEL (DEGREE 8)
PARTITION BY RANGE(PERIOD_END_DT_WID)
INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
(PARTITION Part_01 VALUES LESS THAN
(20100101))
AS SELECT /*+ PARALLEL(F,8) */
F.PERIOD_END_DT_WID,
F.X_PERIOD_END_DT_WID,
F.COMPANY_ORG_WID,
F.GL_ACCOUNT_WID,
F.X_POSTED_TOTAL_AMT,
case when GL_D."GL_ACCOUNT_NUM" =
'S250' then F."X_POSTED_TOTAL_AMT" end
as PLAN_CASES,
FROM W_ACCT_BUDGET_F F,
W_GL_ACCOUNT_D GL_D
WHERE F.GL_ACCOUNT_WID = GL_D.ROW_WID;
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
23. Enhance _SF tables with logic
Identify CASE WHEN statements
which require other dimensions
• Potential great benefit if the join can be
eliminated
• Don’t over do it – table will get less skinny
with each column
Identify any data set splitting from
the RPD
• HR Workforce Events table has both
Events and Snapshots records but they
are always used separately in the RPD
• Usage drives design: Split them out!
• Huge benefit for Event counting metrics
(~10% of table)
23
case when
GL_D."GL_ACCOUNT_NUM" = 'S250'
then F."X_POSTED_TOTAL_AMT" end
as PLAN_CASES,
FROM W_ACCT_BUDGET_F F,
W_GL_ACCOUNT_D GL_D
WHERE F.GL_ACCOUNT_WID =
GL_D.ROW_WID;
Create table
WC_WRKFC_EVT_EVENTS_SF …
… WHERE SNAPSHOT_IND = 0
Create table
WC_WRKFC_EVT_MONTH_SNP_SF …
… WHERE SNAPSHOT_IND = 1
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
24. 24
Real Examples I/O Benefit
Sub Ledger
(custom)
24X
Workforce Snap 11X
Workforce Events 56X
GL Balance 21X
Acct Budget 32X
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
Typical to get a 10X to 20X and even
50X I/O benefit in the _SF vs. the base
_F in size
Reduced AVG_ROW_LEN
COMPRESSION
Record Set Splitting
All without any aggregation
Skinny Dimensions also have benefits:
1. All I/O is a killer and slows down the entire system
2. De-normalize into a Star (eliminate snowflakes & outer joins)
3. Real World Ex, #1: Swap out 2 wide dims for 2 skinny dims improved query
time by 6X
4. Real World Ex. #2: One query going from 11s to 4s with one skinny dim
5. Real World Ex. #3: GL Account Dimension: 37X I/O benefit
25. Pre-build major pieces of commonly used but complex logic into the
Data Model
• Over-relying on the RPD or Reports for logic can harm performance
• Let the ETL for the Performance Layer do the work not the query
Example #1: Large binning and bucketing
CASE WHEN FACT.ORDER_AMT BETWEEN 0 and 100 THEN ‘0-100’ ELSE CASE WHEN FACT.ORDER_AMT
BETWEEN 101 and 200 THEN ‘101-200’ … END
• Build a new dimension table to hold these values –
WC_CUST_ORDER_QTY_BAND_D
Example #2: Date format conversions – dynamically building a new
column with a string concatenation statement:
substring(T66755."PER_NAME_MONTH" , 1, 4) , '') + '-' +
isnull(right(T66755."PER_NAME_MONTH" , 2) , '')
• Build a new column in the W_DAY_D table & index it
25
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
26. Simply a higher level or levels of
a larger dimension
• A combination of several Kimball
concepts
• Granularities will be mixed
Make a new table from the
large, base dimension
• Contains distinct combinations
• Use only commonly used fields
• Get cues from dashboard
prompts, column selectors, report
filters
Create a new ROW_WID
Compress and index as normal,
Parallel if needed for creation
Easy to build and map
26
Create table WC_EMPLOYEE_MD
COMPRESS as
select
ROWNUM AS ROW_WID,
W_ETHNIC_GRP_DESC,
WC_RACE_ETHNIC_DIVRSE_GRP_DESC,
W_SEX_MF_CODE, W_SEX_MF_DESC,
WC_NON_EMPLOYEE_VENDOR_NAME
from (
select distinct
W_ETHNIC_GRP_DESC,
WC_RACE_ETHNIC_DIVRSE_GRP_DESC,
W_SEX_MF_CODE,
W_SEX_MF_DESC,
WC_NON_EMPLOYEE_VENDOR_NAME
from W_EMPLOYEE_D);
This real world example created 5,400
records from a W_EMPLOYEE_D of 9+
Million rows.
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
27. _MD tables are used in the Performance Layer in two places:
1. Link into Skinny Facts
• Use a separate FK in addition to the base _WID
• Fact table has both EMPLOYEE_WID and EMPLOYEE_MD_WID
Thus the _SF can join to all of the following:
• New Mini Dimension (_MD) (~1% rows, some columns)
• New Skinny Dimension (_SD) (100% rows, some columns)
• Base BI Apps/DW Dimension (_D) (100% rows, 100% columns)
The OBI RPD can select which one is best for each query
Benefits of linking into the _SF
1. The same set of fact rows are selected – no benefit
2. Reduced dimension I/O, CPU and buffer space
3. Faster join-back on list reports
4. Very fast prompts, especially when constrained
27
_D
_SD
_MD
Conceptual
Size
Difference
s
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
28. 2. Use them for very high level Fact Aggregates
• Build a Fact aggregate at the Mini-Dimension level
Allows greater field inclusion at no expense to
aggregation compression ratios
• Multiple fields are available - not just one
A good Mini Dimension and Skinny Fact/Aggregate
can serve a large % of dashboard queries
MD’s & Fact Aggregates offer extreme performance:
1. Real World Ex #1: From time-out after 10 minutes to 4 seconds
2. Real World Ex #2: GL Account MD: 131X I/O benefit
28
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
29. Aggregates are used when summary reports exist
Pre-aggregate the dataset to make it smaller & faster
Sometimes they are the only solution
Typically a minimum of a 10:1 ratio is used
Use all database tools as with any fact table
• Partitioning, Indexing, Star Transformations, Compression
For extreme needs, consider merging facts together
• Ex: Monthly Actuals and Budgets
• Be mindful of gaps in datasets and non-conformed dimensions
Advanced implementations use partition management to
build only changed data for faster load times
29
Building The Performance Layer
Performance Tuning Oracle’s BI Applications
30. 30Performance Tuning Oracle’s BI Applications
① Building the underlying tables is relatively simple
② Take cues from dashboard, report & RPD configuration
③ Any savings in I/O helps the overall system
④ Use all of the available database performance tools
⑤ Tremendous benefits are possible with a few tables
Building The Performance Layer- Takeaways
32. Link as much as possible to allow for the best performance across all
scenarios
32
BI Apps
Performance
Layer
Best
Better
Base
Performance Tuning Oracle’s BI Applications
Mapping Into Oracle BI
33. The 3 Fact tables are mapped
like any aggregate
The Skinny Fact (_SF) will
have fewer dimensions and
fewer metrics mapped to it
• Along the Employee dimension however it
is the same as the base _F
OBI will prefer to use the _SF
over the _F
• Uses regular aggregate navigation
concepts
• Uses the _F when needed as a “backup
plan”
33
_A
_SF
_F
Performance Tuning Oracle’s BI Applications
Mapping Into Oracle BI
34. Raise the priority group on the base _D to have
OBI prefer the _SD
As both LTS grains are identical, OBI needs more info to make a
choice
34
_D _SD
Performance Tuning Oracle’s BI Applications
Mapping Into Oracle BI
35. Create a dummy hierarchy level and map the LTSs for the Mini
Dimension and Fact Aggregate to it
The grain of the Mini Dimension is arbitrary
As long as OBI knows it is higher than the other LTSs it will
be preferred (Priority groups not needed)
35
_MD
_A
Performance Tuning Oracle’s BI Applications
Mapping Into Oracle BI
36. 36Performance Tuning Oracle’s BI Applications
Table Mapping
The mapping of tables is straightforward
Link Tables As Much As Possible
Let Oracle BI make the best choice
Mapping Into Oracle BI - Takeaways
38. The whole prototyping process can be done on a
simple star in roughly two weeks
Allow for more time if you have:
• Large data volumes
• Difficult performance targets
• More complex models and logic
• Many disparate report patterns or lots of reports to consider
• More stars are needed (e.g., Actuals and Budgets together)
Development effort depends on # new objects
• Typically only another two weeks needed (ETL & OBI RPD)
• A few more for regression test and deployment
38Performance Tuning Oracle’s BI Applications
Implementation Considerations
39. Use Production data volumes for
accurate analysis
Use Production DDL, ETL code,
OBI RPD and OBI Webcat
Quiet, unused machine for
accurate benchmarking
Use hardware that is as similar to
Prod as possible
• KPI uses a database benchmarking tool to
compare environments
39Performance Tuning Oracle’s BI Applications
Implementation Considerations
40. Additional ETL & RPD Development
• Use SQL scripts instead of Informatica mappings (less effort, faster
execution)
Additional Testing – Regression test is easy
Additional ETL Run Time – may be critical
Additional Database size - minor
Customization Propagation / Impact Analysis
• True of any aggregate
Complex logic will be more difficult
• Financial Analytics - snowflake with multiple segment hierarchies
• HR Workforce Event & Snapshot logic uses effective dates for future
dated events
40Performance Tuning Oracle’s BI Applications
Implementation Considerations
42. www.kpipartners.com
The Leader In Oracle BI & EPM 42
Strategic Consulting | Systems Implementation | Training
Depot Repair Analytics
Fixed Asset Analytics
Manufacturing Analytics
Salesforce.com Analytics
Student Info Analytics
Subledger (SLA) Analytics
and more…
Transform Data Into Insight
Staff built from
Oracle/Siebel/Hyperion
engineering teams
On-site, off-shore and blended
shore delivery models
Exclusive pre-built solutions
for Oracle BI & E-Business
Suite
Oracle BI
Hyperion
Endeca
Exalytics
43. Email: info@kpipartners.com
Web: kpipartners.com/contact
KPI World Headquarters
39899 Balentine Drive
Suite #375
Newark, CA 94560
Phone: (510) 818-9480
Contact Us
The Leader In Oracle BI & EPM 43
New York, NY
Chicago, IL
Boston, MA
Minneapolis, MN
San Diego, CA
Greensboro, NC
North America Offices
Bangalore, India Hyderabad, India
Global Offices
Wider tables slow you down:
Large and wide tables carry more fields than you need
You usually only need a few of them for dashboards
Smaller is faster
Peak performance starts with design
Eliminate conflicting priorities
ETL development effort
ETL load times
Overly simplistic model
Overly complex model
Singular focus on performance
Achieving great “performance” requires a design that works perfectly with how it is used
Same concept as any transactional system
Sports car or a travel coffee mug
This mandates a pure top-down approach to tuning your BI application
Top-Down design yields:
Reduced data weight
Tables which match usage by OBI
Clean Star Model
Prebuilt logic
Specialized design for specialized usage yields optimal performance
Achieving great “performance” requires a design that works perfectly with how it is used
Same concept as any transactional system
Sports car or a travel coffee mug
This mandates a pure top-down approach to tuning your BI application
Top-Down design yields:
Reduced data weight
Tables which match usage by OBI
Clean Star Model
Prebuilt logic
Specialized design for specialized usage yields optimal performance
Oracle published their latest DW Reference Architecture in 2010
It has a clear understanding of the performance limitations of a DW for BI Reporting
It identifies the need for a Performance Layer
Top Down design will force too many changes to the OOB Model
Keep the BI Apps model mostly as it is
Add a Performance Layer:
Source data from the BI Apps tables
Bring only what you need
Improve the model with denormalizations and pre-calculations
Use all of the database performance tools available to you
Optimize to usage!
The Performance Layer is an industry standard architecture
The Performance Layer design is driven only by reporting performance improvements
Travel light by only bringing needed data
You don’t need to alter your BI Apps or DW system
The Leader In Oracle BI & EPM
KPI Partners is the Most Experienced Oracle BI & EPM Systems Implementation Partner
KPI Partners is an Oracle Platinum Partner who specializes in Oracle Business Intelligence (BI) and Oracle Enterprise Performance Management solutions. The award-winning staff at KPI Partners comes directly from the product engineering departments at Oracle, Siebel, and Hyperion. In addition to consulting services, KPI Partners offers training, support, and exclusive pre-packaged analytic solution extensions for Oracle Business Intelligence.
KPI Partners works with both corporate technology departments and corporate business units to develop value-added business intelligence solutions, not just new technology deployments.
Industry Expertise
We deliver enterprise technology solutions through the skill, experience, and power of our team. Our company experience is rooted within the origin of business intelligence and enterprise decision support technology.
Results Driven
We are driven to make a difference. From fundamental concepts to application, we deliver methods, tools, and technologies to help drive enterprise reporting and insight.
Client Focused
We provide the vision, technology, and leadership our clients need for success. We deliver results on time and on budget. Our clients rely on our expertise to help them define their future.
Innovation
We continue to drive the creation of new technology, best practices, and thought-leadership within the space.
World Class
We are recognized as leaders in enterprise-level business intelligence technology. With reputation comes responsibility, and KPI Partners strives to provide white glove, 5-star-level, service and support.