Oracle Database In-Memory and the Query Optimizer

2014 © Trivadis
BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA
Oracle Database In-Memory and the Query Optimizer
UKOUG Tech14, Liverpool (GB) Christian Antognini
11 December 2014
1

2014 © Trivadis
11 December 2014
2
@ChrisAntognini
■Senior principal consultant, trainer and partner at Trivadis in Zurich (CH)
■christian.antognini@trivadis.com
■http://antognini.ch
■Focus: get the most out of Oracle Database
■Logical and physical database design
■Query optimizer
■Application performance management
■Author of Troubleshooting Oracle Performance (Apress, 2008/2014)
■OakTable Network, Oracle ACE Director

2014 © Trivadis
1.In-Memory Column Store
2.In-Memory Aggregation
11 December 2014
3
AGENDA

2014 © Trivadis
11 December 2014
In-Memory Column Store
4

2014 © Trivadis
11 December 2014
5
Architecture Overview

2014 © Trivadis
■Data is populated into the IMCS in chunks called IM Column Unit (IMCU)
11 December 2014
6
IM Column Unit
…
C1
C2
C3
C1
C2
C3
C1
C2
C3

2014 © Trivadis
■They are similar to Exadata storage indexes
■Do not store information about NULL values
■They are automatically created and maintained for each column inside the IMCS
■They are only stored in memory
■They keep track of the minimum and maximum values in each IMCU
■Their aim is to avoid accessing IMCUs that do not contain relevant data
11 December 2014
7
IM Storage Indexes

2014 © Trivadis
■The cost model has been enhanced to be IM aware
■It requires IM statistics
■The total cost is a combination of both non-IM costs and IM costs
■Enhancements are disabled if OPTIMIZER_INMEMORY_AWARE = FALSE or OPTIMIZER_FEATURES_ENABLE < 12.1.0.2
■IMCS is not disabled
■Costs are based on the former cost model
11 December 2014
8
IM-Aware Query Optimizer

2014 © Trivadis
■The following statistics are used by the query optimizer:
■Number of IMCUs
■Number of blocks with data into the IMCUs
■Number of rows stored into the IMCUs
■Fraction of the rows stored into the IMCUs
-Value between 0 and 1
■Number of rows stored into the transaction journal
■IM statistics are computed on the fly during hard parses
■A cursors is not invalidated when the IM statistics it is based on changes
11 December 2014
9
IM Statistics

2014 © Trivadis
■Two operations are related to IM scans:
■TABLE ACCESS INMEMORY FULL
■MAT_VIEW ACCESS INMEMORY FULL
■Also used in case of Exadata storage
■IM scans are selected by the query optimizer regardless of whether data is actually stored into the IMCS
■The current behavior seems buggy to me
11 December 2014
10
IM Scans

2014 © Trivadis
■Pruning based on storage indexes
■Pruning based on compression metadata
■Dictionary-based compression algorithms store metadata that contain a list of distinct values for each column within IMCU
■Evaluation against compressed data is possible
■Except for CAPACITY LOW/HIGH
■Evaluation takes advantage of SIMD
■Execution plans provide a new type of predicate: INMEMORY
■Also used in case of predicates offloaded to Exadata storage
11 December 2014
11
Predicate Evaluation Taking Place During IM Scans

2014 © Trivadis
■The query optimizer can generate predicates that are evaluated during IM scans
■The generated predicates are used as explicit predicates
11 December 2014
12
Generation of Implied Predicates

2014 © Trivadis
11 December 2014
13
IM Scan Example
SELECT * FROM t WHERE id = 1 OR id > 1 AND n < 42
-------------------------------------------
| Id | Operation | Name |
-------------------------------------------
| 0 | SELECT STATEMENT | |
|* 1 | TABLE ACCESS INMEMORY FULL| T |
-------------------------------------------
1 - inmemory("ID">=1 AND ("N"<42 AND "ID">1 OR "ID"=1))
filter("N"<42 AND "ID">1 OR "ID"=1)
Implied predicate

2014 © Trivadis
■The INMEMORY_QUERY initialization parameter enables or disables IM scans at the session or system level
■IM scans can also be controlled through the (NO_)INMEMORY hints
■The hints override the INMEMORY_QUERY parameter
11 December 2014
14
Controlling IM Scans

2014 © Trivadis
■Based on Bloom filters
■Similar concept than parallel joins in previous versions, but executed serially
■Filter created by the left input
■Filter used by the right input
■Only available for hash joins
■Only the probe input have to be “IM-enabled”
■The join is used to filter the false positives
■IM joins can be controlled through the (NO_)PX_JOIN_FILTER hints
11 December 2014
15
IM Joins

2014 © Trivadis
11 December 2014
16
IM Join Example
SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n < 1E3
------------------------------------------------
------------------------------------------------
|* 1 | HASH JOIN | |
| 2 | JOIN FILTER CREATE | :BF0000 |
|* 3 | TABLE ACCESS FULL | T1 |
| 4 | JOIN FILTER USE | :BF0000 |
|* 5 | TABLE ACCESS INMEMORY FULL| T2 |
------------------------------------------------
1 - access("T1"."ID"="T2"."ID")
3 - filter("T1"."N"<1E3)
5 - inmemory(SYS_OP_BLOOM_FILTER(:BF0000,"T2"."ID"))
filter(SYS_OP_BLOOM_FILTER(:BF0000,"T2"."ID"))

2014 © Trivadis
■Table expansion is a cost-based transformation introduced in 11.2 to leverage partially unusable indexes
■It has been made IM aware
■Useful in case not all partitions are “IM-enabled”
■It can be controlled through the (NO_)EXPAND_TABLE hints
11 December 2014
17
IM Table Expansion

2014 © Trivadis
11 December 2014
18
IM Table Expansion Example
SELECT count(d) FROM t WHERE d BETWEEN :b1 AND :b2
---------------------------------------------------------------------------
| Operation | Name | Pstart| Pstop |
---------------------------------------------------------------------------
| SELECT STATEMENT | | | |
| SORT AGGREGATE | | | |
| VIEW | VW_TE_1 | | |
| UNION-ALL | | | |
| PARTITION RANGE SINGLE | | 11 | 11 |
| TABLE ACCESS INMEMORY FULL | T | 11 | 11 |
| PARTITION RANGE SINGLE | | 12 | 12 |
| TABLE ACCESS BY LOCAL INDEX ROWID BATCHED| T | 12 | 12 |
| INDEX RANGE SCAN | I | 12 | 12 |
---------------------------------------------------------------------------
The IM scan is only enabled for the partition 4

2014 © Trivadis
■The plan hash value is independent from the kind of FTS
■TABLE ACCESS FULL
■TABLE ACCESS STORAGE FULL
■TABLE ACCESS INMEMORY FULL
■Features like SQL plan baselines do not break when IMCS is enabled
■The “best” execution plan can change
11 December 2014
19
Plan Hash Value

2014 © Trivadis
■_INMEMORY_QUERY_SCAN controls IM scans (TRUE)
■_OPTIMIZER_INMEMORY_ACCESS_PATH controls whether the query optimizer costs for IM (TRUE)
■_OPTIMIZER_INMEMORY_BLOOM_FILTER controls IM joins (TRUE)
■_OPTIMIZER_INMEMORY_GEN_PUSHABLE_PREDS controls the generation of implied predicates (TRUE)
■_OPTIMIZER_INMEMORY_TABLE_EXPANSION controls IM table expansion (TRUE)
11 December 2014
20
Undocumented Parameters

2014 © Trivadis
11 December 2014
In-Memory Aggregation
21

2014 © Trivadis
■To optimize queries against a star schema, the query optimizer should do the following:
■Start evaluating each dimension table that has restrictions on it
■Assemble a list with the resulting dimension keys
■Use this list to extract the matching rows from the fact table
■This approach cannot be implemented with regular joins
■The query optimizer can join only two data sets at one time
■Joining two dimension tables leads to a Cartesian product
■To solve this problem, Oracle Database implements two query transformations:
■Star transformation
■Vector transformation (new in 12.1.0.2 – requires the In-Memory option)
11 December 2014
22
The Star Schema Challenge

2014 © Trivadis
■In the documentation Oracle refers to it as In-Memory Aggregation
■It isn’t only about aggregation
■It requires the In-Memory option because it takes advantage of some of its features
■INMEMORY_SIZE must be greater than 0
■The query optimizer considers it when
■equijoins are used
■an aggregate function is applied to a column of the fact table
■the query contains a GROUP BY clause
-ROLLUP, CUBE and GROUPING SETS are not yet supported
11 December 2014
24
Vector Transformation (1)

2014 © Trivadis
■It’s a cost-based query transformation
■It can be controlled through the (NO_)VECTOR_TRANSFORM hints
■As with the star transformation, sometimes is not possible to force it
■It introduces new row source operations
■KEY VECTOR CREATE
■KEY VECTOR USE
■VECTOR GROUP BY
11 December 2014
25
Vector Transformation (2)

2014 © Trivadis
1.For every dimension table with a filter on it the following operations take place
■Access table and filter data
■Create key vector
■Aggregate data
■Create temporary table
2.Access the fact table through a FTS and filter data by applying the key vectors
■On Exadata the filter is not yet offloaded
3.Aggregate data with either a vector or hash aggregation
4.Join data from the fact table to temporary tables
5.Join dimension tables without a filter on them
11 December 2014
26
Vector Transformation – Processing Steps

2014 © Trivadis
11 December 2014
27
Example
SELECT c.customer_class, sum(o.order_total)
FROM orders o, customers c, warehouses w
WHERE o.customer_id = c.customer_id
AND o.warehouse_id = w.warehouse_id
AND c.dob BETWEEN :b1 AND :b2
AND w.warehouse_name BETWEEN :b3 AND :b4
GROUP BY c.customer_class
customers
orders
warehouses

2014 © Trivadis
11 December 2014
28
Execution Plan – Access Dimension Tables, Create Key Vectors, and Create Temporary Tables
-------------------------------------------------------------------------
-------------------------------------------------------------------------
| 1 | TEMP TABLE TRANSFORMATION | |
| 2 | LOAD AS SELECT | SYS_TEMP_0FD9DE69C_28565764 |
| 3 | VECTOR GROUP BY | |
| 4 | KEY VECTOR CREATE BUFFERED | :KV0000 |
|* 5 | TABLE ACCESS STORAGE FULL | CUSTOMERS |
| 6 | LOAD AS SELECT | SYS_TEMP_0FD9DE69D_28565764 |
| 8 | HASH GROUP BY | |
| 9 | KEY VECTOR CREATE BUFFERED | :KV0001 |
|* 10 | TABLE ACCESS STORAGE FULL | WAREHOUSES |
...

2014 © Trivadis
11 December 2014
29
Execution Plan – Access Fact Table by Applying Key Vectors, and Join Temporary Tables
...
|* 12 | HASH JOIN | |
| 13 | MERGE JOIN CARTESIAN | |
| 14 | TABLE ACCESS STORAGE FULL | SYS_TEMP_0FD9DE69D_28565764 |
| 15 | BUFFER SORT | |
| 16 | TABLE ACCESS STORAGE FULL | SYS_TEMP_0FD9DE69C_28565764 |
| 17 | VIEW | VW_VT_72AE2D8F |
| 20 | KEY VECTOR USE | :KV0001 |
| 21 | KEY VECTOR USE | :KV0000 |
|* 22 | TABLE ACCESS STORAGE FULL| ORDERS |
-------------------------------------------------------------------------
...
22 - filter(SYS_OP_KEY_VECTOR_FILTER("O"."CUSTOMER_ID",:KV0000) AND
SYS_OP_KEY_VECTOR_FILTER("O"."WAREHOUSE_ID",:KV0001))

2014 © Trivadis
Core Messages
■In-Memory Column Store
■The query optimizer is IM aware
■In-Memory Aggregation
■Very interesting concept, complements the star transformation
11 December 2014
30

2014 © Trivadis
Questions and answers ...
BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA
Christian Antognini
Principal Senior Consultant
christian.antognini@trivadis.com
11 December 2014
31

Oracle Database In-Memory and the Query Optimizer

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Oracle Database In-Memory and the Query Optimizer

Semelhante a Oracle Database In-Memory and the Query Optimizer (20)

Último

Último (20)

Oracle Database In-Memory and the Query Optimizer