Dmm300 – mixed scenarios for sap hana data warehousing and BW: overview and experiences

Public
DMM300 – Mixed Scenarios for SAP HANA Data
Warehousing: Overview and Experiences

© 2016 SAP SE or an SAP affiliate company. All rights reserved. 2Public
Speakers
Bangalore, October 5 - 7
Edwin Dayanandh
Las Vegas, Sept 19 - 23
Sebastian Baumgärtner
Barcelona, Nov 8 - 10
Oliver Schall

Disclaimer
The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of
SAP. Except for your obligation to protect confidential information, this presentation is not subject to your license agreement or
any other service or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this
presentation or any related document, or to develop or release any functionality mentioned therein.
This presentation, or any related document and SAP's strategy and possible future developments, products and or platforms
directions and functionality are all subject to change and may be changed by SAP at any time for any reason without notice.
The information in this presentation is not a commitment, promise or legal obligation to deliver any material, code or functionality.
This presentation is provided without a warranty of any kind, either express or implied, including but not limited to, the implied
warranties of merchantability, fitness for a particular purpose, or non-infringement. This presentation is for informational
purposes and may not be incorporated into a contract. SAP assumes no responsibility for errors or omissions in this
presentation, except if such damages were caused by SAP’s intentional or gross negligence.
All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially
from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only
as of their dates, and they should not be relied upon in making purchasing decisions.

Agenda
Introduction / Motivation for Mixed Case Scenarios
Overview of Mixed Scenario Architectures
Experiences with:
 BW on SAP HANA Transformations
 Generated SAP HANA Views
 Integrating SAP HANA Views

Public
Introduction / Motivation

SAP HANA
Mixed Architecture with SAP BW and SAP HANA
Definition
SAP’s best practice for modern data warehousing
A Mixed Architecture consists of an overall data model that is
implemented at the same time by BW and native SAP HANA
tools
 It combines processes, data and metadata of BW and SAP
HANA native (= best-of-both-worlds to gain additional
insight and flexibility)
 SAP HANA and SAP BW deliver integrated tools to
manage mixed scenarios end-to-end including modeling,
transport mechanisms and consumption interfaces.
 SAP HANA Database as central runtime for reporting and
data warehouse processes offering best in class
performance and scalability for large data warehouses
SAP BW
Sources
BI Clients
SAP HANA
Modeling
BW Modeling
<
Mixed Architecture
SAP HANA Data Warehousing: Models
for SAP BW and SQL DW on SAP HANA
DMM302 (L1)
i

Mixed Case Scenarios
In addition to “classic” BW scenarios, pure native modeling capabilities are now available as well as
BW powered by SAP HANA (BW optimized with SAP HANA features, e.g. BW on SAP HANA
Transformations)
More possibilities / larger flexibility
New know-how / guidelines / best practices required
This session aims to provide insight / experiences how to approach modeling and shows different
options

Mixed Case Scenarios
Application requirements and the existing landscape are drivers for a new architecture
and its focus area
We will introduce selected options in an overview before showing best practices for detailed
implementation aspects

Architecture Overview
Staging Capabilities
SAP BW provides SAP HANA enabled
Transformations with optimizations for
standard functionality as well as the ability to
use SAP HANA Expert Script (“SQL Script
Exit”)
SAP HANA
SAP BW
Repository
Sources
Data Warehousing
SAP HANA Views
SQLScript
Data Acquisition

Generated SAP HANA Views for BW Info
Providers
Neglects reporting layer and does use BWs
staging capabilities
Often driven by client decision which does not
have a tight integration with BW backend (BW
query, BICS)SAP HANA
SAP BW
Repository
Sources
BI Clients
Data Warehousing
SAP
HANA
Views
gen.
SAP
HANA
Views
SAP
HANA
Views
Data Acquisition

Generated SAP HANA Views for BW
Queries
Alternative to previous scenario with usage of
analytic engine
Re-use of certain query properties possible
(e.g. restricted / calculated keyfigures) for BI
clients with shallow BW integrationSAP HANA
SAP BW
Repository
Sources
Data Warehousing
gen.
SAP
HANA
Views
Data Acquisition
Analytic Engine
BW Query BW Query
gen.
SAP
HANA
Views
Workspaces
BI Clients

Integration of modeled SAP HANA Views
from agile scenarios
Replicated and processed data from a
separate schema is integrated into BW
SAP HANA view is consumed in a Composite
Provider / Open ODS View and mixed with
BW dataSAP HANA
SAP BW
Repository
Sources
BI Clients
Data Warehousing
Data Acquisition
Analytic Engine
BW Query
Composite Provider
SAP
HANA
Views

Integration of modeled SAP HANA Views
from agile scenarios
Data management completely native
Integration into BW to use feature-set of
analytic engine
SAP HANA
SAP BW
Repository
Sources
BI Clients
Analytic Engine
BW Query
Composite Provider
SAP
HANA
Views

Public
BW on SAP HANA Transformations
SAP HANA
SAP BW
Repository
Sources
Data Warehousing
SAP HANA Views
SQLScript
Data Acquisition

Why to use Transformations with SAP HANA Execution?
SAP HANA enabled transformations avoid data roundtrips between application server and database
Better parallelization inside the SAP HANA database
Better performance / more flexibility (adaptation of SAP HANA features)

How do Transformations with SAP HANA Execution work?
After the design of the transformation capability which processing mode is possible
Similar to generated program for ABAP the generated SAP HANA Transformation can be displayed

SAP HANA Transformation
(SAP HANA Analysis Process)
CalculationScenario

Prerequisites for Transformations with SAP HANA Execution
Supported targets: Standard + Write-optimized DSO, Advanced DSO (ADSO), SPOs based on DSOs, OpenHub destinations for DB tables or 3rd
party tools
Queries as Info Providers are not supported as a source
ABAP routines are not supported (field routine, characteristic routine*, start routine, end routine, expert routine)
Rule groups are not supported
Read from DSO  entire key must be specified
Near-line connections are not supported
*0SOURSYSTEM and 0LOGSYS are supported

The DTP combines Transformations between two Info Providers with persistence.
SAP HANA Expert Script
0BW_OPER_PROC
Standard
0BW_PROJECTION
Standard
0BW_PROJECTION
HAP
TR_00O2TMRZFVSBOQ6FLWQERAEAH
HAP
TR_00O2TMRZFVSBOQ6FLWQERAEAH2
HAP
TR_00O2TMRZFVSBOQ6FLWQERAEAH1
DTP_00O2TMRZFVSBOQ6FLWQERAEAH

The calculation scenarios for the related transformations are stacked.

During the SAP HANA Execution of a DTP (availability can be checked) an package-wise “insert as
select” statement is generated

Prerequisites for SAP HANA Execution Availability of a DTP
No semantic grouping
Data target is DSO  “subsequent processing without master data” is selected
Data target is Open Hub  data target is database table or 3rd
party tool
Data source is DSO  Delta Init from Active Table (with Archive) not set
Data source is BW DataSource  Extraction is performed from PSA table

Novel BW 7.50 SP04 features
In data flows with multiple transformations SAP HANA Expert Script and ABAP Transformations can be mixed (SAP HANA Expert Script must be first
transformations)
Data Target with SIDs are also supported (e.g. Info Cube, Info Object)
Start-, End- and Field routines can be implemented in SQL Script
Intermediate results can be displayed in the DTP simulation mode
PERI7 conversion exit is supported
DTP with error handling is supported
ADSO with NLS-IQ archive is supported as a source

BW on SAP HANA Transformations – Debugging I
Initialize Debugging
To debug a SAP HANA Script it
is necessary to execute the
DTP in the Execution mode
(not in the simulation mode)
Initialize Debugging
To debug a SAP HANA Script it
is necessary to execute the
DTP in the Execution mode
(not in the simulation mode)
Debug Perspective
To debug the procedure it is
necessary to switch to the
Debug Perspective
Debug Perspective
To debug the procedure it is
necessary to switch to the
Debug Perspective

BW on SAP HANA Transformations – Debugging II
Call Stack Information
 Current processing line number
Call Stack Information
 Current processing line number
Source Code
 Current processing source code
 Add / Delete breakpoints
Source Code
 Current processing source code
 Add / Delete breakpoints
Variables
 Available variables
 Values for scalar variable
 Number of lines for table type variables
Variables
 Available variables
 Values for scalar variable
 Number of lines for table type variables
Data Preview
 Preview for table type variables
Data Preview
 Preview for table type variables
Check out these blogs for more details:
http://scn.sap.com/community/data-warehousing/bw/blog/2016/05/24/hana-based-bw-transformation
http://scn.sap.com/community/data-warehousing/bw/blog/2016/06/17/hana-based-transformation-deep-dive
http://scn.sap.com/community/data-warehousing/bw/blog/2016/06/23/hana-based-bw-transformation--analyzing-a
nd-debugging

Conclusions how to approach design / development
Use standard push-down capabilities (only use SAP HANA Expert Script if necessary)
• allows the execution in ABAP as well as SAP HANA  easier for functional debugging; later SAP HANA as performance
mode
• profit from general improvement of the HAP framework
• transferring ABAP code to SQL Script in AMDPs requires careful review from performance perspective
• new skillset for programming / debugging / performance analysis required (new debugging features with SAP HANA SP10)
• expert routine was often used to avoid “overhead” of the generated program  no driver for SAP HANA Expert Script
• please review Note 2057542 - Recommendation: Usage of SAP HANA-based Transformations
Avoid the usage of many Info Sources for one DTP
• embedded calculation scenarios create complex plans which are complicated to analyze
• even optimized plans might contain more operations than necessary  performance drain

Public
Generated SAP HANA Views
SAP HANA
SAP BW
Repository
Sources
BI Clients
Data Warehousing
SAP
HANA
Views
gen.
SAP
HANA
Views
SAP
HANA
Views
Data Acquisition
SAP HANA
SAP BW
Repository
Sources
Data Warehousing
gen.
SAP
HANA
Views
Data Acquisition
Analytic Engine
BW Query BW Query
gen.
SAP
HANA
Views
Workspaces
BI Clients

Two options available:
• Pull mechanism via SAP HANA modeler (introduced with BW 7.30)
• Push mechanism triggered from BW (available since BW 7.40)
 how can you transform your architecture from pull to push (*)?
(*)
TechEd 2015 (DMM301) focused on maintenance aspects as well as feature coverage (Calculation View via Modeler vs. Composite Provider / External Query View)
Push mechanism simplifies governance and allows simple adoption of new BW features. New WEB Modeler and BW Edition for SAP HANA will not support Pull mechanism.

Pull: is manually triggered from SAP HANA modeler
• An Analytic View for the selected Info Providers will be created in a specified package
• Optionally classic (XML-based) Analytical Privileges can be created (view always applies privileges)

Push: Info Provider property; view generated at Info Provider activation
• Analytic View will be created in a globally specified package at Info Provider activation
• Generated View secured by SQL Analytic Privileges (optionally SQL-APs can be generated
based on BW analysis authorizations)

Push: Info Provider property; view generated at Info Provider activation
Optional calculation view deployment configurable via
transaction RS2HANA_VIEW
Note: generated calculation view directly dependent on
underlying base tables for classic DSO and Info Cube

How do pull architectures typically look like?
• Pull mechanism is able to create content
objects from physical Info Providers (Info Cube,
classic DSO, Info Object, Query as Info
Provider)
• SAP HANA modeler is used to create additional
logic based on calculation views (views
potentially stacked, unions / joins, calculated /
restricted measures)

How to perform the transition? There are different options:
• Direct exchange of Part Providers
• Rebuild using Composite Providers + consumption in Modeler CVs
• Rebuild using Composite Providers + consumption in BW Query (external query views)

Direct exchange of Part Providers
• The modeler provides functionality to substitute the data sources of
the calculation view
• Text columns need to be remapped manually since the naming
convention differs between pull and push views

Direct exchange of Part Providers
• Minimal manual effort (reuses old models but not fully governed by BW  higher maintenance efforts)
• Not feasible in all cases (e.g. no push for Part Providers of SPO possible  expose via Composite
Provider)

Rebuild using Composite Providers + consumption in Modeler CVs
• Create a new Composite Provider and consume the relevant Info Providers as required
• In the SPO case create an empty Composite Provider and add the SPO as a data source

• Assignments can be created analog to other Part Provider

• Composite Provider is exposed as a Calculation View in the global package

Difference to direct substitution of Part Providers:
• Datasources of CP scenario cannot be deployed as calculation views
(scenarios) for classic DSOs and Info Cubes (performance consideration,
e.g. join push-down)
• For other BW providers the external view uses the 0BW:BIA view
(internal logical representation) as a datasource (therefore no difference
as both views are identical)

• Replace Calculation View Node with a Data Source (here: Composite Provider external view)

• Remap text columns due to different naming convention

Rebuild using Composite Providers + consumption in BW Query (external query views)
• If Composite Provider and Query designer features can represent original model a full remodeling
governed by BW is possible

• External View for BW Query is exposed in a package named after the underlying Info Provider

• The external query view consists of a single column view
• The associated calculation scenario uses the Composite Provider
as a datasource and computes the layered query on top

• In addition to SQL access via the external view, the access via BW BICS is possible with this modeling
approach as well

Pull Push
Part Providers of SPOs can be pulled SPO can be exposed as external view via Composite Provider
Modifications possible (calculated attributes, calculated/restricted
measures) are retained at reimport
Only overwrite possible
Package selection available at import Fixed package for all imports
Optional inclusion of display attributes Display attributes not available (switch to NAV necessary)
No NLS connection NLS connection available (regarding pruning refer to Note 2190204)
Advanced Data Store Object (ADSO) not available ADSO push possible (NAV attributes via Composite Provider – Note 2215947)
Not available on Data Provider level Non-cumulative KFs via External Query View
Classic (XML-based) Analytic Privileges SQL Analytic Privileges (automatic generation of BW authorizations as APs)
Request Handling (QUALOK) independent Request Handling implemented via generated BW authorizations
Always based on Analytic Views Deployment as Calculation View possible
Differences between Pull and Push content generation
Comparison of the Two Methods for Creating Views in SAP HANA for BW Objects
http://help.sap.com/saphelp_nw75/helpdata/en/17/8745cfaeed428eb1fa6d86ad2e40e6/content.htm

How to migrate authorizations?
• Pull Views are secured by Classic APs
• Push Views employ the newer SQL APs
• Both checks cannot be combined on a single view
• How to migrate an existing freestyle authorization concept (without additional BW analysis
authorizations)?

How do the generated analysis authorizations for PUSH views look like?
Created external view secured by SQL Analytic Privileges
Analytical Privilege created for the Composite Provider
Automatically assigned to DBMS User
BW table RS2HANA_AUTH_STR contains filter values according to authorizations for current SESSION_USER
Mapping between BW User and SAP HANA User is maintained in transaction SU01 – DBMS Tab

• Per default BW analysis authorizations will be generated at Info Provider activation / transport
• This can be prevented in transaction RS2HANA_VIEW
if necessary  only view with SQL AP check is created
• If needed manual replication can be performed via:
Report RS2HANA_AUTH_RUN (transaction RS2HANA_GEN)
Process chain (useful since authorization
changes in BW are not automatically
propagated to SAP HANA side)

• SQL APs need to be created manually analog to classic APs (no conversion routine available)
• SQL APs are assigned to SQL users and will be checked at runtime

Conclusions / Short Summary
• Transformation to push on physical Info Provider level or Composite Provider / External Query View
level possible
• Selection of architecture depends on
• modeling feature & maintenance requirements (DMM301, TechEd 2015)
• backend architecture (SPOs, DIS/NAV attributes, ADSO usage, privileges)
• project effort

Public
Integrating SAP HANA Views
SAP HANA
SAP BW
Repository
BI Clients
Data Warehousing
Data Acquisition
Analytic Engine
BW Query
Composite Provider
SAP
HANA
Views
Sources
SAP HANA
SAP BW
Repository
Sources
BI Clients
Analytic Engine
BW Query
Composite Provider
SAP
HANA
Views

Why to integrate SAP HANA Views?
Data is located partly or completely outside of BW
Data might have been processed / produced by any SAP HANA native application
Integration into BW is desired to:
• combine data for subsequent processing (e.g. planning)
• use features dedicated to the analytic engine
• create a persistence in BW

How to integrate SAP HANA Views (or external data)?
Two new providers are available in BW 7.40
Composite Provider
• combines data from BW Info Providers (Info Object, DSO, SPO, Info Cube, SAP HANA Views, Open ODS Views) for
reporting
Open ODS View
• integrates / conforms data from external objects like database tables, database or SAP HANA views, virtual tables or BW
Datasources with direct access
• can generate ADSO (Advanced Datastore Object) as persistence in BW

Modeling options 1) Integration of database table, database
view or virtual table only possible via
Open ODS View
2) Integration of SAP HANA View into
Composite Provider via Open ODS View
3) Direct Integration of SAP HANA View into
Composite Provider

Performance Implications
Conforming models to BW (e.g. ALPHA conversion, no integer data type for characteristics)
• basic conversions available in Open ODS View (calculation scenario since BW 7.40 SP8)
• SQL View  not recommended as it can disrupt query optimization especially before SAP HANA SP10
• more complex conversions can be explicitly modeled in a calculation view (cast operations with calculated columns)  might
prevent filter push-down / causes larger intermediate results
• conversion during staging / SAP HANA optimized transformations

Conclusions / Short Summary
Goal: external data should be integrated into BW
Two Info Providers available: Open ODS View and Composite Provider
Consider functional aspects (feature-driven) which Provider to use
Performance aspects influence the modeling approach
• additional encapsulation of information view into Open ODS View might be beneficial
• consider effects of conforming data types (Open ODS View Conversions to staging solution via SAP HANA enabled
transformations)

Design and Performance considerations SAP HANA Composite
Providers
SAP HANA
Composite Provider
Null Value
handling
Referential
Integrity
Compoundin
g problem
Key-based
processing
Navigation
Attributes
Fields w/o
InfoObject
Association
Additional Joins during Query
Execution
Local SID limitations
Value help for Fields
No SAP HANA exception
aggregation pushdown
( <= BW 7.40)
SAP HANA pushdown
operations may not be possible
No SAP HANA pushdown of
exception aggregation
No pushdown of any hierarchy filter
BW handles NULL values as initial
values. Therfore filter on inital values
implicitly includes a filter on NULL
No OLAP Cache
No parallel processing
Avoids Null Value handling for
SAP HANA models (except Left
Outer Join Scenarios). see also
Null value handling
No exception aggregation in
case several source fields are
mapped to one target field
Nearline
StorageNo NLS in case HCPR runtime
is required (e.g. Joins,
Navigational Attribute, etc.)

Design and Performance Guidelines I
The design considerations are referencing to SAP Note 2271658
The following aspects need to be considered to gain best possible query performance
against composite providers
Navigation Attributes
 Switch on required Navi-Attributes on PartProviders to avoid the generation of an additional CalcScenario
which gets joined to the HCPR during Query Execution
Null Value Handling
 ABAP App Server cannot distinguish between initial value and NULL.
 NULL values can occur in HCPRs with Left Outer Joins or in HCPRs which consume SAP HANA models.
 No Pushdown of exception aggregation
 No Pushdown of hierarchy filters
 To avoid special NULL value handling for SAP HANA models is to guarantee that master data exisits for any
transactional record and referential integrity is flagged.

Design and Performance Guidelines II
Fields without InfoObject Association
 Local SIDs are generated during query Execution for fields without InfoObject Association or Associations to
OpenODS Views
 No OLAP cache usage possible
 No parallel processing in the datamanager
 Usually value helps reads masterdata of the associated InfoObject
 Without association value helps tries to read data from HCPR
- In Case HCPR contains CalculationViews with mandatory InputParameters it is only possible to read the data from
HCPR if for all mandatory InputParameters a default value exists. Otherwise no data is shown
Compounding Problem
 See note 1045683 for compounding-issue description.
 Add missing InfoObjects to PartProvider to overcome the Compounding-issue

Design and Performance Guidelines III
Referential Integrity
 The Flag ‚User confirmed Referential Integrity‘ must only be set if master data values exist in BW for any
transactional record returned by the HCPR. Otherwise the result set of a query might be filtered by the
existing values in the BW master data tables
 Referential Integrity in a HCPR with SAP HANA models may be beneficial to avoid the special handling of
NULL values (see also Null value handling)
 No exception aggregation push down in case several source fields are mapped to one target field.
NearLine Storage
 NLS requires a separate data access and is not embedded directly in the CompositeProvider runtime,
therefore NLS can only be used if the HCPR runtime is not required. Joins, navigation attributes which cannot
be read from relevant PartProvider, etc. require HCPR runtime
Key-based processing
 SAP HANA exception aggregation pushdown with (BW <= 7.40) only possible if master data SIDs are
available in the InfoProvider or by joining the master data SID table

SAP TechEd Online
Continue your SAP TechEd
education after the event!
Access replays of
 Keynotes
 Demo Jam
 SAP TechEd live interviews
 Select lecture sessions
 Hands-on sessions
 …
http://sapteched.com/online

Further information
Related SAP TechEd sessions:
DMM104 - SAP HANA Data Warehousing: Overview, Components, and Future Strategy Lecture (1hr)
DMM162 - SAP HANA Data Warehousing: Build and Run a SQL Data Warehouse on SAP HANA Workshop (2hr)
DMM203 - New Approaches for Data Modeling with SAP HANA Lecture (1hr)
DMM260 - Introduction to Data Modeling in SAP HANA Workshop (2hr)
DMM265 - SAP HANA Data Warehousing: Simplified Modeling with SAP BW 7.5 SP4 Workshop (2hr)
DMM268 - SAP HANA Data Warehousing: Mixed Scenario for SAP BW and SQL DW on SAP HANA Workshop (2hr)
DMM269 - End-to-End Model Performance Analysis in Native SAP HANA Platform Scenarios Workshop (2hr)
DMM302 - SAP HANA Data Warehousing: Models for SAP BW and SQL DW on SAP HANA Lecture (2hr)
DMM360 - Advanced Data Modeling in SAP HANA Workshop (2hr)
SAP Public Web
scn.sap.com
www.sap.com
SAP Education and Certification Opportunities
www.sap.com/education
Watch SAP TechEd Online
www.sapteched.com/online

Thanks for attending this session.
Please complete your
session evaluation for
DMM300
Contact information:
F name MI. L name
Title
email address
Feedback

© 2016 SAP SE or an SAP affiliate company. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate
company) in Germany and other countries. Please see http://www.sap.com/corporate-en/about/legal/copyright/index.html for additional trademark information and notices.
Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.
National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its
affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or SAP affiliate company products and
services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as
constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop
or release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future
developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time
for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-
looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place
undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

Dmm300 – mixed scenarios for sap hana data warehousing and BW: overview and experiences

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (20)

Semelhante a Dmm300 – mixed scenarios for sap hana data warehousing and BW: overview and experiences

Semelhante a Dmm300 – mixed scenarios for sap hana data warehousing and BW: overview and experiences (20)

Mais de Luc Vanrobays

Mais de Luc Vanrobays (14)

Último

Último (20)

Dmm300 – mixed scenarios for sap hana data warehousing and BW: overview and experiences