Dmm203 – new approaches for data modelingwith sap hana

Public
DMM203 – New Approaches for Data Modeling
with SAP HANA

© 2016 SAP SE or an SAP affiliate company. All rights reserved. 2Public
Speakers
Bangalore, October 5 - 7
Christoph Morgen
Las Vegas, Sept 19 - 23
Christoph Morgen
Barcelona, Nov 8 - 10
Christoph Morgen

Disclaimer
The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of
SAP. Except for your obligation to protect confidential information, this presentation is not subject to your license agreement or
any other service or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this
presentation or any related document, or to develop or release any functionality mentioned therein.
This presentation, or any related document and SAP's strategy and possible future developments, products and or platforms
directions and functionality are all subject to change and may be changed by SAP at any time for any reason without notice.
The information in this presentation is not a commitment, promise or legal obligation to deliver any material, code or functionality.
This presentation is provided without a warranty of any kind, either express or implied, including but not limited to, the implied
warranties of merchantability, fitness for a particular purpose, or non-infringement. This presentation is for informational
purposes and may not be incorporated into a contract. SAP assumes no responsibility for errors or omissions in this
presentation, except if such damages were caused by SAP’s intentional or gross negligence.
All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially
from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only
as of their dates, and they should not be relied upon in making purchasing decisions.

Agenda
• SAP HANA View Modeling
– Modeling Graphical Calculation Views today
– Converting your Views to Calculation Views
• New SAP HANA Data Modeling Approaches
– Developing with the SAP Web IDE for SAP HANA
– Graphical development of CDS entities
– Web-based Calculation View development enhancements
– Using CDS Views with Calculation Views
– SAP PowerDesigner Modeling for HANA

Public
SAP HANA View Modeling
Today’s approach of modeling Graphical Calculation Views

SAP HANA View Modeling – Overview
Virtual Data Modeling as a key SAP HANA concept
SAP HANA PLATFORMSAP HANA PLATFORM
Database LayerDatabase Layer
Physical TablesPhysical Tables
HANA ViewsHANA Views
Virtual Data Flow Models
• No persistent aggregations | single atomic copy of data
• In-memory engines | performance
• Multidimensional Reporting Models
• Enterprise Applications Virtual Data Models
Operational Reporting | Applications | Analytics

Evolution of SAP HANA View Modeling Approaches
Attribute View
Analytic View
Calculation Views
(Graphical and Script-based)
Graphical Calculation Views*
*and SQLScript-based
Table Functions as data sourceAny of the approaches is still fully supported as of today with SAP HANA SPS12 !

Modeling Graphical Calculation Views
The Calculation View Modeling Toolbox
 Calculation Views of type Dimension
– Composing Dimension-Attributes and -Hierarchies
– Has no measures, does not aggregate
– Creates BI View-metadata / exposed to BI-Tools
 Calculation View of type CUBE
– Has measures, always aggregates
– Creates BI View-Metadata / exposed to BI-Tools
 Calculation Views of type CUBE with StarJoin
– Star-joins fact data with dimension-type calculation views
– Creates BI View-Metadata / exposed to BI-Tools
 Calculation View of type “<default>” / “<empty type>”
– Free-form, custom calculation view
– Does not create BI View-Metadata / not exposed to BI-Tools
Dimension
Cube Star-Join
Cube

The Calculation View Modeling Toolbox
 Compose a dataflow scenario using node-operations
– Projection, Join, Union, Aggregation, Star-Join, Rank
and Graph*
 Select from a range of data sources
– SAP HANA Tables (columnar, …),
CDS entities (design-time data sources)
– SQL Views
– SQLScript Table Functions
– Remote SAP HANA database container objects
– Remote smart data access / virtual data sources
 Stacking of Calculation Views to larger data models
– Link and mapping of parameters between Calculation Views
– Sharing, extraction and propagation of semantic information
*New only within the SAP Web IDE for SAP HANA development envrionment.

SAP HANA View Modeling – Key Concepts
Modeling multidimensional Scenarios in Calculation Views
Multidimensional scenarios in Calculation Views
 Calculation Views of type Dimension
– Composing Dimension-Attributes and -Hierarchies
 Calculation Views of type CUBE with StarJoin
– Star-joins fact data with dimension-type calculation views
like Analytic Views (with same performance guidance's)
– Implicit Calculation View optimization will evaluate
to delegate processing to same execution behavior as Analytic Views
Usage Scenario
 Implement new multidimensional scenarios / new projects
 Seek to refactor Attribute-/Analytic View scenarios
Calculation Views
Note: As of SAP HANA SPS12 Calculation View Star-Join scenarios cover Analytic Views capabilities incl. Temporal Joins

Modeling multidimensional Scenarios in Calculation Views
Hierarchy capabilities in Calculation Views
 Parameter use within Hierarchies
– Use within calculated columns of hierarchies
– Setting Hierarchy root-nodes
 Time dependent hierarchies
 Hierarchy-based SQL processing of Views
– Hierarchy-based filtering and aggregations
 Hierarchy-based Variables
– Hierarchy-based value help and filtering
 Hierarchy-driven SQL-based Privileges
 NotAssigned- and NULL value handling in
Hierarchies
 Full composite key support for Parent-Child
Hierarchies

Modeling with Virtual and Extended Tables
Consuming non-In Memory data in Calculation views
 Dynamic Tiering
– Provides a disk-based columnar SAP HANA database extension using
SAP HANA external tables (extended tables) *
 Smart Data Access
– Provides a virtual access layer to outside SAP HANA data (e.g. other
databases, Hadoop systems, etc.) using so-called virtual tables
Leveraging “warm data”
from SAP HANA
Dynamic Tiering
external tables
Leveraging “cold data”
via SAP HANA
Smart Data Access
virtual tables
 Model and approach these scenarios carefully and monitoring query
performance
– Remote data imposes a natural break in the execution
– Ensure that filters and aggregation is pushed to remote sources
– Use and observer join settings and optimizations
– Implement input source pruning (explicit or implicit) techniques
*Supported external systems and restrictions are documented in SAP Note 1868209

© SAP AG 2011
Enabling SQLScript with Calculation views
• SQLScript Table Functions allow to embed custom scripting
– Side-effect free read-only functions using standard SQLScript
– Consume data from tables, models, functions, predictive algorithms, etc.
– Script based-calculation models can be re-factored into table functions
Using SQLScript with Graphical Calculation Views
Graphical Calculation View
referencing Table Function

Using SQLScript with Graphical Calculation Views
© SAP AG 2011
SQLScript approaches
• Standard SQL versus Calculation Engine (CE Functions) Plan Operators
 Recommendation! When creating functions or procedures with SAP HANA SPS09+ use standard SQL
syntax instead of CE Functions. The SQL optimizer is able to leverage alternative execution engines and
are not restricted only to the Calculation Engine
Recommended approach

Expression Language in Calculation Views – SQL Expressions
General SQL support within Expression Editors
 Explicit expression language selection
– Explicit distinction between Column Engine expression
and SQL Expression was introduced for Calculation
Columns since SAP HANA SPS10
– Filter expressions
– Restricted measure expressions
– Default value expressions for Variables/Parameters
 Expression language conversion behavior
– Harmonized execution modes between Column Engine-
and SQL expression language
– No result differences between use of a SQL vs Column
Store expression due to respective semantic functions
conversion
Example – Add sematic using a NULL-value
Within a column store expression, you want to use the column store
semantic addition behavior
1 + NULL = 1
This may implicitly get converted into a SQL expression
ADD_NAZ(1,NULL) = 1
Within a column store expression, you want to use the SQL semantic
addition behavior
SQLADD(1,NULL) = NULL
This may implicitly get converted into a SQL Function
1 + NULL = NULL
SAP NOTE: 1857202 - SQL Execution of calculation views

Analytic Privileges Editor – SQL-based Analytic Privileges
Create SQL-based Analytic Privileges
 Start with general Attribute based Analytic Privilege, then switch to SQL-based
 Use SQL Hierarchies within SQL Analytical Privileges
SalesHierarchyNode = MAJESTIXSalesHierarchyNode = MAJESTIX

Additional SQL Optimization
Optimized Calculation View Execution
 Since SAP HANA Rev 62 the SQL optimization
can explicitly be determined setting
the “Execute In:” property to “SQL Engine”
 Since SAP HANA SPS9 the optimized execution
behavior implicitly evaluates to leverage
additional SQL optimization
 SAP HANA SPS10 optimization enhancements
include implicit optimized Star Join processing
Optimized execution for SQL-queries against Calculation Views
 The initial Calculation Engine optimization generates a single SQL statement across a stacked model,
which is then passed to the SQL optimizer
 The SQL optimizer adds additional optimizations and delegates operations to the best database execution
operator, e.g. delegate star join-nodes to the OLAP processing engine where possible
Calculation Engine Optimization
Calculation Views
SQL Optimization
includes
• Stacked Views /
SQL unfolding
• Cost-based
optimization
• Delegate
processing to
best operator
* Implicit SQL optimization details are documented in
SAP Note 2223597 and 2291812.

Visualize PlanCalculation View Debugger
Performance Analysis & Query Insight Tools
Timeline – root cause analysis
Operator list
Explain Plan

Advantages of the Calculation View Modeling approach
 Benefit from better optimization
– Queries against stacked models of only Calculation-Views,
provide better overall optimization options
– Better pruning / filter–push down
– Less materialization / no View-types changes
– Table Functions included in overall optimization
e.g. Filter push-down
 Modeling Flexibility
– Componentize larger data models,
create re-usable partial models and stack
– Map parameters and variable between views
 Leverage Performance Analysis tools
– Calculation View Debugger
Stacked Calculation View Data Model

SAP HANA View Modeling – New Approaches
Converting all SAP HANA Views to Calculation Views
Going forward with Graphical Calculation Views
 Graphically modeled Calculation Views target to unify all SAP HANA View capabilities,
with respect to functionality as well as performance
 Convert within the classic SAP HANA Repository
Attribute Views
Analytic Views
Script-based CVs
CVs
Modeled Analytic
Privileges
Calculation Views
Table Functions
SQL-based Analytic Privileges
Convert using
the HANA Studio Migration Tool
Classic HANA Repository
objects*
*this conversion is a preparation step
before migrating to SAP Web IDE for SAP
HANA.

Going forward with Graphical Calculation Views
 Converting your SAP HANA Views
– Dimension-Type graphical Calculation Views replacing Attribute-Views
– Cube-/StarJoin-Type graphical Calculation Views replacing Analytic-Views
– Table Functions as Data Sources in graphical Calculation Views
replacing Scripted-Calculation Views
– SQL-based Analytic Privileges replacing classic modeled Analytic Privileges
 With SPS11 a migration utility is initially provided
to support the conversion process

How to find SAP HANA documentation on this topic?
• See SAP Note https://launchpad.support.sap.com/#/notes/2325817

Public
New Approaches
SAP HANA Data Modeling – New Approaches

Graphical Development Tools – Innovation Path
SAP HANA Web-based Development Workbench
 Database & Application Development
 Initial releases of graphical development tools
SAP HANA Studio – Modeler Eclipse-perspective
 Primarily focused on HANA View modeling artifacts
SAP HANA Studio – Development Eclipse-perspective
 Support for all application and database design-time
artifacts and providing additional repository functionality
– E.g. Script-based CDS development, flowgraphs, ….
SAP Web IDE for SAP HANA
 New Development tooling platform for
SAP HANA, initially delivered in SPS11*
major release with SAP HANA SPS12
 All development tools
– (re-) implemented as
XS Advanced applications
leveraging new UI frameworks of the
SAP Web IDE for SAP HANA and
– commonly building on the new
HANA deployment infrastructure.
 Web-based graphical editors
– Calculation Views
– CDS entities
– Data flows
– Lineage and impact analysis
*SAP Note 2304873 SAP Web IDE for SAP HANA SPS12 - Central Release Note
DEV204 – What's New in Native Application Development with SAP HANA
DEV161 – Intro to SAP HANA Extended Application Services, Advanced Model Development

The new Web-based HANA Development Environment
SAP Web IDE for SAP HANA
the new and future development
environment for SAP HANA
• Using MTA Projects (Multi-Target Application)
• Using HDB Modules
 An isolated HANA Deployment Infrastructure
Container (~ isolated schema as your HANA
development workspace)
 Any access is executed via (and restricted to)
a HDI-container-specific technical user,
NOT by your XSA webIDE development user
 There is no default access to other schemas
on the same system
 Namespaces can be defined, to support name-
structuring all development artefacts
*_HDI_CONTAINER SCHEMA YXZ
„external data“
_SYS_BI
DEV160 – Intro to SAP HANA Database Development with Deployment Infrastructure

SAP HANA Graphical CDS Editor – Overview
Graphical development of CDS objects
• Graphical CDS Editor
• Supported capabilities incl.
 Contexts
 Entities, incl. HANA native types
 Structured Types
 Indexes, incl. Search/Full Text Indexes
 Table Groupings, Partitioning
 Associations

SAP HANA Graphical CDS Editor – Overview
Graphical development of CDS objects
• Graphical CDS Editor
 Contexts
 Entities, incl. HANA native types
 Structured Types
 Indexes, incl. Search/Full Text Indexes
 Table Groupings, Partitioning
 Associations
 References to CDS-Views

SAP HANA CDS Development – Graphical & Script-based Editor
CDS development editor choices
• Graphical and Script-based CDS editors
• Graphical editor
 opens any hdbcds-file
 Persist any non-
graphically
supported
capability
• Script-based editor
 opens any
graphically
developed
hdbcds-file

The new Graphical Calculation View Editor – Overview
New Web-based Editor for Calculation Views
• Supports all major capabilities for
graphical Calculation View Development
 Joins, Unions, Projections, Aggregations,
StarJoins (incl. temporal join), Rank
 Variables, Input Parameters, Calculated
Columns/Filters (incl. SQL expressions), Unit &
Currency Conversion
 Catalog metadata services,
 Initial convenience / supportability functions
e.g. local refactoring, basic data preview,
performance analysis
• New capabilities incl.
 Cross-object impact and lineage analysis
 Graph analysis modeling

Web-based Calculation View Editor – Data Sources
Calculation View Data Sources
• Data Sources from within your HDI
container
 Design-Time objects local to your
development container
 Tables, Table Functions, SQL Views,
Calculation Views, etc. are supported
inputs
• Data Sources from outside your HDI
container
 Synonym definitions are required to
reference to data sources outside of your
current development container

Web-based Calculation View Editor – Star Joins / Temporal Joins
Modeling Star Joins
 Create graphical calculation views
– Join multiple dimensions with the
fact data input set.
– Now, inner-join use supports
temporal Join-conditions
(to match Analytical View capabilities)
Star Join - Join Definition incl.
details (Referential, LeftOuter, …)

Web-based Calculation View Editor – Deployment and Access
Target Deployment concept
• Like the development container, the deployment target
is as well an isolated HDI container
• New -> Deployment to any Schema
– Calculation Views are no longer deployed to _SYS_BIC,
but can be deployed to “any” HDI-container (“schema”)
• New -> Security and Data Access requirements
– XSA default approach to container access is via a technical
user + business user information passed along
o Data analytic privileges shall be managed on HANA
o User-specific object privileges shall be managed by the App
– Non-XSA application / BI Clients
o Grant privilege/container role to access the container's schema
o Grant privilege to read BIMC tables.
*_HDI_CONTAINER SCHEMA YXZ
„external data“
_SYS_BI
o Cross schema data access provided
by synonynm and a XSA data access
service to be defined
SEC161 - Learn from the Experts: Secure and Extend
Advanced SAP HANA Applications (Hands-On Workshop)

Graph Data Modeling and Processing
SAP HANA Graph
Graph data processing capabilities to allow businesses to map
connections between entities in a network and discover relationships
on the fly.
• Property Graph Model with flexible schema with ability to traverse
relationships without the need for predefined modeling
• Built-in high-performant graph algorithms (neighborhood search,
shortest path, strongly connected components, pattern matching) to find
connections without manually creating complex JOIN statements
• Graph viewer tool - allows for quick visualization and dynamic interaction
(i.e. change algorithm parameters) with graph data in real-time
• Graph Analysis Modeling – Integrated with SAP Web IDE for SAP
HANA Calculation View Editor for creating graph analysis models visually
instead of via SQL or SQLScript S A P H A N A
SAP HANA Graph
Property
Graph Model
Graph Engine
Native
Algorithms
Graph Viewer Graph Modeler

Modeling a SAP HANA Graph Analysis
Create Edge and Vertex Tables
With as many attributes as you want
Workspace
Relationships Members
Create Graph Workspace

Modeling a SAP HANA Graph Analysis
Create Graph Analysis Model
Query the Graph Analysis Model
Shortest Path

SAP HANA Lineage- and Impact Analysis
New Web-based Data Modeling Editors
• New capabilities incl.
 Cross HANA-object Lineage and Impact analysis

Converting all SAP HANA Views to HDI Development Objects
Step 1 - Unified object conversion
Convert using the HANA Studio
Migration Tool
Going forward with Graphical Calculation Views in SAP Web IDE for SAP HANA
 If you seek to move and migrate your SAP HANA development towards SAP Web IDE for SAP HANA,
classic HANA repository development objects, need to be converted to HDI-based development objects
–
Attribute Views
Analytic Views
Script-based CVs
CVs
Modeled Analytic
Privileges
Calculation Views
Table Functions
SQL-based Analytic Privileges
HDI-based design-time artefacts
• Calculation Views
• Table Functions
• SQL-based Analytic Privileges
Step 2 - HDI-object conversion
HDI-migration/conversion tools are planned to be delivered with a future
shipment after SPS12 to help move your applications from XS Classic
and the HANA Repository to XS Advanced and HDI.
DEV703 – Migrating Apps Between Models of SAP HANA Extended Application
Services.

New Data Modeling Approaches
SAP HANA Core Data Services
SAP HANA Core Data Services (CDS)
 Design-time language for defining models (tables, views)
as part of native HANA applications in XSA/HDI (or XSC/Repository)
 Targets Application data models
 HANA CDS capabilities include
– Data Control Language, User defined types, Associations,
Annotations, Extensibility, …
– Source based definition
-> deployment managed by HDI (or HANA Repository)
– Runtime catalog objects
-> SQL defined tables and views
– Lifecycle management
– Data model is located in the database and
not part of an application stack or framework on top
 HANA CDS ≠ ABAP CDS
HANA Schema
with table, views
HANA Schema
with additional
CDS metadata
SQL Engine
CDS CDS Catalog
Schema XYZ

Modeling Views – Using CDS or Calculation Views?
SAP HANA Core Data Services Views
 CDS views technically are SQL views
 “Limited” to SQL exposed HANA functionality
 No graphical editor for CDS views currently
 “Simple” views should behave similarly w.r.t. performance
 Do not support analytic annotations
Using one or the other ?
 Analytics / BI-use case
– Calculation View provide rich BI metadata (cubes, hierarchies, ….)
for SQL / MDX, SAP and 3rd
–party BI Tools
– ABAP CDS views support analytic annotations (analytic query),
consumption requires NetWeaver “analytic query layer”
e.g. consumable by Design Studio + AO /
3rd
party NetWeaver-based BI tools (MDX)

 Performance aspects
–Calculation View options provides additional pruning and optimization during their runtime
instantiation process (SAP Note 2223597),
thus likely to provide better performance
o All columns not requested by the query are removed from the instantiated calculation model
o In addition complete join/union data sources might be pruned during instantiation
–Avoid mixing Calculation Views and CDS Views
o To ensure best optimization, do not consume Calculation Views from CDS Views
o Simple CDS Views as data sources to Calculation View can be used

 How important is the tooling / development approach
–Graphical or scripting?
 What‘s your Application scenario?
–Building a distinct application or
mass development of analytic-structures
for BI Tools?
 Functional aspects?
 How important is performance?
CDS
Basic persistency
Tables & SQL views
CDS
Basic persistency
Tables & SQL views
Calculation Views
on top of
CDS tables/SQL views
Calculation Views
Analytical scenarios
Hierarchies, GraphEngine
Calculation Views
Analytical scenarios
Hierarchies, GraphEngine

SAP PowerDesigner – CDS Modeling Support
CDS Modeling Support in SAP PowerDesigner*
 Physical Data Model generation as HANA CDS
– Define CDS entity and Types
– Define CDS Views
– Generate/activate CDS artifacts in HANA
o CDS Repository Artifacts generated and activated in the repository
o CDS HDI: Artifacts generated as .hdbcds files
DMM111 – Modeling the End-to-End Data Warehouse with SAP PowerDesigner
DMM101 – Architecting Your Enterprise with SAP PowerDesigner
*SAP PowerDesigner 16.6 SP01

SAP PowerDesigner – Calculation View Modeling Support
Calculation View Modeling support in SAP PowerDesigner
 View specification describes Output Projection and Sources
– Calculation view defined in HANA Catalog PDM
– Generate Calculation-view skeleton in HANA (Projection and sources)
– Simple Calculation views based on one or two data-sources can be generated
and activated
– Partially reverse-engineer Calculation-views
(data-sources and input columns for impact-analysis purpose)
 Using Calculation-views with CDS entities (HDI)
– Calculation-views and CDS artifacts defined in different models
– CDS artifacts must be first generated (and activated), then reversed as tables
before being used as Calculation-view data-sources.
*SAP PowerDesigner 16.6 SP01

SAP HANA Data Modeling
Key Concepts and New Approaches
Your SAP HANA View Modeling path forward
 You seek to run for longer on SAP HANA SPS12
– Model Calculation Views in SAP HANA Studio
– You don’t have migrate Attribute- and Analytic Views in order to run on SPS12
– It’s a chance to optimize
– New HANA View development should be focused on Calculation Views
– If you build XSA Applications on SPS12,
explore SAP Web IDE for SAP HANA Data Modeling tools
 You are ready to jump or require to move fast to the next SAP HANA release
– Prepare to convert all SAP HANA Views to Graphical Calculation Views
– Start and explore SAP Web IDE for SAP HANA Data Modeling tools
– Familiarize with the new overall development approach
– Validate current feature set, determine if you can start on SPS12

SAP TechEd Online
Continue your SAP TechEd
education after the event!
Access replays of
 Keynotes
 Demo Jam
 SAP TechEd live interviews
 Select lecture sessions
 Hands-on sessions
 …
http://sapteched.com/online
Recommended TechEd 2015 Session
DMM208: New and Best Practices for Data Modeling with SAP HANA - http://events.sap.com/teched/en/session/26543

Further information
Related SAP TechEd sessions:
DMM260 – Introduction to Data Modeling in SAP HANA
DMM360 – Advanced Data Modeling in SAP HANA
DMM269 – End-to-End Model Performance Analysis in Native SAP HANA Platform Scenarios
DMM302 – SAP HANA Data Warehousing: Models for SAP BW and SQL DW on SAP HANA
DMM162 – SAP HANA Data Warehousing: Build and Run a SQL Data Warehouse on SAP HANA
DMM300 – Mixed Scenarios for SAP HANA Data Warehousing: Overview and Experiences
DMM111 – Modeling the End-to-End Data Warehouse with SAP PowerDesigner
DEV204 – What's New in Native Application Development with SAP HANA
DEV160 – Intro to SAP HANA Database Development with Deployment Infrastructure
DEV161 – Intro to SAP HANA Extended Application Services, Advanced Model Development
DEV703 – Migrating Apps Between Models of SAP HANA Extended Application Services.
SAP Public Web
scn.sap.com http://scn.sap.com/community/developer-center/hana http://scn.sap.com/community/hana-in-memory
www.sap.com hana.sap.com https://blogs.saphana.com/2016/05/10/what-is-new-in-sap-hana-sps12/
SAP Education and Certification Opportunities
www.sap.com/education https://www.youtube.com/user/saphanaacademy https://open.sap.com/courses
Watch SAP TechEd Online
www.sapteched.com/online

Thanks for attending this session.
Please complete your
session evaluation for
DMM203
Contact information:
Christoph Morgen
SAP HANA Product Management
SAP SE | Dietmar-Hopp-Allee 16 | 69190 Walldorf | Germany
christoph.morgen@sap.com
Feedback

Dmm203 – new approaches for data modelingwith sap hana

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Dmm203 – new approaches for data modelingwith sap hana

Semelhante a Dmm203 – new approaches for data modelingwith sap hana (20)

Mais de Luc Vanrobays

Mais de Luc Vanrobays (10)

Último

Último (20)

Dmm203 – new approaches for data modelingwith sap hana