Analytics with IMS Assets - 2017

© 2017 IBM Corporation
The Analytics (R)evolution for your
IMS Assets
Hélène Lyon
Distinguished Engineer, European
Technical Executive
z Solutions Architect: IMS, Analytics,
Platform Modernization
IBM Systems, zSoftware Sales
Phone: +33 (0)6 84 64 19 39
e-mail: helene.lyon@fr.ibm.com
*

Executive Summary
Analytics is needed by all enterprise customers!
– Today z Systems owns business critical transactional and batch workload!
– Today z Systems owns business critical data!
– The new z Systems z13 & z13s allow a real technology shift for analytics!
One option is to augment Data Warehouse capabilities to provide
Right-Time Data Analytics including Big Data support.
– Answer business needs around fraud analytics & customer 360 view
Another option is to implement in-line analytics also called Real-Time Analytics
– Add analytics in z/OS based workload
– Answer business needs around fraud detection at the time of the fraud or
customer call optimization to prevent churn and increase cross sell
Bring the analytics to the data for Right-Time Insight
Bring the analytics in the application for Real-Time Decision Management!
European IMS Information Day - Sweden & Germany - May 2017

In the new insight economy, winners infuse analytics
everywhere to drive better outcomes!
Create new business models
(CEO)
Attract, grow, retain
customers
(CMO)
Transform financial
& management
processes
(CFO)
Manage risk
(CRO)
Prioritize IT investment
for innovation
(CIO, CDO)
Optimize
operations
(COO)
Fight fraud and
counter threats
(CSO)
Systems of
Insight
Systems of
Record
Systems of
Engagement

Market Trends & Analytics Needs
• Drive increased
transactional workload
through mobile
• Ask for more
personalized services
and offering
End customers /
consumers
• Aim to improve level of
detail and frequency of
existing analytics
• Have innovative ideas
for new analytics
• Are driven by increasing
demand in regulatory
requirements
Lines of business/
Data scientists
both have a high demand in secure
data management
• Transactional
workload runs on
the mainframe
• Bad perception of
the mainframe
with regards to
• modernity
• cost efficiency
• DW / analytical environment on distributed platforms
• Huge amount of analytics applications
Business IT
Overnight batch/
ETL for analytical
purposes
Scoring Rules
A
Analytics ComponentsA
Analytics
executed
outside of
core
business
applications
zDatazApps

All Data New Dev StylesNew Analytics More People
Business
Value
Embrace all data
Run at the speed
of business
1 Enable all analytics
IBM Analytics Point of View - Make DATA SIMPLE and
ACCESSIBLE to ALL
DATA
Professionals are
leading THE
Transformation!
2
3

OBSERVEOBSERVE INTERPRETINTERPRET DECIDEDECIDE
Prescriptive
What should I do?
Descriptive
What has happened?
Predictive
Why did it happen?
What could happen?
ACTION
Business Rules & OptimizationBusiness Rules & Optimization
DATA
HUMAN INPUTSHUMAN INPUTSSQLSQL
Scoring & ForecastingScoring & Forecasting
<<
From Data Analytics to Insights & Actions with fit-for-purpose
technologies
Cognitive computing extends analytics levels to new kind of data, using new technologies.
JF Puget @ IBMdeveloperWorks How Does Cognitive Computing Relate To Analytics?

The Evolution in the Approach to Getting Value from Data
Operations Data Warehousing Self-service
Analytics
New Business
Imperatives
Maturity High
High
Low
Data-Informed
Decision Making
• Full dataset analysis
(no more sampling)
• Extract value from
non-relational data
• 360o
view of all
enterprise data
• Exploratory analysis
and discovery
Warehouse
Modernization
• Data lake
• Data offload
• ETL offload
• Queryable archive
and staging
Lower the Cost
of Storage
Ensure resiliency
and availability
Business
Transformation
• Create new business
models
• Risk-aware decision
making
• Fight fraud and
counter threats
• Optimize operations
• Attract, grow, retain
customers
Value
We
are
here

SoE
Analytics evolution to support all Analytics Apps on all Data –
The Mainframe Use case
Applications Data
SoI
HDFSMap / Reduce
Spark
Historical data in DB2 for z/OS &
IBM DB2 Analytics Accelerator
Other Data
BI Reporting Data Warehouse / Data Marts
The Data Lake Evolution
Operational Data stored in
VSAM, IMS, DB2
SoR Core Business supported by
CICS, IMS, WAS
z/OSRulesRulesRulesRules
ScoreScoreScoreScore
executionexecutionexecutionexecution
Machine Learning
The Predictive Analytics EvolutionScore
Creation
IT Operational Data

The two Flavors of IBM DB2 Analytics Accelerator
DB2 Analytics Accelerator
for z/OS
A workload optimized, appliance add-on to DB2
for z/OS that enables the integration of analytic
insights into operational processes to drive
business critical analytics & exceptional business
value.
Speed
Dramatically improve query response – up
to 2000X faster – to support time-sensitive
decisions
Right-time. Low latency. Trusted. Accurate.
Savings
Minimize data proliferation
Lower the cost of storing and managing
historical data
Free up compute resources
Simplicity
Simplify infrastructure, reduce ETL and
data movement off-platform
Non-disruptive installation
Security
Safeguard valuable data under the control
and security of DB2 for z/OS
Protected. Secured. Governed.
DB2 Analytics Accelerator
on Cloud
High-speed analysis of enterprise
data with cloud agility, flexibility and
ease of deployment
High-speed analysis
Rapid insight from enterprise data in a
secure cloud environment
Fast and Simple Deployment
Improved agility and quick time to value
Secure cloud environment
Comprehensive data encryption
capabilities based on a dedicated,
bare-metal deployment
Reduce cost
Speed implementation on analytics
projects to reduce overall
implementation costs

z Systems Analytics Areas complement existing Analytics
Environments.
IBMDB2Analytics
Accelerator
In transaction rules and
score execution
Intraday capability for ad-hoc
queries & predictive analytics
Availability of historical
data (in raw format)
Accelerated reporting to
fulfill internal and regulatory
requirements
Ability to transform
data before offload to
DWH or reporting
Ability to create new
models at any time
Quasi Real Time
availability of data
for analytics
Instant access to raw data
for new report generation in
hours instead of days
Load and merge of ANY non
DB2 z/OS data
Scoring Rules
A
zDatazApps
Scoring
Rules
Explore data to
uncover hidden
insights
A

Data Warehouse and Data Lake
A Data Lake is…
+An analytics sandbox for exploring data to
gain insight
+An enterprise-wide catalog to find data across
the enterprise and to link from business term to
technical metadata
+An environment for enabling reuse data
transformations and queries
+An environment where users can access vast
amounts raw data
+An environment for developing and proving
an analytics model and then moving into
production; experience in production may drive
further experimentation in the data lake
A Data Lake is not…
- A data warehouse or data mart of all of the data
in an enterprise
- A high-performance production environment
- A production reporting application
- A purpose-built system to solve a specific
problem

The Data Lake: Subsystems
Data Lake (Systems of Insight)
Information Management and Governance Fabric
Catalog: Management,
Governance, Protection
Self-
Service
Access
Enterprise
IT Data
Exchange
Self-Service
Access
Analytics Teams:
Analytics, DWH, …
Data Governance, Risk
and Compliance Team
Information
Curator
LoB Teams:
Risk Modeling,
Fraud Mgmt, …
Data Lake
Operations
Enterprise IT
Other Data
Lakes
Systems of
Engagement:
CC, e-Mail,
Touchpoints,
Notes, …
Systems of
Automation
Systems of
Record:
ATM, Loan,
Deposit, …
New Sources:
Social Media,
Twitter, …
Data Usage
Data Lake Repositories
Hadoop
(non-structured) DB2 for z/OS
DB2 Accelerator
(VSAM, ...)
Teradata

Use Case: Data Access across DB2 for z/OS & Hadoop Platform
Enables z data (DB2 for z/OS, VSAM or IMS in the Accelerator . . .) to be integrated
and queried in context of non-structured data on distributed Hadoop, e.g.
– Sentiment analytics using e.g. e-mail and CC transcripts, Twitter data ...
– Integrates z customer data (profile, transactions) with TWC weather data
– Execution of complex analytical queries on z data (e.g. DB2 for z/OS) via DB2 Analytics
Accelerator
Big SQL federation capability provides single point of entry for SQL queries
– Split queries are generated automatically by Big SQL 4.2
– Merge, JOINs, ... at Big SQL 4.2 federation layer transparent to application
DB2 for z/OS
&
DB2 Analytics
Accelerator
IBM BigInsights
HDFS
z/OS Big SQL 4.2
Hive / HCatalog
Application
SQL
Split_Query_1
Split_Query_2
On distributed
platform or cloud
including z/OS
On any platform
including z/OS

Spark, a Framework for Analytics Applications
Fast Runtime Environment
– Interactive or batch processing
– Based on data in-memory processing
• High performance for multi-step processes where Spark can
pass the data directly without using disk storage.
– Parallel processing
Interface to Data
– Accessing Hadoop based HDFS data, Cassandra,
Hbase, …
– Accessing any traditional databases using JDBC
Interface for Applications – Ease of Use APIs supported
by modern languages
– Stack of libraries including SQL, Machine Learning,
GraphX, and Spark Streaming
– Over 80 high-level operators that make it easy to build
parallel applications
– Many languages supported including Java, Scala,
Python and R
• Spark is actually written in Scala
Spark is NOT a datastore, NOT a
replacement for Hadoop!

VSAM
z/OSKey
Business
Transaction
& Batch
Systems
Spark Applications: IBM
and Partners
AdabasIMSDB2 z/OS
Distributed
Teradata
HDFS
Apache Spark Core
Spark
Stream
Spark
SQL
MLib GraphX
RDD
DF
RDD
DF
Optimized data access
IBM z/OS Platform for Apache Spark
and *many* more . . .
Spark can run on z/OS close to z/OS-based Applications & Data
Values:
Data-in place analytics,
without need to ETL or move
data outside platform for
analytic purposes
Optimized access and z/OS
governed ‘in-memory’
capabilities for core business
data
Unique capability to access
almost all z/OS sources with
Apache Spark SQL & many
non-z data sources
Almost all zIIP eligible
Integration of analytics
across core systems, social
data, website information,
etc.
and *many* more including SMF, OPERLOG, SYSLOGs, . . .

Use Case: Spark applications accessing data in DB2 for z/OS
Enables data scientists (and similar
roles) to use Apache Spark to explore
data and develop analytical models
without moving data outside z Systems
– IT Operation analytics
– Fraud discovery and prevention models
– Customer segmentation for up-sell and
cross-sell
– Sentiment analytics with z customer
profile and z transactional data
Data sources
– DB2 for z/OS and other z/OS
subsystems
• i.e. VSAM, IMS, log data, …
– IBM DB2 Analytics Accelerator for fast
SQL processing & In-DB transformation
Spark on z/OS
(Spark SQL & Spark Mllib)
DB2 for z/OS
JDBC Access
Other data sources possible
DB2 Analytics
Accelerator
Application
(Python, Scala, ...)

Use case: End to End IT Operational Analytics
High Level Product Placement
Analytics Applications
Analytics Applications
Framework
“IT Data Lake”
Data Layer & Indexing
z/OS IT Data
Acquisition
Omegamon/CICSPA/IMSPA&PI/TAW
IDAA
Splunk
DB2
zOI-zOperational
Insights
TDS
Optimized Data
Layer from Spark
on z/OS
CDP
IOAz
DashBoards
&
Insights
z Insight
Pack
LogStashIDAA Loader
zSecure/Qradar
ML on
z/OS
Spark
Elastic
DB
Kibana
Batch

What is Machine Learning?
Identifies patterns in historical data
Builds behavioral models from patterns
Makes recommendations
Netflix personalized
movie recommendations
Waze personalized
driving experience
7 out of 10 financial
customers would take
recommendations from
a robo advisor
Machine learning is everywhere,
influencing nearly everything we do…
“The science of getting computers to
act without being explicitly
programmed”
“Systems that can learn from data”

Data Science: Two Main Interests
1) Exploration: We don’t have any special attribute we
want to predict. Rather we want to understand the
structure present in the data. Are there clusters? Non-
obvious relationships?
- Often referred to as “unsupervised learning”
- E.g., K-means clustering
Use Cases -> Understanding categories of customers,
cross-selling opportunities, etc…
2) Prediction: The data contains a particular attribute
(called the target attribute) and we want to learn how
the target attribute depends on the other attributes.
- Also referred to as “supervised learning”
- E.g., Support vector machines
Use Cases -> Building a model to predict customer
churn, fraud, etc…

Data
Prep
The Machine Learning Workflow - What are Data Scientists
doing?
DATA
Data
Prep
Build
Train
TestDeploy
Score
Act
(rules)
Data Preparation
– Data Scientists choose the data to
analyze and prepare them
Build / Train / Test phases
– Data scientists choose their modeling
program (SPSS Modeler, SAS, R, Python,
Open source, …)
– They train & test the models..
DEPLOY phase
– Data scientists export models and deploy
them in any runtime environment,
potentially z/OS.
SCORE phase
– Frequently scoring is executed in batch,
or as a service close to the data
warehouse, but certainly not with a 200
milliseconds response time in a z/OS
environment!
ACT phase – Get the Business Value!
– Business rules can be deployed to
replace or complement real-time scoring.
– Colocation of rules with transaction is
needed to optimize performance.The scoring and the rules execution can
take place in the transaction.

The first on-premise machine learning solution from IBM
to provide a feedback loop for progressively better model predictions
Understand model
performance with
automatic refresh that re-
trains models using fresh
data
Deploy models as REST
APIs for application
development
IBM Machine Learning for z/OS
Create, train and deploy self-learning models
Train DeployCreate
Create better models
faster with Cognitive
Assistant for Data
Scientists

A Use Case for Machine Learning: Insight-full Real-Time
Decision
This is the time during which a
transactional event is still occurring
–Someone is shopping at a store
–Someone is on the phone with a
customer service representative
–An electronic payment is being processed
–…
By acting on threats or opportunities as
they arise…
–Revenues can be increased (up-sell,
cross-sell)
–Customer churn can be reduced
In – transaction real-time analytics is about leveraging the power of analytics “in the
transactional moment” to achieve a more favorable outcome for a transactional event,
while the event is in progress The person visiting a
store buys more than he
or she otherwise would
have
What would have been an
over-payment is stopped
before it gets out the door
The person on the phone,
who was about to cancel
a service, instead re-ups
A commission of fraud is
stopped before it is
effected

Potential z/OS or non-z/OS
service integration
Decision – The 100% z/OS based Runtime Environment
Business
Rules
Policy
Regulation
Best Practices
Know-how
Business Critical
Queries
Transaction&
BatchWorkload
1
2
3
Add-on
Other Analytics services
Orchestration
withSimple
API
Risk
Clustering
Segmentation
Propensity
ML
Model Execution
ML
Model
Creation
Federated Data Analytics
For performance &
security!

Decision – The Journey
Three technology pillars enable real-time analytics in a z/OS application runtime
environment
– (1) Predictive analytics with Machine Learning on z/OS
– (2) Business Rules with Operational Decision Management – ODM on z/OS
– (3) Business critical queries with IBM DB2 & DB2 Analytics Accelerator
Each of the three pillars just mentioned delivers significant value on its own.
Combined, they can work together to bring agility in decision management.
Optionally adding when SLAs for specific use case permit
– Integration with Spark solutions for federated data analytics, CPLEX mathematical
algorithm for Optimization and Watson-based cognitive services
An organization can go right to “ultimate” in-transaction real-time analytics
capability, or take a phased approach – and realize benefits at each intermediate
stage in any order depending on the use case.

Access to Mainframe data without moving it
outside z Systems secured environment.
Reduce data latency
Minimize Cost & Complexity
Improve Data Governance & Security
A new approach for Enterprise Analytics with z Systems
Enrich transactions with real-time
analytics allowing optimized decision
management
Improve Business value of mainframe
applications
Conviction 3: z Systems is the Most Securable for Data!
Conviction 4: Integrate Open Source technology in the z
Systems Environment
Conviction 2: Accelerate insight and simplify implementation with z13 &
IBM DB2 Analytics Accelerator
Centralized data security
Tracking of activity to address audit and compliance
requirements
Highly available cryptography
Did you miss the LinuxONE announcement?
Linux Your Way, Linux Without Limits, Linux Without Risk
Conviction 1: Bring the analytics to the data for Right-Time insight
Bring the analytics in the application for Real-Time Decision Management!

Analytics with IMS Assets - 2017

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Analytics with IMS Assets - 2017

Similar to Analytics with IMS Assets - 2017 (20)

Recently uploaded

Recently uploaded (20)

Analytics with IMS Assets - 2017