Analytic Platforms with Mark Madsen, John O'Brien and ParAccel
Live Webcast Dec. 5, 2012
There's a good reason why so many people are talking about analytic platforms these days. The surge in popularity of Big Data, coupled with the need to reconcile this new source of insights with Business Intelligence and Data Warehousing, has fueled a wave of innovation for synthesizing analytical capabilities. What are the latest innovations in analytic platforms? Check out this episode of Hot Technologies to find out!
Veteran Analysts Mark Madsen of Third Nature and John O'Brien of Radiant Advisors will offer their insights on what to look for in a robust analytic platform. They'll then take a briefing from Walter Maguire of ParAccel, who will provide details about his company's platform offering, which includes a high-performance analytic database, Hadoop integration, and innovative extensions that allow companies to embed analytics in business process, create big data apps, and create on demand access to 100s of new data sources.
Visit: http://insideanalysis.com
4. ANALYTIC
PLATFORMS
ž Analytic
Platforms
represent
the
next
major
phase
in
the
evolution
of
Business
Intelligence
and
Analytics
ž These
platforms
should
foster
collaboration
and
transparency
ž Users
should
be
enabled
to
access
and
analyze
the
data
they
want,
quickly
and
effectively
5. ANALYST:
Mark
Madsen
THE
LINE
UP
CEO,
Third
Nature
Inc.
ANALYST:
John
O’Brien
Principal
&
CEO,
Radiant
Advisors
GUEST:
Walter
Maguire
Director
of
Analytics,
ParAccel
24. Enabling
Big
Data
ApplicaHons
Walter
Maguire,
Director
of
AnalyHcs
Copyright 2012 ParAccel, Inc. 24
25. ParAccel
Analy.c
Pla?orm
is…
…built
for
high
performance,
interac.ve
analy.cs.
On
Demand
Integra.on
Integrated
Analy.cs
Database
ParAccel
Analy.c
Pla?orm
Basic
AnalyHcs
Teradata
Advanced
AnalyHcs
Hadoop
Analy.c
Engine
Streaming
Data
Columnar
ApplicaHons
Compression
Compiled
Parallel
Processing
SQL
OpHmizaHon
Data
Scale
In-‐Memory
Op.on
Available
Plan
OpHmizaHon
AnalyHc
Scale
ExecuHon
OpHmizaHon
User
Scale
Comms
OpHmizaHon
InteracHve
Scale
I/O
OpHmizaHon
Copyright 2012 ParAccel, Inc. 25
26. ParAccel
technology
is
the
first
to
deliver
on
Coopera.ve
Analy.c
Processing
SQL-‐Based
Business
Advanced
Analy.c
Intelligence
and
Analy.cs
Applica.ons
Repor.ng
Tools
ParAccel
Analy.c
Pla?orm
Enterprise
Hadoop
Data
Warehouse
On
Demand
Integra.on
Embedded
3rd
Party
Big
Data
Machine
Opera.onal
Streaming
Analy.cs
Info
Logs
Apps
Data
Data
Data
Provider
Copyright 2012 ParAccel, Inc. 26
27. ParAccel
ODI
Services
makes
our
pla?orm
the
analy.c
engine
for
en.re
ecosystems.
ParAccel
Analy.c
Pla?orm
Enterprise
Hadoop
Data
Warehouse
1. Share
both
data
and
processes
in
both
direcHons
2. Transform
incoming
data
for
analyHc
performance
3. Interact
with
many
programming
languages
(Java,
Python,
more)
4. Persist
or
stream
data
through
analyHc
processing
5. Rapidly
build
new
On
Demand
IntegraHon
modules
On
Demand
Integra.on
Services
Embedded
3rd
Party
Big
Data
Machine
Opera.onal
Streaming
Analy.cs
Info
Logs
Apps
Data
Data
Data
Provider
Copyright 2012 ParAccel, Inc. 27
28. One
Size
Does
Not
Fit
All:
Why
an
Ecosystem?
ReporHng
AnalyHcs
Archiving
Dashboards
Data
Mining
Filtering
StaHc
Analysis
Dynamic
Analysis
Text
Search
OLAP
Complexity
Text
AnalyHcs
TransformaHon
Copyright 2011 ParAccel, Inc. 28
29. The
Best
Way
to
Do
Analy.cs
on
Hadoop
Data
Create
a
high-‐performance,
node-‐to-‐node,
bi-‐
direcHonal,
connecHon
between
Hadoop
and
an
analyHc
plaXorm
that
is
capable
of
sharing
both
data
and
processes
so
that
the
analyHc
plaXorm
becomes
an
extension
of
the
Hadoop
cluster
and
you
can
uHlize
the
lingua
franca
of
analyHcs,
SQL.
Copyright 2012 ParAccel, Inc. 29
30. Read
from
Hadoop:
INSERT
INTO
mytable
SELECT
*
FROM
HadoopIn(with
hfs_name( hadoopfile )
mr_job( xyz )
pa_schema( mytable ));
Write
to
Hadoop:
SELECT
num_rows
FROM
HadoopOut(on
(select
*
from
mytable)
WITH
hdfs_name(
hadoopfile ));
Copyright 2012 ParAccel, Inc. 30
31. What s
Next
for
the
Hadoop
ODI?
HCatalog
Integra.on
• Apache
HCatalog
is
a
table
and
storage
management
layer
for
Hadoop
Provides
table
abstracHon
for
HDFS
file
for
various
data
processing
tools
• ODI
Scan
filters
UDF
Filters
from
the
SQL
will
be
pushed
down
to
Hadoop
as
parHHon
filters
Greatly
simplify
invesHgaHve
workflow
on
large
volumes
of
data
in
Hadoop
before
bringing
it
into
ParAccel
Simplify
development
of
Hadoop
to
ParAccel
integraHons
Copyright 2012 ParAccel, Inc. 31
32. ODI
Services
Architecture
Overview
Leader
Node
ODI
Services
Service
Mgmt.
Service
Context
Compute
Node
Compute
Node
Perl
Perl
Python
Python
Services
Services
Services
Java
Java
ODI
ODI
Bash
Bash
R
R
Etc.
Etc.
Compute
Node
Perl
Python
Services
Java
ODI
Bash
R
Etc.
33. ODI
Services
Architecture
Overview
Leader
Node
• Job Progress & Status
• Installation
ODI
Services
• Logging
Service
Mgmt.
• Balancing
Service
Context
• Optimization
Compute
Node
Compute
Node
Perl
Perl
Python
Python
Services
Services
Services
Java
Java
ODI
ODI
Bash
Bash
R
R
Etc.
Etc.
Compute
Node
Perl
Python
Services
Java
ODI
Bash
R
Etc.
34. ODI
Services
Architecture
Overview
Leader
Node
• Job Progress & Status
• Installation
ODI
Services
• Logging
Service
Mgmt.
• Balancing
Service
Context
• Optimization
Compute
Node
Compute
Node
Perl
Perl
Python
Python
Services
Services
Services
Java
Java
ODI
ODI
Bash
Bash
R
R
Etc.
Etc.
STDIN
STDOUT
STDERR Compute
Node
Metadata Perl
Mgmt Framework Python
Services
Java
ODI
Bash
R
Etc.
35. ODI
Services
Architecture
Overview
Leader
Node
• Job Progress & Status
• Installation
ODI
Services
• Logging
Service
Mgmt.
• Balancing
Service
Context
• Optimization
Compute
Node
Compute
Node
Perl
Perl
Python
Python
Services
Services
Services
Java
Java
ODI
ODI
Bash
Bash
R
R
Etc.
Etc.
STDIN
STDOUT
STDERR Compute
Node
Metadata Perl
• Command line
Mgmt Framework Python
executable
Services
Java
• 3rd party
ODI
Bash
interpreter (e.g.
R
Perl, Python,
Etc.
Java VM)
36. Developing
and
Deploying
ODIs
Write
command
line
executable
or
interpreted
script
Test
with
ODI
Services
test
harness
Load
to
lead
node
Lead
node
distributes
ODI
across
the
compute
nodes
Copyright 2011 ParAccel, Inc. 36
37. Developing
and
Deploying
ODIs
o
Enables
a
spectrum
of
use
cases
from
fast
prototyping
to
one-‐off
and
producHon
data
loads/unloads
o
No
need
to
code
to
C++
APIs
or
be
exposed
to
any
complexity
o
Fast
development
o
Handles
parallelism
for
you
o
Simple
protocol
o
Logging
o
Monitoring
progress
Copyright 2011 ParAccel, Inc. 37
38. ODI
services:
examples
Event
Capture
Smart
Meter
Logging
RFID
Tag
Capture
Tweets,
Facebook,
consolidated
social
streams
Web
services
(Salesforce,
Eloqua,
Omniture,
etc.)
Enterprise
Semi-‐Structured
sources
(Outlook,
Gmail,
Zendesk,
etc.)
Embedded
business
processes
(ex:
call
center,
distribuHon
rouHng)
Copyright 2011 ParAccel, Inc. 38
39. Coopera.ve
Analy.c
Processing
is
the
Future
SQL-‐Based
Business
Advanced
Analy.c
Intelligence
and
Analy.cs
Applica.ons
Repor.ng
Tools
ParAccel
Analy.c
Pla?orm
Enterprise
Hadoop
Data
Warehouse
On
Demand
Integra.on
Embedded
3rd
Party
Big
Data
Machine
Opera.onal
Streaming
Analy.cs
Info
Logs
Apps
Data
Data
Data
Provider
Copyright 2012 ParAccel, Inc. 39