SlideShare a Scribd company logo
1 of 30
Download to read offline
1 © Hortonworks Inc. 2011–2018. All rights reserved
Getting Started with Apache NiFi Across
Industries
Cindy Maike, VP of Industry Solutions – Hortonworks
Dan Potter, VP of Product Marketing - Attunity
2 © Hortonworks Inc. 2011–2018. All rights reserved
Dan Potter
VP Product Marketing
ATTUNITY
Cindy Maike
VP Industry Solutions
HORTONWORKS
3 © Hortonworks Inc. 2011–2018. All rights reserved
Agenda
• Connected Digital World Opportunities
• The Changing Data Landscape
• Industry Use Cases
• Attunity Replicate
• Q&A
4 © Hortonworks Inc. 2011–2018. All rights reserved
Mobile apps Online shopping Medical records Wearables Music streaming
5 © Hortonworks Inc. 2011–2018. All rights reserved
BIGGEST
BUSINESS MODEL
TRANSFORMATION SINCE
INDUSTRIAL
REVOLUTION
PROCEDURAL PROCESSING
CUSTOMER
SUPPLY
ENTERPRISE
PRODUCT
CONNECTED COMMUNITIES
ENTERPRISE CUSTOMER PRODUCT SUPPLY
6 © Hortonworks Inc. 2011–2018. All rights reserved
Opportunities to Create Business Value
Across Connected Communities
Cost savings from
predictive
maintenance
New revenues from
location aware
E-Commerce
Supply Chain
Data Exchange
New Models for
Commerce and Trade
Monetization of Data
Real-Time Operational
Models
New revenues by
monetized customer
profile and location data
Improve supply chain
efficiencies through
sharing of real-time data
© Hortonworks Inc. 2011–2018. All rights reserved
7 © Hortonworks Inc. 2011–2018. All rights reserved
CUSTOMER
SUPPLY
ENTERPRISE
PRODUCT
MODERN BUSINESS
MODELS
REQUIRES WE CREATE
CONNECTED COMMUNITIES
TO GET MORE
BUSINESS VALUE
• Real-time visibility
• Product performance
• Customer satisfaction
New models drive With High-Velocity
• Adjustments
• Remedies
• Responses
© Hortonworks Inc. 2011–2018. All rights reserved
8 © Hortonworks Inc. 2011–2018. All rights reserved
MULTIPLE CLUSTERS AND SOURCES
MULTIHYBRID
DATAPLANE SERVICE (DPS)
MANAGE, GOVERN, SECURE
DATA
LIFECYCLE
MANAGER
DATA
STEWARD
STUDIO*
ISV
SERVICES
*not yet available, coming soon
EXTENSIBLE SERVICES
IBM DSX*CLOUD-
BREAK*
DATA
ANALYTICS
STUDIO*
CONNECTED DATA PLATFORMS
HORTONWORKS
DATA PLATFORM (HDP®)
DATA-AT-REST
HORTONWORKS
DATAFLOW (HDF™)
DATA-IN-MOTION
MODERN DATA USE CASES
EDW
OPTIMIZATION
CYBER SECURITY DATA SCIENCE
ADVANCED
ANALYTICS
PARTNER
SOLUTIONS
IOT/ STREAMING
ANALYTICS
HORTONWORKS
CONNECTION
ENTERPRISE SUPPORT
PREMIER SUPPORT
EDUCATIONAL SERVICES
PROFESSIONAL SERVICES
COMMUNITY CONNECTION
HORTONWORKS
PLATFORM SERVICES
OPERATIONAL SERVICES
SMARTSENSE™
Global Data Management With Hortonworks and Attunity
9 © Hortonworks Inc. 2011–2018. All rights reserved
Example: Real-Time Supply Chain Analytics
ENTERPRISE
TRANSACTION
SOURCES
ERP, MES, SCM,
WMS, TMS, ETC. ENTERPRISE DATA LAKE
6
REAL-TIME
INVENTORY ACTION
ACT
1
I
N
G
E
S
T
1 INGEST
2 STORE
3 PROCESS
4 •Data Discovery
•Business Intelligence
ANALYZE
MONITOR 5
Deploy
4
LEARN
•Develop Models
•Machine Learning
Model Inputs
• Historic click-streams, POS data
• Historic inventory
locations/levels
• Historic Weather
• Historic order histories,
inventory locations and levels
• Historic disruptions
REAL-TIME DATA SOURCES
Connected Supply Chain
RFIDSocial WEB
POS
GPS
10 © Hortonworks Inc. 2011–2018. All rights reserved
Hortonworks & Attunity for Real-Time Connected Industries
ENTERPRISE
TRANSACTION
SOURCES
ENTERPRISE DATA LAKE
1
I
N
G
E
S
T
4
4
Deploy
INGEST
DATA IN
MOTION
MONITOR
STORE AND PROCESS
DATA AT
REST
ANALYZE
LEARN
1 5
ERP, MES, SCM,
WMS, TMS, ETC.
•Data Discovery
•Business Intelligence
DATA AT
REST
REAL-TIME DATA SOURCES
Connected Supply Chain
RFIDSocial WEB
POS
GPS
6
REAL-TIME
INVENTORY ACTION
ACT
Model Inputs
• Historic click-streams, POS data
• Historic inventory
locations/levels
• Historic Weather
• Historic order histories,
inventory locations and levels
• Historic disruptions
Illustration of Supply Chain
12 © Hortonworks Inc. 2011–2018. All rights reserved
12© 2017 Attunity
MODERN DATA INGEST
METADATA
HIVE
OPTIMIZED
STREAM
OPTIMIZED
CHANGE DATA CAPTURE
CLOUD ON PREM
WAREHOUSE MAINFRAME RDBMS SAP
▪ CDC (log-based) for
high performance, low
latency and low impact
▪ Single platform for all
key enterprise systems
▪ Hive-optimized for
HDP and Stream-
optimized for HDF
▪ Point-and-Click with
NO coding and NO
agents
13© 2018 Attunity
METADATA
HIVE
OPTIMIZED
STREAM
OPTIMIZED
Hive-optimized for HDP
and Stream-optimized for
HDF
Point-and-Click with NO
coding and NO agents
Automates creation of
analytics-ready Hive
dataset
Reconciles source data and
metadata updates
Transformation processing
pushed down to Hive
CHANGE DATA CAPTURE
CLOUD ON PREM
WAREHOUSE MAINFRAME RDBMS SAP
TRANSFORM & UPDATE
ATTUNITY + HORTONWORKS
14 © Hortonworks Inc. 2011–2018. All rights reserved
Industry Use Cases
15 © Hortonworks Inc. 2011–2018. All rights reserved
Insurance Industry - Business use cases
Emerging Tech, Real-time data and the Connected World
Smart
Cities &
Buildings
Smart
Factories /
Commercial
Connected
Life /
Health /
Medicine
IoT /
Robotics
Telematics
Shared
Economy
Smart
Homes
Cyber / AI /
Analytics
16 © Hortonworks Inc. 2011–2018. All rights reserved
Driving Actionable Intelligence in Insurance
Catastrophic
Event Data
Customer
Onboarding Data
Seismic
Data
Biometrics
Data
Usage-Based
Driver Data
Cyber Threat
Metadata
RISK & UNDERWRITING
ANALYSIS
USAGE-BASED
INSURANCE
CLAIMS ANALYTICS NEW PRODUCT
DEVELOPMENT
CYBER RISK
ANALYTICS
Drones &
Aerial Imagery
Claims Docs,
Notes &
Diaries
Weather &
Environment
Underwriting
Analysis
Policy
Histories
Photos
17 © Hortonworks Inc. 2011–2018. All rights reserved
Nifi and Attunity Replicate in Insurance
Insurance
Risk & underwriting analysis; Usage-based insurance; Claims analytics; New product
development, and cyber-risk analytics
· Apache NiFi: Usage-Based Driver Data, Weather & Environment, Drones & Aerial Imagery,
Seismic Data, Biometrics Data, Cyber Threat Metadata, Catastrophic Event Data, Photos
· Attunity Replicate: Underwriting Analysis, Customer Onboarding Data, Underwriting Analysis,
Policy histories, Claims Docs, Notes & Diaries
18 © Hortonworks Inc. 2011–2018. All rights reserved
Actionable Intelligence Powers Modern Manufacturing
Defect
Testing Data
Product
Designs
MES
System
s
RFID
Streams
SCADA
Systems
Shop Floor
Sensors
PREVENTATIVE
MAINTENANCE
SUPPLY CHAIN
OPTIMIZATION
YIELD MAXIMIZATION QUALITY
CONTROL
RECALL AVOIDANCE
ERP
Systems
Supplier
Receipts
Machine
Data
Assembly
Line Sensors
Data
Historians
Work
Orders
19 © Hortonworks Inc. 2011–2018. All rights reserved
Data Drives the Connected Car – Must include SAP SCM, etc.
Insurance
Premiums
Warranties
Recalls
Pricing
Models
Design
Innovation
Autonomous
Driving
Connected
City Infotainment
Sensors
Scheduled
Maintenance
Predictive
Maintenance
Route
Optimization
INSURANCE
COMPANIES
GOVERNMENT
AGENCIES
INFOTAINMENT
PROVIDERS
SOFTWARE
COMPANIES
AUTO
MAKERS
20 © Hortonworks Inc. 2011–2018. All rights reserved
Attunity Replicate Enables Global Manufacturing and Automotive
Transformation
Manufacturing
Preventative Maintenance, Supply Chain Optimization, Yield Maximization, Quality Control, and
Recall Avoidance Development
· Apache NiFi: Machine Data, Testing Data, Assembly Line Sensors, RFID Streams, Shop Floors
· Attunity Replicate: Scada systems, Work Orders, Supplier Receipts, ERP Systems, MES, Data
Historians, Product Designs
21 © Hortonworks Inc. 2011–2018. All rights reserved
Actionable Intelligence Transforms Energy and Utilities
Asset
Data
Customer
Surveys
Weather &
Environmental
Service Fleet
GPS Data
Smart Meter
Streams
Commodity
Prices
REVENUE
PROTECTION
SINGLE VIEW
OF CUSTOMER
PREDICTIVE EQUIPMENT
MAINTENANCE
CONSERVATION VOLTAGE
REDUCTION
COMMODITY TRADING
Social
Media
GIS
Data
SCADA Outage
Histories
CIS
Records
EDW
22 © Hortonworks Inc. 2011–2018. All rights reserved
Attunity Replicate Provides Pumps More Relevant Data for Energy
and Utilities
Utilities
REVENUE PROTECTION; SINGLE VIEW OF CUSTOMER; PREDICTIVE EQUIPMENT
MAINTENANCE; CONSERVATION VOLTAGE REDUCTION; COMMODITY TRADING
· Apache NiFi: smart meter streams, GIS data, social media, weather & environmental, CIS
record, customer surveys, commodity prices
·Attunity Replicate: SCADA, EDW, asset data, outage histories
23 © Hortonworks Inc. 2011–2018. All rights reserved
Actionable Intelligence Powers Today’s Financial Services
OFAC
Lists
Credit
Records
ATM
Streams Transactions
& Wires
Stock
Tickers
Trade
Settlements
DIGITAL CUSTOMER
360
RISK DATA
AGGREGATION
ANTI-MONEY LAUNDERING FRAUD
DETECTION
TRADE
SURVEILLANCE
Mobile
App Data
Trade
Data
Web
Logs
Banker
Notes
Demographic
Data
Customer
Transaction
Data
24 © Hortonworks Inc. 2011–2018. All rights reserved
Attunity Replicate Connects You to Your Money and Integrates with
Key Market Resources
Financial Services
DIGITAL CUSTOMER 360; RISK DATA AGGREGATION; ANTI-MONEY LAUNDERING; FRAUD
DETECTION; TRADE SURVELLIANCE
· Apache NiFi: web logs, trade data, mobile app data, ATM streams, OFAC lists, transactions &
wires, demographic data, trade settlements.
· Attunity Replicate: customer transaction data, OFAC lists, banker notes, credit records, stock
tickers
STREAMING REAL-TIME DATA
26© 2018 Attunity
Requirement Detail
Latency Need to update data sets in near real-time
Scale High volume and variety of data ingestion
Flexibility Change platforms quickly
Time to Value Dependency on developers
Performance Negative impact on production
Efficiency Slow, disruptive full load replication
NEEDS AND CHALLENGES
27© 2018 Attunity
AUTOMATE YOUR DATA PIPELINE
Consume
ANALYZE
MONITOR
SAP
RDBMS
DATA
WAREHOUSE
FILES
MAINFRAME
Land Structure Provision
CREATE
UPDATE
ENRICH
SUBSET
STANDARDIZE
MERGE
FORMAT
Raw
Deltas
Historical
Data Store
HDS
ODS
Snapshot
Model
PREPARE
UNIFY
Mart
Views
Data ingested into
data lake
Data continuously
updated and merged
into historic data
store
Subsets created to
meet analytic
requirements
Create marts for
queries and analysis
28 © Hortonworks Inc. 2011–2018. All rights reserved
Why Attunity with Hortonworks?
Data Ingestion Solution
of Choice
Modern Data Platforms Modern Industry Use Cases
• Data Lakes
• Process Optimization
• Machine Learning & AI
Supporting The Use Cases of the Now and Future
29 © Hortonworks Inc. 2011–2018. All rights reserved
Questions?
30 © Hortonworks Inc. 2011–2018. All rights reserved
Download Apache NiFi
for Dummies Today!
https://hortonworks.com/ebook/ni-fi-
dummies/
Industry Solution Guide
for Using Apache NiFi
and Attunity Replicate
https://hortonworks.com/info/industr
y-solution-guide-using-apache-nifi-
attunity-replicate/
31 © Hortonworks Inc. 2011–2018. All rights reserved
Thank you
For more information, go to: www.Attunity.com/nifibook

More Related Content

More from Hortonworks

IBM+Hortonworks = Transformation of the Big Data Landscape
IBM+Hortonworks = Transformation of the Big Data LandscapeIBM+Hortonworks = Transformation of the Big Data Landscape
IBM+Hortonworks = Transformation of the Big Data LandscapeHortonworks
 
Premier Inside-Out: Apache Druid
Premier Inside-Out: Apache DruidPremier Inside-Out: Apache Druid
Premier Inside-Out: Apache DruidHortonworks
 
Accelerating Data Science and Real Time Analytics at Scale
Accelerating Data Science and Real Time Analytics at ScaleAccelerating Data Science and Real Time Analytics at Scale
Accelerating Data Science and Real Time Analytics at ScaleHortonworks
 
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATATIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATAHortonworks
 
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...Hortonworks
 
Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Delivering Real-Time Streaming Data for Healthcare Customers: ClearsenseDelivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Delivering Real-Time Streaming Data for Healthcare Customers: ClearsenseHortonworks
 
Making Enterprise Big Data Small with Ease
Making Enterprise Big Data Small with EaseMaking Enterprise Big Data Small with Ease
Making Enterprise Big Data Small with EaseHortonworks
 
Webinewbie to Webinerd in 30 Days - Webinar World Presentation
Webinewbie to Webinerd in 30 Days - Webinar World PresentationWebinewbie to Webinerd in 30 Days - Webinar World Presentation
Webinewbie to Webinerd in 30 Days - Webinar World PresentationHortonworks
 
Driving Digital Transformation Through Global Data Management
Driving Digital Transformation Through Global Data ManagementDriving Digital Transformation Through Global Data Management
Driving Digital Transformation Through Global Data ManagementHortonworks
 
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming FeaturesHDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming FeaturesHortonworks
 
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...Hortonworks
 
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDCUnlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDCHortonworks
 
4 Essential Steps for Managing Sensitive Data
4 Essential Steps for Managing Sensitive Data4 Essential Steps for Managing Sensitive Data
4 Essential Steps for Managing Sensitive DataHortonworks
 
5 Steps to Create a Company Culture that Embraces the Power of Data
5 Steps to Create a Company Culture that Embraces the Power of Data5 Steps to Create a Company Culture that Embraces the Power of Data
5 Steps to Create a Company Culture that Embraces the Power of DataHortonworks
 
Exploring the Heated-and Completely Unnecessary- Data Lake Debate
Exploring the Heated-and Completely Unnecessary- Data Lake DebateExploring the Heated-and Completely Unnecessary- Data Lake Debate
Exploring the Heated-and Completely Unnecessary- Data Lake DebateHortonworks
 
Sprint's Data Modernization Journey
Sprint's Data Modernization JourneySprint's Data Modernization Journey
Sprint's Data Modernization JourneyHortonworks
 
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data PlatformModernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data PlatformHortonworks
 
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017 Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017 Hortonworks
 
Benefits of Transferring Real-Time Data to Hadoop at Scale
Benefits of Transferring Real-Time Data to Hadoop at ScaleBenefits of Transferring Real-Time Data to Hadoop at Scale
Benefits of Transferring Real-Time Data to Hadoop at ScaleHortonworks
 
Streamline Apache Hadoop Operations with Apache Ambari and SmartSense
Streamline Apache Hadoop Operations with Apache Ambari and SmartSenseStreamline Apache Hadoop Operations with Apache Ambari and SmartSense
Streamline Apache Hadoop Operations with Apache Ambari and SmartSenseHortonworks
 

More from Hortonworks (20)

IBM+Hortonworks = Transformation of the Big Data Landscape
IBM+Hortonworks = Transformation of the Big Data LandscapeIBM+Hortonworks = Transformation of the Big Data Landscape
IBM+Hortonworks = Transformation of the Big Data Landscape
 
Premier Inside-Out: Apache Druid
Premier Inside-Out: Apache DruidPremier Inside-Out: Apache Druid
Premier Inside-Out: Apache Druid
 
Accelerating Data Science and Real Time Analytics at Scale
Accelerating Data Science and Real Time Analytics at ScaleAccelerating Data Science and Real Time Analytics at Scale
Accelerating Data Science and Real Time Analytics at Scale
 
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATATIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
 
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
 
Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Delivering Real-Time Streaming Data for Healthcare Customers: ClearsenseDelivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense
 
Making Enterprise Big Data Small with Ease
Making Enterprise Big Data Small with EaseMaking Enterprise Big Data Small with Ease
Making Enterprise Big Data Small with Ease
 
Webinewbie to Webinerd in 30 Days - Webinar World Presentation
Webinewbie to Webinerd in 30 Days - Webinar World PresentationWebinewbie to Webinerd in 30 Days - Webinar World Presentation
Webinewbie to Webinerd in 30 Days - Webinar World Presentation
 
Driving Digital Transformation Through Global Data Management
Driving Digital Transformation Through Global Data ManagementDriving Digital Transformation Through Global Data Management
Driving Digital Transformation Through Global Data Management
 
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming FeaturesHDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
 
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
 
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDCUnlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDC
 
4 Essential Steps for Managing Sensitive Data
4 Essential Steps for Managing Sensitive Data4 Essential Steps for Managing Sensitive Data
4 Essential Steps for Managing Sensitive Data
 
5 Steps to Create a Company Culture that Embraces the Power of Data
5 Steps to Create a Company Culture that Embraces the Power of Data5 Steps to Create a Company Culture that Embraces the Power of Data
5 Steps to Create a Company Culture that Embraces the Power of Data
 
Exploring the Heated-and Completely Unnecessary- Data Lake Debate
Exploring the Heated-and Completely Unnecessary- Data Lake DebateExploring the Heated-and Completely Unnecessary- Data Lake Debate
Exploring the Heated-and Completely Unnecessary- Data Lake Debate
 
Sprint's Data Modernization Journey
Sprint's Data Modernization JourneySprint's Data Modernization Journey
Sprint's Data Modernization Journey
 
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data PlatformModernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
 
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017 Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017
 
Benefits of Transferring Real-Time Data to Hadoop at Scale
Benefits of Transferring Real-Time Data to Hadoop at ScaleBenefits of Transferring Real-Time Data to Hadoop at Scale
Benefits of Transferring Real-Time Data to Hadoop at Scale
 
Streamline Apache Hadoop Operations with Apache Ambari and SmartSense
Streamline Apache Hadoop Operations with Apache Ambari and SmartSenseStreamline Apache Hadoop Operations with Apache Ambari and SmartSense
Streamline Apache Hadoop Operations with Apache Ambari and SmartSense
 

Recently uploaded

Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda
 
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditManual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditSkynet Technologies
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsYoss Cohen
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfROWELL MARQUINA
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessWSO2
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Nikki Chapple
 

Recently uploaded (20)

Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
 
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditManual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance Audit
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platforms
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdf
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with Platformless
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
 

Getting Started with Apache NiFi Across Industries

  • 1. 1 © Hortonworks Inc. 2011–2018. All rights reserved Getting Started with Apache NiFi Across Industries Cindy Maike, VP of Industry Solutions – Hortonworks Dan Potter, VP of Product Marketing - Attunity
  • 2. 2 © Hortonworks Inc. 2011–2018. All rights reserved Dan Potter VP Product Marketing ATTUNITY Cindy Maike VP Industry Solutions HORTONWORKS
  • 3. 3 © Hortonworks Inc. 2011–2018. All rights reserved Agenda • Connected Digital World Opportunities • The Changing Data Landscape • Industry Use Cases • Attunity Replicate • Q&A
  • 4. 4 © Hortonworks Inc. 2011–2018. All rights reserved Mobile apps Online shopping Medical records Wearables Music streaming
  • 5. 5 © Hortonworks Inc. 2011–2018. All rights reserved BIGGEST BUSINESS MODEL TRANSFORMATION SINCE INDUSTRIAL REVOLUTION PROCEDURAL PROCESSING CUSTOMER SUPPLY ENTERPRISE PRODUCT CONNECTED COMMUNITIES ENTERPRISE CUSTOMER PRODUCT SUPPLY
  • 6. 6 © Hortonworks Inc. 2011–2018. All rights reserved Opportunities to Create Business Value Across Connected Communities Cost savings from predictive maintenance New revenues from location aware E-Commerce Supply Chain Data Exchange New Models for Commerce and Trade Monetization of Data Real-Time Operational Models New revenues by monetized customer profile and location data Improve supply chain efficiencies through sharing of real-time data © Hortonworks Inc. 2011–2018. All rights reserved
  • 7. 7 © Hortonworks Inc. 2011–2018. All rights reserved CUSTOMER SUPPLY ENTERPRISE PRODUCT MODERN BUSINESS MODELS REQUIRES WE CREATE CONNECTED COMMUNITIES TO GET MORE BUSINESS VALUE • Real-time visibility • Product performance • Customer satisfaction New models drive With High-Velocity • Adjustments • Remedies • Responses © Hortonworks Inc. 2011–2018. All rights reserved
  • 8. 8 © Hortonworks Inc. 2011–2018. All rights reserved MULTIPLE CLUSTERS AND SOURCES MULTIHYBRID DATAPLANE SERVICE (DPS) MANAGE, GOVERN, SECURE DATA LIFECYCLE MANAGER DATA STEWARD STUDIO* ISV SERVICES *not yet available, coming soon EXTENSIBLE SERVICES IBM DSX*CLOUD- BREAK* DATA ANALYTICS STUDIO* CONNECTED DATA PLATFORMS HORTONWORKS DATA PLATFORM (HDP®) DATA-AT-REST HORTONWORKS DATAFLOW (HDF™) DATA-IN-MOTION MODERN DATA USE CASES EDW OPTIMIZATION CYBER SECURITY DATA SCIENCE ADVANCED ANALYTICS PARTNER SOLUTIONS IOT/ STREAMING ANALYTICS HORTONWORKS CONNECTION ENTERPRISE SUPPORT PREMIER SUPPORT EDUCATIONAL SERVICES PROFESSIONAL SERVICES COMMUNITY CONNECTION HORTONWORKS PLATFORM SERVICES OPERATIONAL SERVICES SMARTSENSE™ Global Data Management With Hortonworks and Attunity
  • 9. 9 © Hortonworks Inc. 2011–2018. All rights reserved Example: Real-Time Supply Chain Analytics ENTERPRISE TRANSACTION SOURCES ERP, MES, SCM, WMS, TMS, ETC. ENTERPRISE DATA LAKE 6 REAL-TIME INVENTORY ACTION ACT 1 I N G E S T 1 INGEST 2 STORE 3 PROCESS 4 •Data Discovery •Business Intelligence ANALYZE MONITOR 5 Deploy 4 LEARN •Develop Models •Machine Learning Model Inputs • Historic click-streams, POS data • Historic inventory locations/levels • Historic Weather • Historic order histories, inventory locations and levels • Historic disruptions REAL-TIME DATA SOURCES Connected Supply Chain RFIDSocial WEB POS GPS
  • 10. 10 © Hortonworks Inc. 2011–2018. All rights reserved Hortonworks & Attunity for Real-Time Connected Industries ENTERPRISE TRANSACTION SOURCES ENTERPRISE DATA LAKE 1 I N G E S T 4 4 Deploy INGEST DATA IN MOTION MONITOR STORE AND PROCESS DATA AT REST ANALYZE LEARN 1 5 ERP, MES, SCM, WMS, TMS, ETC. •Data Discovery •Business Intelligence DATA AT REST REAL-TIME DATA SOURCES Connected Supply Chain RFIDSocial WEB POS GPS 6 REAL-TIME INVENTORY ACTION ACT Model Inputs • Historic click-streams, POS data • Historic inventory locations/levels • Historic Weather • Historic order histories, inventory locations and levels • Historic disruptions Illustration of Supply Chain
  • 11. 12 © Hortonworks Inc. 2011–2018. All rights reserved 12© 2017 Attunity MODERN DATA INGEST METADATA HIVE OPTIMIZED STREAM OPTIMIZED CHANGE DATA CAPTURE CLOUD ON PREM WAREHOUSE MAINFRAME RDBMS SAP ▪ CDC (log-based) for high performance, low latency and low impact ▪ Single platform for all key enterprise systems ▪ Hive-optimized for HDP and Stream- optimized for HDF ▪ Point-and-Click with NO coding and NO agents
  • 12. 13© 2018 Attunity METADATA HIVE OPTIMIZED STREAM OPTIMIZED Hive-optimized for HDP and Stream-optimized for HDF Point-and-Click with NO coding and NO agents Automates creation of analytics-ready Hive dataset Reconciles source data and metadata updates Transformation processing pushed down to Hive CHANGE DATA CAPTURE CLOUD ON PREM WAREHOUSE MAINFRAME RDBMS SAP TRANSFORM & UPDATE ATTUNITY + HORTONWORKS
  • 13. 14 © Hortonworks Inc. 2011–2018. All rights reserved Industry Use Cases
  • 14. 15 © Hortonworks Inc. 2011–2018. All rights reserved Insurance Industry - Business use cases Emerging Tech, Real-time data and the Connected World Smart Cities & Buildings Smart Factories / Commercial Connected Life / Health / Medicine IoT / Robotics Telematics Shared Economy Smart Homes Cyber / AI / Analytics
  • 15. 16 © Hortonworks Inc. 2011–2018. All rights reserved Driving Actionable Intelligence in Insurance Catastrophic Event Data Customer Onboarding Data Seismic Data Biometrics Data Usage-Based Driver Data Cyber Threat Metadata RISK & UNDERWRITING ANALYSIS USAGE-BASED INSURANCE CLAIMS ANALYTICS NEW PRODUCT DEVELOPMENT CYBER RISK ANALYTICS Drones & Aerial Imagery Claims Docs, Notes & Diaries Weather & Environment Underwriting Analysis Policy Histories Photos
  • 16. 17 © Hortonworks Inc. 2011–2018. All rights reserved Nifi and Attunity Replicate in Insurance Insurance Risk & underwriting analysis; Usage-based insurance; Claims analytics; New product development, and cyber-risk analytics · Apache NiFi: Usage-Based Driver Data, Weather & Environment, Drones & Aerial Imagery, Seismic Data, Biometrics Data, Cyber Threat Metadata, Catastrophic Event Data, Photos · Attunity Replicate: Underwriting Analysis, Customer Onboarding Data, Underwriting Analysis, Policy histories, Claims Docs, Notes & Diaries
  • 17. 18 © Hortonworks Inc. 2011–2018. All rights reserved Actionable Intelligence Powers Modern Manufacturing Defect Testing Data Product Designs MES System s RFID Streams SCADA Systems Shop Floor Sensors PREVENTATIVE MAINTENANCE SUPPLY CHAIN OPTIMIZATION YIELD MAXIMIZATION QUALITY CONTROL RECALL AVOIDANCE ERP Systems Supplier Receipts Machine Data Assembly Line Sensors Data Historians Work Orders
  • 18. 19 © Hortonworks Inc. 2011–2018. All rights reserved Data Drives the Connected Car – Must include SAP SCM, etc. Insurance Premiums Warranties Recalls Pricing Models Design Innovation Autonomous Driving Connected City Infotainment Sensors Scheduled Maintenance Predictive Maintenance Route Optimization INSURANCE COMPANIES GOVERNMENT AGENCIES INFOTAINMENT PROVIDERS SOFTWARE COMPANIES AUTO MAKERS
  • 19. 20 © Hortonworks Inc. 2011–2018. All rights reserved Attunity Replicate Enables Global Manufacturing and Automotive Transformation Manufacturing Preventative Maintenance, Supply Chain Optimization, Yield Maximization, Quality Control, and Recall Avoidance Development · Apache NiFi: Machine Data, Testing Data, Assembly Line Sensors, RFID Streams, Shop Floors · Attunity Replicate: Scada systems, Work Orders, Supplier Receipts, ERP Systems, MES, Data Historians, Product Designs
  • 20. 21 © Hortonworks Inc. 2011–2018. All rights reserved Actionable Intelligence Transforms Energy and Utilities Asset Data Customer Surveys Weather & Environmental Service Fleet GPS Data Smart Meter Streams Commodity Prices REVENUE PROTECTION SINGLE VIEW OF CUSTOMER PREDICTIVE EQUIPMENT MAINTENANCE CONSERVATION VOLTAGE REDUCTION COMMODITY TRADING Social Media GIS Data SCADA Outage Histories CIS Records EDW
  • 21. 22 © Hortonworks Inc. 2011–2018. All rights reserved Attunity Replicate Provides Pumps More Relevant Data for Energy and Utilities Utilities REVENUE PROTECTION; SINGLE VIEW OF CUSTOMER; PREDICTIVE EQUIPMENT MAINTENANCE; CONSERVATION VOLTAGE REDUCTION; COMMODITY TRADING · Apache NiFi: smart meter streams, GIS data, social media, weather & environmental, CIS record, customer surveys, commodity prices ·Attunity Replicate: SCADA, EDW, asset data, outage histories
  • 22. 23 © Hortonworks Inc. 2011–2018. All rights reserved Actionable Intelligence Powers Today’s Financial Services OFAC Lists Credit Records ATM Streams Transactions & Wires Stock Tickers Trade Settlements DIGITAL CUSTOMER 360 RISK DATA AGGREGATION ANTI-MONEY LAUNDERING FRAUD DETECTION TRADE SURVEILLANCE Mobile App Data Trade Data Web Logs Banker Notes Demographic Data Customer Transaction Data
  • 23. 24 © Hortonworks Inc. 2011–2018. All rights reserved Attunity Replicate Connects You to Your Money and Integrates with Key Market Resources Financial Services DIGITAL CUSTOMER 360; RISK DATA AGGREGATION; ANTI-MONEY LAUNDERING; FRAUD DETECTION; TRADE SURVELLIANCE · Apache NiFi: web logs, trade data, mobile app data, ATM streams, OFAC lists, transactions & wires, demographic data, trade settlements. · Attunity Replicate: customer transaction data, OFAC lists, banker notes, credit records, stock tickers
  • 25. 26© 2018 Attunity Requirement Detail Latency Need to update data sets in near real-time Scale High volume and variety of data ingestion Flexibility Change platforms quickly Time to Value Dependency on developers Performance Negative impact on production Efficiency Slow, disruptive full load replication NEEDS AND CHALLENGES
  • 26. 27© 2018 Attunity AUTOMATE YOUR DATA PIPELINE Consume ANALYZE MONITOR SAP RDBMS DATA WAREHOUSE FILES MAINFRAME Land Structure Provision CREATE UPDATE ENRICH SUBSET STANDARDIZE MERGE FORMAT Raw Deltas Historical Data Store HDS ODS Snapshot Model PREPARE UNIFY Mart Views Data ingested into data lake Data continuously updated and merged into historic data store Subsets created to meet analytic requirements Create marts for queries and analysis
  • 27. 28 © Hortonworks Inc. 2011–2018. All rights reserved Why Attunity with Hortonworks? Data Ingestion Solution of Choice Modern Data Platforms Modern Industry Use Cases • Data Lakes • Process Optimization • Machine Learning & AI Supporting The Use Cases of the Now and Future
  • 28. 29 © Hortonworks Inc. 2011–2018. All rights reserved Questions?
  • 29. 30 © Hortonworks Inc. 2011–2018. All rights reserved Download Apache NiFi for Dummies Today! https://hortonworks.com/ebook/ni-fi- dummies/ Industry Solution Guide for Using Apache NiFi and Attunity Replicate https://hortonworks.com/info/industr y-solution-guide-using-apache-nifi- attunity-replicate/
  • 30. 31 © Hortonworks Inc. 2011–2018. All rights reserved Thank you For more information, go to: www.Attunity.com/nifibook