Speaker: Ted Orme

1. 3 November 2016
When Big Data Meets Fast
Data
Big Data London 2016
Ted Orme
VP Technology EMEA

2. © 2016 Attunity
• The Evolution of Hadoop
• Enterprise ready
• From batch to real-time
• Now add Cloud
• It’s all About the Architecture
• Lambda / Kappa
• How Change Data Capture (CDC) Enables Big and Fast Data
When Big Data Meets Fast Data

3. © 2016 Attunity
The Evolution of Hadoop
• Operationalising Hadoop
• Real-time and streaming
• Hadoop and the cloud
The First 10 Years

4. © 2016 Attunity
• Hadoop has won the battle
• Moved out of Dev and Test
• Enterprise demands more
• Closer to production
• Business impact
Sqoop
Financial Services Use Case
• 1000+ business applications
• Classic data lake
• Moved to level 3 infrastructure
supporting level 1 applications
Operationalising Hadoop

5. © 2016 Attunity
The Evolution of Hadoop
• Operationalising Hadoop
• Real-time and streaming
• Hadoop and the cloud
The First 10 Years

6. © 2016 Attunity
Real-time Streaming with Kafka
• Real-time processing
• Real-time monitoring
• Real-time Hadoop
• Big and Fast Data
• Scalable to 1000’s applications
• Streaming Data to Kafka
• Multiple data Feeds
• Multiple data Consumers
• Online Credit card transaction failure
 Under 20 seconds a real person calling you back
and helping with the transaction
Online Retail – Use Case

7. © 2016 Attunity
The Evolution of Hadoop
• Operationalising Hadoop
• Real-time and streaming
• Hadoop and the cloud
The First 10 Years

8. © 2016 Attunity
Combining Hadoop and the Cloud
TargetsSources
On Premises
Cloud Platform
HadoopRDBMS
Data Warehouse
Hadoop
RDBMS
Data Warehouse

9. © 2016 Attunity
Universal Data Availability
Results
4500 applications
DB2 MF SQL Oracle
• Consolidating massive data volumes
for global analytics
• Hadoop Data Lake with Kafka
• Minimising labour and cost
• Realising faster insights and
competitive advantage
Fortune 100 automotive maker

10. © 2016 Attunity
The Evolution of Hadoop
• Operationalising Hadoop
• Financial services – enterprise use cases
• Real-time and streaming
• Online retail – customer success
• Hadoop and the cloud
• Global Data Lake

11. It’s all About the Architecture
Lambda or Kappa

12. © 2016 Attunity
Lambda – Batch, Speed, Server
Speed Layer
View1
n 2 1
View2 View3
Batch LayerData Feeds
Server Layer
Query1
n 2
Query2

13. © 2016 Attunity
Kappa –Speed, Server then Batch
Speed Layer
View1
n 2 1
View2 View3
Batch Layer
Data Feeds
Server Layer
Query1
n 2
Query2

14. © 2016 Attunity
Common Examples – Hadoop Vendors
Synchronous Req/
Response Near Real Time Batch
KAFKA
Stream
Search
RDBMS
Apps Monitoring
Real-timeNoSQL
Stream
Process
HADOOP
Impala
DWH
Hive
Spark Map-Reduce

15. How Change Data Capture (CDC) Enables Big and
Fast Data
With Attunity Replicate

16. © 2016 Attunity
CDC Enables the Flow  Kafka
CDC
MSG
n 2 1
MSG MSG
Data Streaming
Transaction
logs
In memory optimised metadata management
and data transport
JSON
data
format
Bulk
Load
MSG
n 2 1
MSG MSG
Data Streaming JSON
data
format
Message
broker
Batch data
Message
broker

17. © 2016 Attunity
Configurable, Pre-defined Automation
• Intuitive and easy to use web-based
interface
• Simple to configure and manage
replication tasks
• Single interface for any supported
source to target
• Easy management with alerts and
notifications
Intuitive User Experience

18. © 2016 Attunity
"table": "table-name",
"schema": "schema-name",
"op": "operation-type",
"ts": "change-timestamp",
"data": [{"col1": "val1"}, {"col2": "val2"}, …., {"colN": "valN"}]
"bu_data": [{"col1": "val1"}, {"col2": "val2"}, …., {"colN":
"valN"}],
Easily Create and Manage Kafka Endpoints
Eliminate manual coding
• Drag and drop interface for all
sources and targets
• Monitor and control data stream
through web console
• Bulk load or CDC
• Multi-topic and multi-partitioned
data publication
Attunity Replicate
Command Line

19. © 2016 Attunity
1. Easy – Configurable, powered by automation (no dev or scripting)
2. Change Data Capture (CDC) – Efficient, low latency and no downtime
3. Zero Footprint – No agent on source database server
4. Heterogeneous – Supports many data sources and targets
5. Optimised – Integrated with native target system APIs
Attunity Replicate’s Differentiators

20. © 2016 Attunity
• Accelerate data delivery across enterprise and cloud
• Empower rapid utilisation of data by the business
• Continually optimise with intelligent insight
Attunity Corporate Overview
Over 2000 Customers in 65 Countries
Financial Services Manufacturing / Industrials GovernmentHealth Care
Technology / Telecommunications Other Industries
Enterprise Data Management
On Premise | Cloud | Across Platforms
Global Organisation
USA
EMEA
APAC

21. © 2016 Attunity
Attunity – Trusted Technology and Partner
 Trusted by Microsoft with 3 OEMs, bundled inside SQL Server
 Trusted by Amazon (AWS) with technology licensing for cloud migration service
 Trusted by IBM and Oracle with respective OEMs of Attunity technology
 Trusted by Teradata as a reseller for Data warehouse and Hadoop market
 Trusted by HP as a reseller for Data warehouse and analytics market
 Trusted by Accenture, Capgemini and Cognizant as SI partners
 Trusted by Hortonworks, Cloudera and MapR for Hadoop solutions
 Trusted by over 2000 customers in over 65 countries

22. © 2016 Attunity
CDC enables the Evolution of Hadoop
• Operationalising Hadoop
• Financial services – enterprise use cases
• Real-time and streaming
• Online Retail – customer success
• Hadoop and the cloud
• Global Data Lake

23. © 2016 Attunity
Contact Us
info-uk@attunity.com
Come see us @booth 411
Follow Us
www.attunity.com/resources
@attunity
http://Linkd.in/attunity
Q&A and Additional Resources
FREE
TRIAL
attunity.com/replicateexpress

24. Thanks!
attunity.com