The document discusses Treasure Data, a big data analytics service that allows customers to analyze large volumes of data in the cloud without needing specialist resources. Some key points: - Treasure Data was founded to deliver cost-effective big data analytics without months of setup or expensive infrastructure. - It offers a subscription-based cloud service that can store over 100 billion records and process thousands of messages per second. - Major customers include Fortune 500 companies processing tens of billions of records. - The service integrates with Heroku and allows developers to easily collect, store and query data to power applications. - It aims to simplify big data analytics in the cloud compared to on-premise Hadoop deploy

1. Treasure Data:
Big Data Analytics on Heroku
Muga Nishizawa, Chief Software Architect

2. Muga Nishizawa (@muga_nishizawa)
Chief Software Architect, Treasure Data

3. Treasure Data Overview
 Founded to deliver big data analytics in days not months without
specialist IT resources for one-tenth the cost of other alternatives
 Service based subscription business model
 World class open source team
• Founded world’s largest Hadoop User Group
• Developed Fluentd and MessagePack
• Contributed to Memcached, Hibernate, etc.
 Treasure Data is in production
• 20 customers incl. Fortune 500 companies
• 100+ billion records stored
 Processing 10,000 messages per second
3

4. Our Customers – Fortune Global 500
leaders and start-ups including:
4

5. One Hundred Billion Records and
Growing!
120
100
80
60
40
20
Sep Nov Jan Mar May Jul Aug
2011 2011 2012 2012 2012 2012 2012
5

6. Treasure Data Service
“Store Your Data Now for Future Insights”
6

7. Treasure Data Service
“Store Your Data Now for Future Insights”
User
Apache
App
Treasure Data
RDBMS columnar data storage
App
Other data sources
MAPREDUCE JOBS
HIVE, PIG (to be supported)
td-command
Query
Query
Processing
API
JDBC, REST Cluster
User BI apps
7

8. Treasure Data Service
“Store Your Data Now for Future Insights”
User
Apache
2012-02-04 01:33:51
App
myappdb.buylog { Data
Treasure
RDBMS columnar data storage
App
“user”: ”12345”,
Other data sources
“path”: “/buyItem”,
“price”: 150,
MAPREDUCE JOBS
td-command
HIVE, PIG (to be supported)
“referer”: “/landing”
Query
} Query
Processing
API
JDBC, REST Cluster
User BI apps
8

9. Treasure Data Service
“Store Your Data Now for Future Insights”
User
Apache
$ td query -w -d myappdb
App "SELECT Treasure Data
App RDBMS TD_TIME_FORMAT(time, data storage
columnar "yyyy-MM-dd", "PDT"
COUNT(1) AS cnt
Other data sources FROM buylog
GROUP BY TD_TIME_FORMAT(time, "yyyy-MM-dd"
MAPREDUCE JOBS
ORDER BY cnt"
HIVE, PIG (to be supported)
td-command
Query
Query
Processing
API
JDBC, REST Cluster
User BI apps
9

10. Treasure Data Service
“Store Your Data Now for Future Insights”
User
Apache
App
Treasure Data
RDBMS columnar data storage
App
Other data sources
+------------+------+
MAPREDUCE JOBS
| day | cnt |
HIVE, PIG (to+------------+------+
be supported)
td-command
| 2012-05-26 | 4981 | Query
Query
Processing
JDBC, REST| 2012-05-27 | 4481 |
API
Cluster
User BI apps | 2012-05-28 | 481 |
+------------+------+
10

11. Comparing On-Premise & Cloud Big Data Mkts
Cloud
Database- Big Data-as-a-
as-a- Service
Service
Traditional
DBMS Hadoop
(ODS, Data Mart) Data
Warehouse
On-Premise
Low Data Volume High
© 2012 Forrester Research, Inc. Reproduction Prohibited 11

12. Treasure Data as Heroku Add-on
12

13. Demo with Heroku
13

14. Synergy Effect for Data-Driven
Development!
×
14
1
0

15. The Power of the Cloud
Easier to Scale
Easier to Maintain
Easier to Iterate
15
1
1

16. Implementation Process
Traditional DW and
On-Premise Big Data
16

17. Implementation Process
Traditional DW and Heroku
On-Premise Big Data ×
Treasure Data
Dramatically streamlined
Implementation process
17

18. Viki.com: “Global Hulu”
18
1
4

19. Viki Before
 Hard to manage Hadoop
 Complicated data collection
19

20. Viki After
 No more Hadoop maintenance
 Versatile data collector, td-agent
20

21. Please Try It!
21

22. How Does It Work?
22

23. Query Processing
Query Language
Query Execution
Columnar Data
Object Storage
23

24. 1/4: Compile SQL into MapReduce
SELECT COUNT(DISTINCT ip) FROM tbl;
24

25. 2/4: MapReduce is executed in parallel
SELECT COUNT(DISTINCT ip) FROM tbl;
cc2.8xlarge cluster compute instance (up to 100 nodes * 32 threads)
25

26. 3/4: Columnar Data Access
SELECT COUNT(DISTINCT ip) FROM tbl;
10Gbps Network
Read ONLY the Required Part of Data
26

27. 4/4: Object-based Storage
27

28. Enjoy Data-Driven Development!
28

29. Big Data for the Rest of Us
www.treasure-data.com | @TreasureData

32. Great Investors
 Bill Tai
 Naren Gupta –Nexus Ventures, Director of Red Hat, TIBCO
 Dave Stamm – Clarify, Daisy Systems, Enkata
 Othman Laraki –Twitter
 James Lindembaum, Adam Wiggins and Orion Henry – Heroku
 Anand Babu Periasamy and Hitesh Chellani –Gluster
 Yukihiro “Matz” Matsumoto – Creator of Ruby, now at Heroku
 Dan Schienman – Former Cisco SVP
 Jean-Philippe Emelie Marcos – Tango, D.E. Shaw
 + executives from Cisco, Red Hat, Salesforce.com, GREE
32

33. What are your options?
 Traditional  OnPremise Hadoop
• Never design for analytic
processing
• Too many people
• Too much software from too
many sources
 Cloud Hadoop
 Too much complexity
• Partial solution
 Too long to get live
• Vendor lock-in
 Too expensive to maintain
 Can only innovate at speed of
vendor
33

34. Confidenti
34
al

35. Example Use Case – MySQL to TD
35

36. Example Use Case – MySQL to TD
36