O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
1© Cloudera, Inc. All rights reserved.
Extreme Sports & Beyond:
Exploring a new frontier in data with GoPro
2© Cloudera, Inc. All rights reserved.
Josh is the Manager of Data Architecture and Operations at
GoPro working within the...
3© Cloudera, Inc. All rights reserved.
Let’s do this!
4© Cloudera, Inc. All rights reserved.
Our Story
When we got here…
5© Cloudera, Inc. All rights reserved.
Growing data needs
6© Cloudera, Inc. All rights reserved.
Dev Ops
• Infrastructure
• Hadoop Admin
Engineering
• Data pipeline
• ETL processin...
7© Cloudera, Inc. All rights reserved.
Origin Story
•Make Friends
•Haul Ass
•Maintain Balance
•No Half-Assery
•Integrity. ...
8© Cloudera, Inc. All rights reserved.
GoPro Desktop Application
9© Cloudera, Inc. All rights reserved.
GoPro Desktop Application on Tableau
10© Cloudera, Inc. All rights reserved.
How the Magic Happens
The Philosopher’s Stone…
…or TPS for short
11© Cloudera, Inc. All rights reserved.
High Level Architecture
ETL Cluster
•File dumps
•MR
•Hive
Secure Data
Mart
•End Us...
12© Cloudera, Inc. All rights reserved.
Data Pipeline
Ingest Cluster
ELBHTTP
Pipeline for processing of streaming logs
To ...
13© Cloudera, Inc. All rights reserved.
Data Pipeline
/path1/…
/path2/…
/path3/…
ToETLCluster
/path4/…
14© Cloudera, Inc. All rights reserved.
Data Pipeline
ETL Cluster
HDFS
HIVE Metastore
To SDM Cluster
From Ingest Cluster
I...
15© Cloudera, Inc. All rights reserved.
Data Delivery!
HDFS
HIVE Metastore
Applications
Thrift
ODBC
Server
User
Studio
Stu...
16© Cloudera, Inc. All rights reserved.
Trifacta
17© Cloudera, Inc. All rights reserved.
GoPro Desktop Application on Tableau
18© Cloudera, Inc. All rights reserved.
PG #
RC Playbook: Your guide to
success at GoPro
Questions?
19© Cloudera, Inc. All rights reserved.
Learn more about Cloudera, Tableau, & Trifacta
http://www.cloudera.com/partners/so...
20© Cloudera, Inc. All rights reserved.
Thank You
Próximos SlideShares
Carregando em…5
×

Extreme Sports & Beyond: Exploring a new frontier in data with GoPro

757 visualizações

Publicada em

GoPro is a powerful global brand, thanks in large part to its innovative cameras and accessories that capture moments other cameras just miss: surfing in Maui, skiing in Tahoe, recording your child’s first steps. And today, the company is nearly as well known for its user-generated social and content networks.

Join us for this special webinar hosted by Tableau, Trifacta, and Cloudera—featuring GoPro. We’ll dive into GoPro’s data strategy and architecture, from ingest and processing to data prep and reporting, all on AWS.

Publicada em: Software
  • Seja o primeiro a comentar

Extreme Sports & Beyond: Exploring a new frontier in data with GoPro

  1. 1. 1© Cloudera, Inc. All rights reserved. Extreme Sports & Beyond: Exploring a new frontier in data with GoPro
  2. 2. 2© Cloudera, Inc. All rights reserved. Josh is the Manager of Data Architecture and Operations at GoPro working within the Data Science & Engineering team. Prior to GoPro, Josh led global supply chain operations analytics efforts at Apple. He has 15 years of project, analytics, operations, and business intelligence experience in a variety of industries. He holds a BS in Industrial Engineering and an MBA from the University of Texas at Austin. David is a Principal Engineer in the Data Science and Engineering team at GoPro and the designer of their Spark- Kafka data ingestion pipeline. David has been developing scalable data processing pipelines and eCommerce systems for over 20 years in Silicon Valley. David’s current big data interests include streaming data as fast as possible from devices to near real-time dashboards and switching his primary programming language to Scala from Java after 17 years. He holds a BS in Computer Science from The Ohio State University. Our Speakers Josh Byrd Manager, Data Architecture & Operations, GoPro David Winters Principal Engineer, Data Science & Engineering, GoPro
  3. 3. 3© Cloudera, Inc. All rights reserved. Let’s do this!
  4. 4. 4© Cloudera, Inc. All rights reserved. Our Story When we got here…
  5. 5. 5© Cloudera, Inc. All rights reserved. Growing data needs
  6. 6. 6© Cloudera, Inc. All rights reserved. Dev Ops • Infrastructure • Hadoop Admin Engineering • Data pipeline • ETL processing Architecture • Design • Applications Project Management • Agile Data Science and Engineering We build the platform
  7. 7. 7© Cloudera, Inc. All rights reserved. Origin Story •Make Friends •Haul Ass •Maintain Balance •No Half-Assery •Integrity. Always •Be a HERO Yes, this comes from the top…
  8. 8. 8© Cloudera, Inc. All rights reserved. GoPro Desktop Application
  9. 9. 9© Cloudera, Inc. All rights reserved. GoPro Desktop Application on Tableau
  10. 10. 10© Cloudera, Inc. All rights reserved. How the Magic Happens The Philosopher’s Stone… …or TPS for short
  11. 11. 11© Cloudera, Inc. All rights reserved. High Level Architecture ETL Cluster •File dumps •MR •Hive Secure Data Mart •End User Query •Impala / Sentry •Parquet Analytics Apps •HUE •Tableau •Trifacta •R Ingest Cluster •Log file streaming •Kafka •Spark Induction Framework •Batch files •Pre-processing •Java Original Cluster
  12. 12. 12© Cloudera, Inc. All rights reserved. Data Pipeline Ingest Cluster ELBHTTP Pipeline for processing of streaming logs To ETL Cluster
  13. 13. 13© Cloudera, Inc. All rights reserved. Data Pipeline /path1/… /path2/… /path3/… ToETLCluster /path4/…
  14. 14. 14© Cloudera, Inc. All rights reserved. Data Pipeline ETL Cluster HDFS HIVE Metastore To SDM Cluster From Ingest Cluster Induction framework
  15. 15. 15© Cloudera, Inc. All rights reserved. Data Delivery! HDFS HIVE Metastore Applications Thrift ODBC Server User Studio Studio - Staging GDA Report SDM From ETL Cluster
  16. 16. 16© Cloudera, Inc. All rights reserved. Trifacta
  17. 17. 17© Cloudera, Inc. All rights reserved. GoPro Desktop Application on Tableau
  18. 18. 18© Cloudera, Inc. All rights reserved. PG # RC Playbook: Your guide to success at GoPro Questions?
  19. 19. 19© Cloudera, Inc. All rights reserved. Learn more about Cloudera, Tableau, & Trifacta http://www.cloudera.com/partners/solutions/trifacta.html http://www.cloudera.com/partners/solutions/tableau.html
  20. 20. 20© Cloudera, Inc. All rights reserved. Thank You

×