Yahoo! Hadoop User Group - May Meetup - HBase and Pig: The Hadoop ecosystem at Twitter, Dmitriy Ryaboy, Twitter

Hadoop, Pig, HBase at Twitter ,[object Object],[object Object],[object Object]

Who is this guy, anyway ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

In This Talk ,[object Object],[object Object],[object Object],[object Object],[object Object]

Not In This Talk ,[object Object],[object Object],[object Object],[object Object],[object Object]

Daily workload ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Twitter data pipeline (simplified) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Logs ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Tables ,[object Object],[object Object],[object Object],[object Object]

ETL ,[object Object],[object Object],[object Object]

Mutability ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Aren't you guys Cassandra poster boys? poster boys? ,[object Object],[object Object],[object Object],[object Object],[object Object]

HBase schema for MySQL exports, v1. ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

HBase schema v1, cont. ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

HBase schema, v2. ,[object Object],[object Object],[object Object],[object Object],[object Object]

Why Pig? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

HBase Loader enhancements ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

HBase Loader TODOs ,[object Object],[object Object],[object Object],[object Object],[object Object]

Elephant Bird ,[object Object],[object Object],[object Object],[object Object],[object Object]

Bad records kill jobs ,[object Object],[object Object],[object Object],[object Object]

Runaway UDFs kill jobs ,[object Object],[object Object],[object Object],[object Object],[object Object]

Use Counters ,[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],Lazy deserializaton FTW lazy deserialization

Also see ,[object Object],[object Object],[object Object],[object Object],[object Object]

Questions ? Follow me at twitter.com/squarecog TM

Photo Credits ,[object Object],[object Object],[object Object]

Yahoo! Hadoop User Group - May Meetup - HBase and Pig: The Hadoop ecosystem at Twitter, Dmitriy Ryaboy, Twitter

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (19)

Semelhante a Yahoo! Hadoop User Group - May Meetup - HBase and Pig: The Hadoop ecosystem at Twitter, Dmitriy Ryaboy, Twitter

Semelhante a Yahoo! Hadoop User Group - May Meetup - HBase and Pig: The Hadoop ecosystem at Twitter, Dmitriy Ryaboy, Twitter (20)

Mais de Hadoop User Group

Mais de Hadoop User Group (13)

Último

Último (20)

Yahoo! Hadoop User Group - May Meetup - HBase and Pig: The Hadoop ecosystem at Twitter, Dmitriy Ryaboy, Twitter