Hadoop与数据分析

Hadoop 与数据分析淘宝数据平台及产品部基础研发组周敏日期： 2010-05-26

Outline ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

打扑克与 MapReduce Input split shuffle output 分牌各自齐牌交换再次理牌搞定

统计单词数 The weather is good This guy is a good man Today is good Good man is good the 1 weather 1 is 1 good 1 today 1 is 1 good 1 this 1 guy 1 is 1 a 1 good 1 man 1 good 1 man 1 is 1 good 1 a 1 good 1 good 1 good 1 good 1 good 1 man 1 man 1 the 1 weather 1 today 1 guy 1 is 1 is 1 is 1 is 1 this 1 a 1 good 5 guy 1 is 4 man 2 the 1 this 1 today 1 weather 1

趋势分析 http://www.trendingtopics.org/ 截图

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Hadoop 生态系统

Hadoop 实现 Data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Data data data data data Results Data data data data Data data data data Data data data data Data data data data Data data data data Data data data data Data data data data Data data data data Data data data data Hadoop Cluster DFS Block 1 DFS Block 1 DFS Block 2 DFS Block 2 DFS Block 2 DFS Block 1 DFS Block 3 DFS Block 3 DFS Block 3 MAP MAP MAP Reduce

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Hadoop 案例 (1)

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Hadoop 案例 (2)

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Hadoop 案例 (3)

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Hadoop 案例 (4)

B A D A A C B C B C D Group Co-group Function Aggregate Filter Filter Thinking in MapReduce(1)

[object Object],[object Object],Hive 的魔力

为什么淘宝采用 Hadoop? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

还有谁在用 Hadoop? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Web Servers Log Collection Servers Filers Data Warehousing on a Cluster Oracle RAC Federated MySQL Web 站点的典型 Hadoop 架构

淘宝 Hadoop 与 Hive 的使用 Hadoop Rich Client MetaStore Server Mysql Scheduler Thrift Server Web JobClient CLI/GUI Client Program Web Server

[object Object],[object Object],[object Object],[object Object],[object Object],调试

[object Object],[object Object],[object Object],[object Object],[object Object],Profiling

[object Object],[object Object],监控

Hadoop与数据分析

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (7)

Semelhante a Hadoop与数据分析

Semelhante a Hadoop与数据分析 (20)

Mais de George Ang

Mais de George Ang (20)

Hadoop与数据分析