Hadoop Map Reduce

Hadoop Map Reduce Apurva Jadhav Senior Software Engineer TheFind Inc. (diagrams borrowed from various sources)

Introduction ,[object Object],[object Object],[object Object],[object Object],[object Object]

Hadoop Core ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Typical cluster ,[object Object],[object Object]

Hadoop Distributed File System ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

HDFS Architecture Namenode (Master) Datanode Client Datanode Datanode Read Write Replication Meta data ops Stores FS metadata – namespace, block locations Stores the data blocks as linux files

Hadoop Map Reduce ,[object Object],[object Object],[object Object],[object Object],[object Object]

Word Count Problem ,[object Object],[object Object],[object Object],[object Object],[object Object]

Why Map Reduce ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Map Reduce Architecture ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Jobtracker tasktracker tasktracker tasktracker Input Job (mapper, reducer, input) Data transfer Assign tasks

Map Reduce Programming Model ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Map Reduce Programming Model ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Word Count Map Reduce Job ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Map Reduce Architecture - 2 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Map Reduce Architecture - 2 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Jobtracker tasktracker tasktracker tasktracker Input Job (mapper, reducer, input) Data transfer Assign tasks

Map Reduce Advantages ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Conclusion ,[object Object],[object Object],[object Object],[object Object]

References ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Hadoop Map Reduce

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Hadoop Map Reduce

Semelhante a Hadoop Map Reduce (20)

Mais de VNIT-ACM Student Chapter

Mais de VNIT-ACM Student Chapter (12)

Último

Último (20)

Hadoop Map Reduce