This document provides an overview of the Hadoop framework. It describes the key components of Hadoop including the NameNode, DataNodes, JobTracker, TaskTracker, and SecondaryNameNode. The NameNode manages file metadata and location information stored across DataNodes. The JobTracker schedules and tracks jobs on TaskTrackers running on slave nodes. The SecondaryNameNode helps recover metadata if the NameNode fails. Hadoop uses a master/slave architecture with the NameNode and JobTracker on the master and DataNodes and TaskTrackers on slave nodes.

1. Hadoop
Edit by Cassell Hsu
2013.04.19

2. Hadoop
Master
Slave
Slave
Slave

3. Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode

4. NameNode
• Where is NameNode?
• Master
• HDFS(Hadoop Distributed File System)
• What is NameNode?
• 資料之位置資訊
？？
• 資料之屬性

5. NameNode
• 位置資訊?
• 所有資料皆存放在 – DataNode

6. DataNode
• What is DataNode
• 存放資料
• Where is DataNode
• HDFS
• Slaves (and Master)

7. UUsseerr
DDaattaaNNooddee
NNaammeeNNooddee
DDaattaaNNooddee
DDaattaaNNooddee
A
64Mb
128
Mb B
64Mb
A
64Mb
B
64Mb
Check hdfs-site.xml

8. Hadoop
Master
NNaammeeNNooddee DDaattaaNNooddee
Slaves
DDaattaaNNooddee
DDaattaaNNooddee
DDaattaaNNooddee

9. Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode

10. JobTracker
• What is JobTracker?
• 排程工作
• Where is JobTracker?
• Master
誰來工作？

11. JobTracker & TaskTracker
JobTracker TaskTracker
Where Master Slaves
What 排程工作執行工作

12. Hadoop
Master
NNaammeeNNooddee DDaattaaNNooddee
Slaves
DDaattaaNNooddee
JJoobbTTrraacckkeerr
TTaasskkTTrraacckkeerr

13. Hadoop
• Master
• NameNode
• JobTracker
• SecondaryNameNode

14. SecondaryNameNode
• What is SecondaryNameNode?
• NameNode發生錯誤時補救
• Where is SecondaryNameNode?
• Master

15. Hadoop
Master
NNaammeeNNooddee DDaattaaNNooddee
Slaves
DDaattaaNNooddee
JJoobbTTrraacckkeerr
TTaasskkTTrraacckkeerr
SSeeccoonnddaarryyNNooddee

16. MapReduce
HDFS Final Result
User Master
Reduce
Slave1
Slave2
A
A1
A2
Task
NameNode
Task
Result1
Result2

17. MapReduce
• 檔案切割
• Hadoop上區塊切割
• 程式指定