An Introduction to Apache Hama

•Transferir como ODP, PDF•

0 gostou•1,256 visualizações

Mike Frampton

An Introduction to Apache Hama, what is it ? Why is it useful for scientific computations on big data ?

Tecnologia

Apache Hama
● What is it ?
● What is it used for ?
● How does it work ?
● Architecture
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Hama – What is it ?
● A Distributed computing framework
● For massive scientific calculations
● For distributed computing
● Uses Bulk Synchronous Parallel Computing ( BSP )
● Written in Java
● Licensed via Apache
● Available for Hadoop greater than 1.0.x
● Requires Java 1.6.x or higher
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Hama – What is it used for ?
● large scale computations that
– exceed the abilities of Map reduce
● Machine learning
● Matrix
● Graph
● Network Algorithms
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Hama – How does it work ?
● A series of super steps
● Each of which is
– Concurent computation on local data
– Point to point communication
– Synchronise all processors
● Uses Hadoop RPC for communication
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Hama – Architecture
Hama consists of BSP Master, GroomServers, ZooKeeper
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Hama – Architecture
● Role of BSPMaster
– Maintain groom server statuses
– Controlling cluster super steps
– Maintaining job progress information
– Scheduling Jobs
– Assign tasks to groom servers
– Fault control
– Provide cluster control to users
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Hama – Architecture
● Role of GroomServer
– Performs BSP tasks
– Reports status to master
– Runs on a single HDFS data node
– Starts with a BSP Peer instance
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Hama – Architecture
● Role of ZooKeeper
– Manages barrier synchronisation of BSP Peers
www.semtech-solutions.co.nz info@semtech-solutions.co.nz

Contact Us
● Feel free to contact us at
– www.semtech-solutions.co.nz
– info@semtech-solutions.co.nz
● We offer IT project consultancy
● We are happy to hear about your problems
● You can just pay for those hours that you need
● To solve your problems

Mais conteúdo relacionado

Mais de Mike Frampton

Apache GobblinMike Frampton

Apache Singa AIMike Frampton

Apache RangerMike Frampton

OrientDBMike Frampton

PrometheusMike Frampton

Apache TephraMike Frampton

Apache KuduMike Frampton

Apache BahirMike Frampton

Apache ArrowMike Frampton

JanusGraph DBMike Frampton

Apache IgniteMike Frampton

Apache SamzaMike Frampton

Apache FlinkMike Frampton

Apache EdgentMike Frampton

Apache CouchDBMike Frampton

An introduction to Apache MesosMike Frampton

An introduction to PentahoMike Frampton

An introduction to Apache ThriftMike Frampton

An introduction to Apache CassandraMike Frampton

An example Hadoop InstallMike Frampton

Mais de Mike Frampton (20)

Apache Gobblin

Apache Singa AI

Apache Ranger

OrientDB

Prometheus

Apache Tephra

Apache Kudu

Apache Bahir

Apache Arrow

JanusGraph DB

Apache Ignite

Apache Samza

Apache Flink

Apache Edgent

Apache CouchDB

An introduction to Apache Mesos

An introduction to Pentaho

An introduction to Apache Thrift

An introduction to Apache Cassandra

An example Hadoop Install

Último

CNv6 Instructor Chapter 6 Quality of Servicegiselly40

Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j

Finology Group – Insurtech Innovation Award 2024The Digital Insurer

Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC

SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren

Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent

08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls

Boost PC performance: How more available memory can improve productivityPrincipled Technologies

Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited

Understanding the Laravel MVC ArchitecturePixlogix Infotech

08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge

GenCyber Cyber Security Day PresentationMichael W. Hawkins

Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK

Histor y of HAM Radio presentation slidevu2urc

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia

Slack Application Development 101 Slidespraypatel2

Scaling API-first – The story of a global engineering organizationRadu Cotescu

Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko

How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes

An Introduction to Apache Hama

1. Apache Hama ● What is it ? ● What is it used for ? ● How does it work ? ● Architecture www.semtech-solutions.co.nz info@semtech-solutions.co.nz

2. Hama – What is it ? ● A Distributed computing framework ● For massive scientific calculations ● For distributed computing ● Uses Bulk Synchronous Parallel Computing ( BSP ) ● Written in Java ● Licensed via Apache ● Available for Hadoop greater than 1.0.x ● Requires Java 1.6.x or higher www.semtech-solutions.co.nz info@semtech-solutions.co.nz

3. Hama – What is it used for ? ● large scale computations that – exceed the abilities of Map reduce ● Machine learning ● Matrix ● Graph ● Network Algorithms www.semtech-solutions.co.nz info@semtech-solutions.co.nz

4. Hama – How does it work ? ● A series of super steps ● Each of which is – Concurent computation on local data – Point to point communication – Synchronise all processors ● Uses Hadoop RPC for communication www.semtech-solutions.co.nz info@semtech-solutions.co.nz

5. Hama – Architecture Hama consists of BSP Master, GroomServers, ZooKeeper www.semtech-solutions.co.nz info@semtech-solutions.co.nz

6. Hama – Architecture ● Role of BSPMaster – Maintain groom server statuses – Controlling cluster super steps – Maintaining job progress information – Scheduling Jobs – Assign tasks to groom servers – Fault control – Provide cluster control to users www.semtech-solutions.co.nz info@semtech-solutions.co.nz

7. Hama – Architecture ● Role of GroomServer – Performs BSP tasks – Reports status to master – Runs on a single HDFS data node – Starts with a BSP Peer instance www.semtech-solutions.co.nz info@semtech-solutions.co.nz

8. Hama – Architecture ● Role of ZooKeeper – Manages barrier synchronisation of BSP Peers www.semtech-solutions.co.nz info@semtech-solutions.co.nz

9. Contact Us ● Feel free to contact us at – www.semtech-solutions.co.nz – info@semtech-solutions.co.nz ● We offer IT project consultancy ● We are happy to hear about your problems ● You can just pay for those hours that you need ● To solve your problems

10. Contact Us ● Feel free to contact us at – www.semtech-solutions.co.nz – info@semtech-solutions.co.nz ● We offer IT project consultancy ● We are happy to hear about your problems ● You can just pay for those hours that you need ● To solve your problems

An Introduction to Apache Hama

Recomendados

Recomendados

Mais conteúdo relacionado

Mais de Mike Frampton

Mais de Mike Frampton (20)

Último

Último (20)

An Introduction to Apache Hama