A short introduction to Apache Bigtop, what is it and what can it be used for ? How can it be used to test and deploy the Hadoop stack ?

1. Apache Bigtop
●
What is it ?
●
What is it for ?
●
Automated Testing
●
Platforms
●
Dependencies
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

2. Apache Bigtop – What is it ?
●
For testing Hadoop and related tools
●
For test development
●
Uses Jenkins continuous integration
●
Based on Cloudera's CDH
●
BigTop for Hadoop is equivalent to a Linux release
–
i.e. Ubuntu, Fedora, CentOS
●
Use it for deploying the Hadoop stack
●
Guarantees version inter-operability within the stack
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

3. Apache Bigtop – What is it for ?
It is for testing Hadoop at a number of levels
–
Packaging
–
Platform
–
Runtime
–
Upgrades
–
Integration testing
–
Validation !
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

4. Automation Methodology – An Example
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

5. Apache Bigtop – Platforms
Platforms supported
–
Ubuntu 10.10
–
CentOS 5/6
–
Fedora 18
–
Mageia 1
–
openSUSE 12.2
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

6. Apache Bigtop – Dependencies
The following tools are required
Java JDK 1.6
– Apache Maven
– tar
– subversion
– gcc-c++
– fuse
– autoconf
– libtool
– asciidoc
–
www.semtech-solutions.co.nz
Apache Ant
wget
git
gcc
make
protobuf-compiler
automake
sharutils
xmlto
info@semtech-solutions.co.nz

7. Apache Bigtop – The Players
Who has bought into the use of Bigtop ?
–
Canonical
–
Cloudera
–
Ebay
–
EMC
–
Hortonworks
–
Illumos
–
Intel
–
Trendmicro
–
WANdisco
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

8. Light Humour
A bit of humour ........
www.semtech-solutions.co.nz
info@semtech-solutions.co.nz

9. Contact Us
●
Feel free to contact us at
–
www.semtech-solutions.co.nz
–
info@semtech-solutions.co.nz
●
We offer IT project consultancy
●
We are happy to hear about your problems
●
You can just pay for those hours that you need
●
To solve your problems

10. Contact Us
●
Feel free to contact us at
–
www.semtech-solutions.co.nz
–
info@semtech-solutions.co.nz
●
We offer IT project consultancy
●
We are happy to hear about your problems
●
You can just pay for those hours that you need
●
To solve your problems