Testing Big Data

1. TESTING BIG DATA
Adoniram Mishra
Benazeer Khan

2. Agenda
❖ What is Big Data?
❖ Hadoop Ecosystem
❖ Testing Challenges
❖ Test Pyramid

3. What is Big Data?

5. What is Hadoop

6. Hadoop Ecosystem

7. HDFS

8. MapReduce

9. High Level Example
Data
Source
HBase HiveData
Source
Data
Source
I
M
P
O
R
T HQL
UI
Reports
Other
applications
M
R
M
R
M
R

10. Testing Challenges

11. Long Feedback Cycle

12. Data

13. Deadlines

14. Automation

15. Countering Challenges
❖ Sampling.
❖ Basic testing concepts - Boundary Value testing, Equivalence class
partitioning.
❖ Automation

16. Nightly Build
Tests
Hive Query
Validator
Unit Tests
UI Test
Test Pyramid

17. Hive Query Validator
Hive Query Validator
Query 1
Query 2
Query 3
Correct
Correct
Incorrect

18. Nightly tests & Smoke Tests
SMOKE
MODULE 3MODULE 1
MODULE 2
MODULE 4

20. About Us
● Adoniram Mishra (adoniram@thoughtworks.com)
● Benazeer Khan (benak@thoughtworks.com)
Follow us at:
Facebook: https://www.facebook.com/groups/vodqa/
Twitter: https://twitter.com/vodqancr