Big Data Testing | Edureka

WHAT IS BIG DATA TESTING
STRATEGIES BEHIND BIG DATA TESTING
www.edureka.co
BIG DATA FORMS
BIG DATA TESTING ENVIRONMENT
TESTING BIG DATA
BIG DATA TESTING TOOLS
WHY DO WE NEED BIG DATA TESTING?
CHALLENGES FACED IN TESTING BIG DATA
BIG DATA TESTING V/S TRADITIONAL TESTING

WHY DO WE NEED BIG DATA TESTING
www.edureka.co

www.edureka.co

www.edureka.co
Big Data Testing can be defined as the procedure that involves
examining and validating the functionality of the Big Data
Applications. Big Data is a collection of a huge amount of data
that traditional storage systems cannot handle.
www.edureka.co

www.edureka.co

www.edureka.co
Testing an Application that handles terabytes of data would
take the skill from a whole new level and out of the box
thinking.

Real-Time Processing
Interactive Processing
Batch Processing
www.edureka.co

Structured
Semi-Structured
Unstructured
BIG DATA FORMS
www.edureka.co

www.edureka.co

The basic requirements that makeup Data Testing are as follows.
1. Space for Storing, Processing and Validating Terra bytes of data should be available.
2. The Cluster and its respective nodes should be responsive
3. Data Processing resources like the powerful CPU should be available
www.edureka.co

BIG DATA TESTING
www.edureka.co

BIG DATA TESTING
www.edureka.co
Data is first loaded from source to Big Data System using extracting tools. The Storage might be
HDFS, MongoDB or any similar storage. Then, the loaded data is cross-checked for errors and
missing values.
Data Ingestion

BIG DATA TESTING
www.edureka.co
In this stage, the key-value pairs for the data get generated. Later,
the MapReduce logic is applied to all the nodes and checked if the
algorithm works fine or not. A data validation process takes place
here to make sure the output is generated as expected.
Data Processing
www.edureka.co

BIG DATA TESTING
www.edureka.co
At this stage, the output generated is ready to be migrated to the
data warehouse. Here, The transformation logic is checked, the data
integrity is verified and the key-value pairs at the location are
validated for accuracy.
Data Validation
www.edureka.co

UNIT TESTING
The complete Big Data Application is divided into segments and each
segment is rigorously tested with multiple possibilities for an expected
outcome. If the segment fails, then it is sent back to the developments
and improvements.
www.edureka.co

FUNCTIONAL TESTING
Functional Testing can be otherwise called as the different
phases in testing the big data application. The Big Data
Application is designed to deal with huge blocks of data.
www.edureka.co

Data Storage Phase
Data Processing Phase
Data Ingestion Phase
Data Integrity Phase
Data Validation Phase
Report Generation Phase
www.edureka.co

NON-FUNCTIONAL TESTING
The Non-Functional Testing phase takes care of the three major
dimensions and characteristics of Big Data. The Volume, Velocity,
and finally the Variety of the Big Data.
www.edureka.co

1
3
5
2
4
Data Quality
Monitoring
Infrastructure
Data Security
Data
Performance
Fail-over Test
Mechanism
www.edureka.co

PERFORMANCE TESTING
Performance testing highly concentrates on the performance
delivered by all the components of the big data system.
www.edureka.co

Data Collection Phase
Data Ingestion Phase
Data Processing
Component Testing
www.edureka.co

PERFORMANCE TESTING APPROACH
www.edureka.co

PARAMETERS OF PERFORMANCE TESTING
Concurrency
Caching
Commit
logs
Timeout
Data
Storage
Performance
Testing
Parameters
www.edureka.co

ARCHITECTURE TESTING
Architecture testing concentrates on establishing a stable
Hadoop Architecture. The architecture of Big Data Processing
Application plays a key role in achieving smooth operations.
www.edureka.co

ARCHITECTURE TESTING
1
2
3
4
Performance Degradation
Node Failure
High Data Latency
High Maintainance
www.edureka.co

www.edureka.co

Data Processing
Data Storage
Data Ingestion
Data Migration
www.edureka.co

Data Ingestion Data Processing Data Storage Data Migration
www.edureka.co

KEY CHALLENGES FACED IN TESTING BIG DATA
www.edureka.co

KEY CHALLENGES FACED IN TESTING BIGDATA
• Big Data Testing is highly complicated, the process requires a highly
skilled official.
• Automated Big Data Testing procedures are and not suited for
unexpected errors.
• Virtual Machine latency creates latency in tests and managing
multimedia is a hazzle
• The volume of the data is one major challenge for testing
• Test environment and automation should be developed for different
platforms
• Each component is from a different technology, hence, requires isolated
testing
• No single tool can perform end to end testing
• High Degree of scripting is required for designing test cases
• Customized Solutions are required to increase performance and test
critical areas
www.edureka.co

BIG DATA TESTING v/s TRADITIONALTESTING
www.edureka.co

Big Data Testing Supports all types of data testing
Traditional testing supports only structured data testing
BIG DATA TESTING v/s TRADITIONAL DATA TESTING
www.edureka.co

Big Data Testing requires Research and Development
Traditional Data Testing doesn’t need Research and Development
www.edureka.co

Data Size is Unlimited in Big Data Testing
Data Size is limited in Big Data Testing
www.edureka.co

Big Data Testing requires special Environment
Traditional Data Testing doesn’t require special Environment
www.edureka.co

Only the highly skilled and qualified candidates are preferred
Basic operations knowledge is enough to run tests
www.edureka.co

Big Data Testing | Edureka

Recomendados

Recomendados

Mais conteúdo relacionado

Mais de Edureka!

Mais de Edureka! (20)

Último

Último (20)

Big Data Testing | Edureka