Overview 1. The main goals of performance testing 2. The Advantages of Performance Tests 3. The Disadvantages of Performance Tests 4. Types of performance tests 5. Determining a successful performance testing project Enjoy!

1. PERFORMANCE
TESTING
DAVID TZEMACH
WWW.DTVISIONTECH.COM
JAN 1 2016

2. OVERVIEW
• PERFORMANCE TESTING IS ONE OF THE MOST IMPORTANT TESTING METHODS YOU NEED
TO GET FAMILIAR WITH.
• THE MAIN REASON IS THAT THIS TYPE OF TEST IS ONE OF THE BEST WAYS TO EVALUATE
YOUR SYSTEMS' STABILITY, RELIABILITY, AND EFFECTIVENESS ETC., UNDER A GIVEN
WORKLOAD.
• WHILE EXECUTING PERFORMANCE TESTS, WE MUST VALIDATE THE SYSTEM'S ABILITY TO
DELIVER AN EXPECTED PERFORMANCE BASED ON THE PREDEFINED REQUIREMENTS AND
SPECIFICATIONS.

3. PERFORMANCE TESTS
MAIN GOALS

4. TEST THE APPLICATION STABILITY
• IS THE APPLICATION STABLE TO USE WHEN RUNNING FOR LONG TIME?
• ARE THERE ANY ARCHITECTURES THAT LEADS TO STABILIZATION ISSUES?
• IS THE APPLICATION STABLE TO USE UNDER DIFFERENT LOADS?
• IS THE APPLICATION STABLE ENOUGH FOR THE CLIENT USE?

5. TEST THE APPLICATION SCALABILITY
• WHAT ARE THE ENVIRONMENTS THAT WE CAN USE TO EXECUTE THE SOFTWARE?
• HOW MANY USERS CAN INTERACT WITH THE SOFTWARE AT THE SAME TIME?
• HOW MANY REQUESTS THE SOFTWARE CAN HANDLE ON SPECIFIC TIME?
• IS THE SOFTWARE CAN HANDLE A LARGE DATA GROWTH OVER TIME?

6. TEST THE APPLICATION RESPONSIVENESS
• HOW FAST THE SYSTEM INTERACTS UNDER DIFFERENT ARCHITECTURES?
• HOW FAST THE SYSTEM CAN PROCESS A DIFFERENT TYPE OF DATA?
• HOW FAST THE SYSTEM CAN RETURN THE EXPECTED OUTPUT?
• HOW FAST THE SYSTEM CAN HANDLE USER REQUEST?
• HOW FAST THE SYSTEM WILL START?

7. TEST THE APPLICATION RELIABILITY
• IS THE APPLICATION RELIABLE FOR USE BY THE END USERS?
• ARE THE APPLICATION CRITICAL BUGS REMOVED BEFORE WE DEPLOY THE
SYSTEM ON USER'S ARCHITECTURES?
• IS THE APPLICATION CAN HANDLE SYSTEM FAILURES WITHOUT LOSING
THE CUSTOMER DATA ?

8. PERFORMANCE TESTS
THE ADVANTAGES

9. THE ADVANTAGES (1)
• PERFORMANCE TESTS MAY INVOLVE SOME ADDITIONAL CORRESPONDING TESTING TYPES
THAT LEAD TO GREATER COVERAGE (ENDURANCE, SPICE, AND LOAD ETC.).
• PERFORMANCE TESTS ARE ALSO USED TO IMPROVE THE END USER EXPERIENCE.
• PERFORMANCE TESTS CAN BE USED TO EVALUATE AND COMPARE TWO SYSTEMS AGAINST
EACH OTHER.
• POOR PERFORMANCE RESULTS MAY INDICATE FOR A CONFIGURATION/FUNCTIONAL
DEFECT.

10. THE ADVANTAGES (2)
• PERFORMANCE TESTS CAN EVALUATE MULTIPLE ASPECTS OF THE SOFTWARE
(STABILITY, OVERALL PERFORMANCE SPEED, BOTTLENECKS, EFFECTIVENESS ETC.).
• PERFORMANCE TESTS WILL ALLOW THE TESTING TEAM TO RAISE THE PROBLEMATIC
AREAS THAT ARE AFFECTING THE APPLICATION'S PERFORMANCE.
• PERFORMANCE TESTS ALLOWS THE COMPANY TO UNDERSTAND THE OVERALL LIMITS
OF THE SYSTEM.

11. THE ADVANTAGES (3)
• PERFORMANCE TESTS ARE ONE OF THE BETTER WAYS TO EVALUATE THE
SOFTWARE UNDER DIFFERENT ARCHITECTURES AND OVERLOADS.
• PERFORMANCE TEST ARE USED TO EVALUATE THE PRODUCT AGAINST THE
PREDEFINED REQUIREMENTS AND SPECIFICATIONS.
• PERFORMANCE TESTS, WILL PROVIDE CONFIDENCE IN THE PRODUCT
(BOTH FROM STOCKHOLDERS AND MANAGEMENT).

12. PERFORMANCE TESTS
THE DISADVANTAGES

13. THE DISADVANTAGES (1)
• PERFORMANCE TESTS IS A PROCESS THAT TAKE TIME, VERY LONG TIME.
• IN MOST CASES, PERFORMANCE TESTS ARE EXECUTED WITH 3RD PARTY
APPLICATIONS THAT NEED FURTHER LEARNING AND INVESTIGATION.
• SOMETIMES YOU CANNOT ANTICIPATE THE EXPECTED RESULTS.
• PERFORMANCE TESTS ARE A COSTLY PROCESS.

14. THE DISADVANTAGES (2)
• SOMETIMES IT’S VERY HARD TO UNDERSTAND THE ROOT CAUSE OF A PROBLEM.
• PERFORMANCE TESTS NEED A DEDICATED AND LARGE TESTING ENVIRONMENT.
• PERFORMANCE TESTS NEED TO BE CAREFULLY DESIGNED AND EVALUATED.
• PERFORMANCE TESTS MAY NOT FIND FUNCTIONAL BUGS THAT YOU CAN FIND WITH
OTHER TESTING TYPES.

15. PERFORMANCE AND
CORRESPONDING
TESTING METHODS

16. SPIKE TESTS
TESTING THE SOFTWARE WITH SUDDEN OVERLOADS, THE MAIN GOAL IS
VALIDATE THAT THE SOFTWARE CAN HANDLE SITUATIONS WHERE THERE IS
A SUDDEN JUMP IN THE USER ACTIVITY OR DRAMATIC CHANGES IN LOAD.

17. RELIABILITY / ENDURANCE / SOAK TESTS
TESTING THE SYSTEM WITH DIFFERENT LOAD FOR A LONG PERIOD OF TIME,
THE MAIN GOAL IS TO SEE THAT THE SYSTEM CAN HANDLE SUSTAINED USE
FOR A LONG PERIOD OF TIME.

18. LOAD TESTS
TESTING THE SOFTWARE WITH INCREASING SET OF DATA (NEAR TO THE
SOFTWARE LIMITATION), THE MAIN GOAL IS TO SEE THAT THE SOFTWARE
CAN HANDLE SUCH LOADS WITHOUT HAVING ANY SIDE EFFECTS
(UNEXPECTED CRUSHES, MEMORY LEAKS, BOTTLENECKS ETC.).

19. SCALABILITY TESTS
TESTING THE SOFTWARE PERFORMANCE WHILE INCREASING THE VOLUME
OF USERS/ENVIRONMENT AND DETERMINE HOW THE SOFTWARE WILL
ADJUST TO THE GROWING SCALE.

20. STRESS TESTS
TESTING THE SOFTWARE UNDER LOAD THAT EXCEED THE SYSTEM
LIMITATION, THE MAIN GOAL IS TO CAUSE SYSTEM FAILURES, ANALYZE THE
CRASH, AND DETERMINE THE WAY THAT THE SOFTWARE RECOVER IN CASE
OF SUCH FAILURE.

21. FAIL-OVER/FAULT-TOLERANCE TESTS
TESTING THAT THE SOFTWARE CORRESPONDING TO FAILURES WHILE
INVOLVING TWO OR MORE PHYSICAL NODES, IN THIS TYPE OF TESTS, WE
NEED TO VALIDATE THAT IN CASE OF FAILURE THE USERS CAN CONTINUE
WITH THEIR WORK AND THE APPLICATION DATA/SERVICES ARE MOVED TO
THE AVAILABLE “ACTIVE” NODE.

22. REDUNDANCY TESTS
TESTING THE APPLICATION REDUNDANCY MECHANISM IN CASE OF LOAD,
THE MAIN GOAL IS TO DETERMINE THE EFFECTIVENESS OF THE BALANCING
SYSTEM.

23. VOLUME TESTS
TESTING THE APPLICATION WITH DIFFERENT AMOUNT OF DATA, THE MAIN
GOAL HERE IS TO UNDERSTAND THE LIMITS OF OUR APPLICATION AND
DEMONSTRATE HOW A SPECIFIC AMOUNT OF DATA CAN AFFECT THE
PREDEFINED REQUIREMENTS.

24. DEFINING A SUCCESSFUL
PERFORMANCE TESTING PROJECT

25. YOU SUCCEEDED IN IMPROVING THE CURRENT PERFORMANCE (1)
SO YOU CONDUCT YOUR TESTS, SPEND THOUSANDS OF DOLLARS, BUT THE
APPLICATION BEHAVES THE SAME WAY AS IT WAS PRIOR TO STARTING YOUR TESTS.
IF THAT’S THE RESULT THEN WE FAILED, BUT IF WE SUCCEEDED TO IMPROVE THE
PERFORMANCE OF THE APPLICATION THEN WE CAN SAY THAT WE ACHIEVE ONE OF
THE MAIN GOALS THAT WE START THE TEST IN THE FIRST PLACE

26. YOU SUCCEEDED IN FINDING THE APPLICATION BUGS EARLY (2)
WELL, YOU ALL KNOW THAT BUGS ARE CHEAPER WHEN YOU FIND THEM
ON THE EARLY STAGES OF THE TESTING PROCESS, THIS FACT IS RELEVANT
TO PERFORMANCE TESTS AS IT’S RELEVANT TO ANY OTHER TESTING
PROCESS.

27. YOU CAN DELIVER DOCUMENTATION THAT REFLECTS THE PROCESS (3)
LIKE ANY OTHER TESTING PROJECT, WE ALWAYS NEED TO ASK OURSELVES A SIMPLE QUESTION “DID
WE SPEND THE TIME WE HAD IN AN EFFICIENT WAY..?”
WELL, TO ANSWER THIS QUESTION WE NEED TO EXAMINE HOW IT’S ACTUALLY SPENT DURING THE
PROJECT, THE BEST WAY TO ACCOMPLISH THIS TASK IS TO EXAMINE THE DOCUMENTATION THAT WE
CREATE AND USED DURING THE PROCESS.
• EXAMPLES:
• THE TEST STRATEGY THAT WE USED.
• STP.
• STD.
• STR.
• ETC.

28. YOU KNOW THE NUMBERS INSTEAD OF ASSUMPTIONS (4)
A TRUE SUCCESS MUST INCLUDE “NUMBERS”, EVERY ASSUMPTION THAT YOU HAVE PRIOR TO THE
TESTS SHOULD BE TRANSLATED INTO NUMBERS, THAT YOU CAN ANALYZE AND EXAMINE IN DIFFERENT
PHASES OF THE PROJECT.

29. YOU CREATE A BASELINE FOR FUTURE PROJECT (5)
PERFORMANCE TESTS WILL CONSUME A HUGE AMOUNT OF RESOURCES, TIME AND
MONEY, A GREAT SUCCESS FACTOR IS ACHIEVED IF YOU CAN USE THE CURRENT TESTING
ON FUTURE PROJECTS, IT’S LIKE “RECYCLING”, THINK ABOUT THIS FOR A SEC, EACH
PERFORMANCE TESTS WILL LEAD TO FURTHER COSTS ON HARDWARE, SOFTWARE AND
TESTING TOOLS. IF YOU SUCCEED TO MAINTAIN THIS OBJECTS, YOU CAN USE THEM ON
FUTURE PROJECTS AND REDUCE THE COSTS.

30. FOR ADDITIONAL KB’S PLEASE
VISIT MY BLOG
WWW.DTVISIONTECH.COM