PowerPoint presentation by 3Pillar's Ciprian Balea, QA Lead, which was delivered at the Romanian Testing Conference (RTC) 2014 in Cluj-Napoca, Romania on May 15, 2014.

1. Automated Performance Testing for Desktop
Applications by Ciprian Balea

2. 2
THE SCARY QUESTION OF APPLICATION
DEVELOPMENT
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
How does it perform in real conditions?
Never as first expected!

3. 3
BENCHMARKING
• What impact does the app have on the system running it? (is the system
still reliable)
• How does the app perform when running on the host system? (does it
keep all functionality)
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Ideally,
• consuming as little resources as possible without loosing functionality
• the app should not break /slow the system down

4. 4
HOW TO PREDICT THE FUTURE?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Run it multiple times and average out.
A single run doesn’t accurately predict real life usage.

5. 5
HOW TO GET A CREDIBLE RESULT?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Avoid inconsistencies induced by the outside world
=> no Internet connection, own webserver & assets

6. 6
WHAT’S OUR APPLICATION ABOUT?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
A Custom Web Browser

7. 7
HARDWARE SETUP
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

8. 8
BIG PROBLEM #1: HOW TO KNOW HOW
MUCH TIME THE PAGE TOOK TO LOAD
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Based on the HTML content, loading time cannot be computed
(element loading order is not consistent across multiple runs).
Small solution:

9. 9
BIG PROBLEM #2: HOW TO READ
SYSTEM RESOURCE USAGE WITHOUT
3RD PARTY TOOLS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Small solution (a):

10. 10
BIG PROBLEM #2: HOW TO READ
SYSTEM RESOURCE USAGE WITHOUT
3RD PARTY TOOLS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Small solution (b):

11. 11
BIG PROBLEM #3: HOW TO MANIPULATE
THE APPLICATION’S GUI
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Small solution:

12. 12
WHAT DATA WE COLLECT?
• Test 1
• Webpage load time
• CPU usage
• Memory usage
• Disk I/O
• Test 2
• Timing a “complex” operation
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

13. 13
TEST 1:
READING SYSTEM
STATISTICS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

14. 14
HOW IS THE DATA COLLECTED?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Not at the start of the game
Not at the end of the game
Throughout the
entire game

15. 15
HOW IS THE DATA COLLECTED?
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

16. 16
RUNNING THE TEST
• One test sequence:
• clear cache
• open the browser
• start reading system data
• load webpage
• close the browser
• save result in an Excel spreadsheet
• Full test:
• Restore the “clean” snapshot of the test
environment
• Automatically runs the test sequence for 1000
times on the native browser
• Automatically runs the test sequence for 1000
times on the custom browser
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

17. 17
RUNNING THE TEST
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

18. 18
TEST 2:
TIMING A “COMPLEX”
OPERATION
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

19. 19
“COMPLEX” OPERATION
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential
Open the browser
Load a webpage
Bookmark page
Go to homepage
Open bookmark
Wait for page to load

20. 20
RESULTS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

21. 21
RESULTS
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

22. 22
CONCLUSIONS
Performance testing is very important as it can reveal
performance flaws.
It is very time and effort consuming to do it manually.
It can be automated with some initial effort, but the
returned value is huge.
Offers very conclusive results on what needs to be
improved in the application to boost performance.
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

23. 23
Q&A
© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential

24. THANK
S!Now let’s get to work!
24© Copyright 2014. 3Pillar | All rights reserved Strictly Confidential