2. About Myself
• Rajdeep Varma
• 5 years in Quality Engineering. I work at
ThoughtWorks
• Passionate about test automation and agile
practices
• Tested a number of web apps and recently
started testing mobile apps
3. I will talk about
• Our Problem: Long running automation
• Our Solution: parallel execution of tests by
multiprocessing on same machine.
4. Typical CI Pipeline Of Mobile Project
Git Push
Compile
+
Unit Test
Build App UI Tests Distribute
“Fast Feedback”
5.
6. Stats
• 5 git repo
• Avg. 30 commits a day
• Approx. 100 Automated UI tests.
• “GitPush” TO “TestedOK” > 1 Hrs
7. With slow tests
Git Push
UI Tests
Deploy to
next
stage
Umm…slow tests—Chai Break—damn! test failed :(
Fixed!
<pain>its so
slow<pain/>
— Pray —
14. Options
• Multiple parallel builds in CI
• Proprietary cloud based tools
• Create a tool that matches our need
http://images.clipartpanda.com/choice-clipart-fork-choice-hi.png
21. How it works?
<process spawner>
1_digit_addition
2_digit_addition
3_digit_addition
4_digit_addition
5_digit_addition
6_digit_addition
7_digit_addition
8_digit_addition
ENV[ADB_DEVICE_ARG] = emulator-
5554
ENV[TEST_PROCESS_NUMBER] = 1
Proces
s
Spawn
er
ENV[ADB_DEVICE_ARG] = emulator-
5556
ENV[TEST_PROCESS_NUMBER] = 2
ENV[ADB_DEVICE_ARG] = emulator-
5558
ENV[TEST_PROCESS_NUMBER] = 3
ENV[ADB_DEVICE_ARG] = emulator-
5560
ENV[TEST_PROCESS_NUMBER] = 4
S
U
M
M
A
R
I
Z
E
R
E
S
U
L
T
stdou
t
22. Reporting
config/cucumber.yml
default: --format html --out my_report_<%=ENV['DEVICE_INFO']%>.html
Reports from 3 devices:
my_report_GT_N7100.html
my_report_Samsung_Galaxy_Note_2___4_3___API_18___720x1280_1.html
my_report_Samsung_Galaxy_Note_2___4_3___API_18___720x1280_2.html
23. Optimization in test
distribution
• Issue with distribution based on cucumber
“featue” files
• Distribution based on “Scenario” instead of
features
• --group-by-scenarios
24. Advantages of fast tests
• Fast Feedback
• Faster development cycle for BDD environment
• Modern machines have multiple cores and
parallelization can make Maximum use of
Hardware
• Faster tests makes developers & Testers happy
25. what did we achieve?
• Speed
• Scalability
• Reporting
• Process specific action using
ENV[TEST_PROCESS_NUMBER]
26.
27. Choose One
a) Lots of automated tests for good coverage
b) Faster automation run
c) BOTH!