Quality Engineering Approaches by Yotam Sharan A lecture given by Yotam on October 8, 2013 at eBay Israel Development Center, as part of eBay's QA Conference.

1. QUALITY ENGINEERING APPROACHES
Yotam Sharan,
Quality Engineering Manager
Oct 2013
Ver. 1.1_P

3. MYTHS & FACTS
QA documentation
Run sanity tests
Full-regression time
COTS test tools
Test Automation
Mocks and tools
No
Developers
Hours-run of
No
X-org F.W.
QA does

4. 4
Every 30 sec. a woman’s handbag is sold on
Every 7 sec. a pair of shoes is sold on
Every 1 sec. transaction volume is $2000 on
Every 10 sec. an Israeli buys an item on
Every 5 sec. an Israeli seller uploads a new listing to
Quality Engineering Approaches
THE SMALL NUMBERS

5. 55
2B pageviews every day
>1TB data turned every second
>100PB processed every day
100B DB calls every day
>50TB new data every day
Quality Engineering Approaches
THE BIG NUMBERS

6. • Buy/Sell anything, anywhere, anyhow
• Big Data
• 10K app servers, 50MLOC
• Multiple releases every day
• Services & Systems Dependencies
• Partial Test env. Vs. Live Data
• Manual Testing
110M
ACTIVE USERS
25M
SELLERS
$175B
COMMERCE VOLUME
FROM BUSINESS TO TECHNICAL CHALLENGES

7. Quality Engineering Approaches 7
OPEN-SOURCE TOOLS
JUnit / TestNG
Selenium/WD
SoapUI
Hudson & Jenkins
Jbehave
Jmeter
Sonar

8. • Exploratory testing
• Regression testing
• Compatibility testing
• Automation testing
• Unit testing
• Code review
• Paired testing
• Test-driven development
• Behavior-driven development
• Bug bashes
• Capacity planning
• Load and Performance testing
• Accessibility testing
• Mocks and stubs
• Analytic testing tools (FindBugs, HTTPWatch, CAL, FireBug, Sonar)
TEST INFECTION
• Integration testing
• E2E testing
• Smoke testing
• Sprint Planning/sizing
• Test case review
• Dogfood testing
• Sprint demo
• Security testing
• Usability testing
• Definition of Done
• CI monitors
• Production monitors
• Live Site Testing
• CrowdTesting

9. Automation testing
Automate everything, regression first. Plan the right automation approach.
Business Value
Providing thin vertical slices of functionality with business value, every iteration.
Continuous Integration
Constant integration testing to assure build integrity and quality, every drop.
Design for Testability
Ensure that it is easier to develop and apply tests for the designed product.
Early Start
The whole-team approach engages test engineers at early design phases.
AGILE DEV & TESTING PRINCIPLES

10. Everyone needs to let go of titles and self-image and engage in a constant
feedback, open, transparent, punishment-free work environment.
Developer test and Testers develop
Analysts, programmers and PM write test scenarios
All engineers help define requirements
Quality is everyone’s responsibility
QUALITY DRIVEN APPROACH

11. Sequential Execution
Test execution: 7000 minutes
Quality Engineering Approaches 11
WEBDRIVER GRID
Parallel Execution
Test execution: 60 minutes
150 VMs/Browsers on WD Grid

12. Quality Engineering Approaches 12
WEBDRIVER GRID
Selenium
Grid Hub
Log
Analyzer:
test
logs, stack
trace, server
logs, config
urat’n
WD
Listeners

13. Quality Engineering Approaches 13
INTEGRATED QE PROCESSES
• Automation Framework for component reuse
• Generic test cases for data-driven testing
• Fully-integrated env. for anyone to run tests easily

14. “In God we trust; all others must bring data” – W. Edwards Deming
Engineering clockspeed
Adopting/Adjusting best practices
Instant feedback: CI, Pair-x, RT dashboards, Quality indicators (bugs, code QI)
Constant feedback: Stories, Planning, Test results, Reviews, Retrospective, Surveys
FEEDBACK & FINE-TUNE

15. CONTINUOUS INTEGRATION & CODE QUALITY

16. Oct 2013
We value
your thoughts…
Win a Prize!
http://x.co/QAconf