Traditional test processes are not adaptive to extensive changes in software. Agile process emphasizes on ability to adapt to changing business needs, customer collaboration, integrated teams and frequent delivery of business values. Agile is an umbrella term that describes a variety of methods including XP and Scrum. The talk will discuss pitfalls of the traditional testing process. Traditional testing process happens very late in the SDLC Where as Agile process focuses on test-first approach. The talk will explain benefits of going agile. Principles and practices of agile process will be discussed and agile methodologies Scrum and Extreme Programming will be discussed in detail. Purpose of Scrum, its effectiveness, timings and managing the scrum will be discussed. Some of the practices for XP like Pair Programming, Test Driven Development will be discussed. The Talk will also cover the QA role in agile world. The talk will cover the implementation issues while shifting from traditional to agile process. Talk will also include an interactive game for illustration of concepts.

1. Agile Testing – Principles and
Practices
Anil Karade
Talentica Software (I) Pvt. Ltd.

2. Agenda

Overview of Traditional Processes

Agile Methodology

Scrum

Agile Testing Principles

Agile Testing Practices With Case Study

Agile Management Tools Used

Benefits

3. How Traditional Test Processes Work

Traditional Planning :
Analysis Design + Dev Testing
Release

Actual :
Analysis Design + Dev Testing
Release

4. Traditional Testing Practices

“Last Defender of Quality” Stance

Detailed preparation and upfront planning

Heavyweight documentation

Strict entrance and exit criteria with signoffs

Attempts at process enforcement

5. What is Agile ?

Agile – A continuous stream of business values

Agile methods in contrast to traditional ones produce
completely developed and tested features at frequent
intervals of 2-4 weeks.

Iterative approaches mean we can trade features for time
instead of sacrificing quality
S1 S2 S3 Sn
Release Release Release Release
Completed, potentially shippable Features
Designed, Coded and Tested

6. Waterfall Vs Agile
Feature Usage

7. Values Expressed in Agile Manifesto:

Individuals and interactions over processes and
tools

Working software over comprehensive
documentation.

Customer collaboration over contract negotiation

Responding to change over following a plan.

8. Scrum

Scrum in game of Rugby

Agile principles will not work without
integrated team mindset

9. Scrum

10. Scrum

Scrum - an Agile approach to software development

Scrum projects make progress in a series of Sprints

Sprint - Timeboxed Period (2 – 4 weeks)

During Sprint, team does:
−
Analysis, Design, Code and Test

Product is potentially shippable after every Sprint

11. Agile Testing - Principles

Shifting QA Focus

From Being last line of defense to providing information,
feedback and visibility.

The adverse relationship between testers and developers
must be replaced with a spirit of collaboration.

12. Principles Contd…

Quality Responsibility –
•
Traditional model – Responsibility of independent QA
group
•
Agile Model - Quality is everyone’s (team) responsibility”.
QA execute test cases, developers do too.

Testing Phase-
•
Traditional model – Testing is a distinct phase that
happens in the end
•
Agile model - QA teams test continuously to measure the
quality and provide feedback.
•
Quality is not added later. It evolves gradually.

13. Principles Contd…

Shortening Feedback Loops

14. Agile Testing Practices with Case Study

Scrum Implementation
•
We follow 2 Weeks Sprint
•
Release sprint (1 week) before going to production
•
At the end of Release sprint final release to production is done

15. Agile Testing Practices with Case Study

Challenges Faced
•
Change in mindset
•
Fear to lose identity
•
Collaboration with programmers and customers
•
Not Understanding Agile Concepts – ‘Scrummerfall’

16. Scrummerfall

17. Testing Practices

QA Role in Release Planning

In contrast to traditional processes testers are involved
from day one of the project.

Sizing Stories – QA gives estimate for testing effort for the
upcoming stories. In some cases testing a piece of
functionality might take longer than coding it.

Prioritizing – Stories that have many unknown areas or
have high business impact should be planned in an early
iteration, so there is time to recover if story blows up.

Infrastructure – Plan for CI setup, test environment, test
data set up

18. Testing Practices Contd...
Clear Requirements = Half Done
Practice we followed -
Full Kit Teams -
•
Separate full kit team was formed that included BA, Dev, QA
experts and Customer.
•
This team worked on getting the requirements clarified from
everyone’s point of view.
•
This ensured that we are minimizing the risk of finding the
defects that will challenge the requirements in the end stages.

19. Testing Practices Contd...

QA Role at Iteration Kick -Off :
•
Learn stories in details
Example Story
Story – 1
As a customer I want to know how much my order will cost to
Ship based on the shipping speed I select so that I can
choose a different shipping speed if I want to.

20. Testing Practices Contd...

QA Role at Iteration Kick -Off :
•
Writing Task Cards –
Dev tasks for Story -1
•
Coding for Shipping API
•
UI Design
•
UI coding
Testing tasks for Story -1
•
Write high level test cases
•
Write detailed test cases
•
Acceptance testing for API
•
Detailed functional testing for API
•
Exploratory testing
•
Automation
•
Review high level test cases with developers

21. Testing Practices Contd...

Coding and Testing :
•
QA works closely with programmers, customers and other team
members to produce stories in small testing-coding-reviewing-
testing increments.
•
Write detailed tests for a story as soon as coding begins. Follow
conversational test case creation approach for writing test cases
Contd…

22. Testing Practices Contd...

Conversational Test Case Creation –
•
Create business facing test cases in conversation with BA.
e.g. business flows overlooked by BA
•
Create technology facing test cases in conversation with
Developers. e.g. browser dependencies, performance, etc
•
Creating test cases in isolation will not assure coverage.

23. Testing Practices Contd…

Identify Risks / Prioritize Test Cases
•
With Agile testing QA not only needs to test sooner, but they
need to uncover high risk areas faster.
•
Execute the test cases in order of the risk associated with it, so
that at the end of iteration we do not end up in finding critical
bugs where we don't have sufficient time to fix.

24. Testing Practices Contd...
Risk Analysis for Story -1
# Item Impact Probability Risk
1 Incorrect cost displayed 4 2 8
2 User can’t choose different 5 1 5
shipping option
3 Estimated cost doesn’t match 3 4 12
actual cost at checkout
4 Invalid postal code entered and 4 1 4
not caught by validation
5 User can’t change shipping 5 2 10
address
6 User changes shipping address, 5 4 20
but cost doesn’t change
accordingly

25. Testing Practices Contd…
Test Early (Behind GUI) –
•
Always ask programmers how a small set of coding effort can be
tested earlier. One of the examples we followed is API testing.
•
With traditional approach QA used to test only in stable builds,
whereas with agile we are uncovering the risks earlier by API
testing, database testing

26. Testing Practices Contd…
Behind GUI (API) Testing –
Testing for Shipping cost API
Assumption – only valid destinations are considered as input
Start with simple happy path testing-
Weight Destination Postal Cost ($)
Code
5 lbs 80104 7.25

27. Testing Practices Contd…

Coding and Testing Progress Together –

Iterative back and forth in coding and testing.

Try Canadian postal code. Result – It returns exception
Revised happy path test –
Weight Destination Country Code Cost
Postal code
5 lbs 80104 US 7.25
5 lbs T2J 2M7 CA 9.40

Any Clarification - Programmer + QA + Customer = Power of Three

28. Testing Practices Contd…
•
Tests that critique the product –

Be aware of all nice to have, enhancement requests.

Don’t jeopardize other stories by spending too much of time adding
things that doesn’t have a big ROI.

Pair Testing

Which bugs to Logs ?

29. Testing Practices Contd…

Exploratory Testing
•
Exploratory testing is done when we get a working model of a
functionality
•
Simultaneously develop and execute test cases while exploring
functionality

30. Testing Practices Contd...

Lightweight Documentation

Instead of writing verbose, comprehensive test
documentation focus on essence of test.

Use collaborative tools – wiki, Review board etc..

Centralize documents – e.g. centralize generic test cases

Leverage documents for multiple purpose

31. Testing Practices Contd...

Automation

Automation is the key to agile.

Automation helps QA in focusing on other important
aspects of testing like functional, exploratory testing.

Focus on automating business logic bypassing GUI when
UI is changing continuously.

Have customizable smoke tests

Use lightweight automation tools for automating unit tests

Tools used: JUnit, Selenium

32. Testing Practices Contd...

Release Sprint Testing
•
Following testing activities need to occur during release
sprint before finally shipping the product.
•
Final pass of end-to-end system integration test
•
Final performance/security test
•
Documentation touch ups
•
User Acceptance Test
•
Time required is greatly reduced (from initial 3 weeks to 1
week, in our case ) as compared to traditional process and
less likelihood of any surprises.

33. Testing Life Cycle
Sprint 1 (2 Weeks)
Test Analysis,
Acceptance Test,
Investigative Test,
Automated Regression Test
Investigative Test :
- Exploratory Test
Sprint 2 (2 Weeks) - Scenario Test
Test Analysis, - System Test
Acceptance Test,
Investigative Test,
Automated Regression Test
Release Sprint (1 week)
end-to-end system Test
Final Performance /Security Test
UAT
Production

34. Agile Tools

We used VersionOne and JIRA for scrum management.

Excel backlog sheets were also used.
http://agilesoftwaredevelopment.com/scrum/simple-product-backlog

35. Agile Implementation Benefits

Increased customer satisfaction

On time deliveries without compromising quality

Iteration wise testing makes QA more focused on
specific functionality which results in effective
testing.

Due to stable builds test cases execution was fast

Reduction in number of defects observed in testing

36. Thank You