talk about tipical agile testing practice

1. Agile Testing
Principles and Practice

2.  Test manager, Program and project
manager
 11+ years’ delivery experience
 6+ years’ agile experienceMary Jiang
姜 融

3. About you?
 Managers, testers, ScrumMasters…?
 How long has your team used agile
principles/practices?
 Which kind of testing your team are
working on?

4. Group Discussion (10 minutes)
What is your biggest concern regarding agile testing?
Or
What interests you most about agile testing?

5. Presentation Time
After discussion every Group needs to select and present
your top 3 items

6. Agile Context

7. (Image provided by Wikimedia, reproduced here
under a Creative Commons license.)

8. Agile context –Manifesto
Individuals and
interactions
over processes and tools
Responding to change
over following a plan
Customer collaboration
over contract negotiation
Working software
over comprehensive documentation

9. Scrum Flow

10. Agile Testing

11. What is agile testing?
 A software testing practice that follows the principles
of agile software development
 Involves all members of a cross-functional agile team,
with special expertise contributed by testers
 Testing from the customer perspective as early as
possible
 Includes everything beyond unit and component level
testing
 Does not just mean testing on agile project

12. Agile Testing vs. Traditional Testing

13. Traditional testing V-Model

14. The pain of traditional testing

15. Testers’ pain from traditional testing

16. Is it agile testing?

17. Is it agile testing?

18. Is it agile testing?

19. What does agile testing mean?
Challenge
 Continuous Change
 Less documentation
 Keep everyone on the same page
 Face to customer
 More testing effort
Opportunity
 Everyone owns quality
 Earn business value earlier
 Break the boundaries
 Involve the customer
 Quick feedback

20. Whole team approach
 Scrum Team commitment to quality
 Scrum Team responsible for testing
 Product owner responsible for acceptance testing
 Development team responsible for all level’s testing

21. Agile testing mind-set
 Don’t be afraid
 Enjoy learning new skills and taking on new challenges
 Not limited in the areas you can test
 No judgment
 Results-oriented
 Collaborative
 Passionate

22. Ten principles for agile testers
 Provide continuous feedback
 Deliver value to the customer
 Enable “face-to-face” communication
 Have courage
 Keep it simple
 Practice continuous improvement
 Respond to change
 Self-organize
 Focus on people
 Enjoy
-Lisa Crispin and Janet Gregory

23. Change doesn’t come easy
 Be patient
 Let them feel pain
 Build your credibility
 Work on your own professional development
 Beware of the quality police mentality

24. Typical activities
for agile testing
• PROJECT INITIATION AND
PRE-PLANNING
• SPRINT N-1 AND SPRINT N
• CLEAN UP AND DELIVERY

25. Testing activities in
project initiation
and pre-planning
 Definition of “Done”
 Release planning
 Test planning
 Preparation for visibility
 Team commitment

26. “
”
Nothing is done until it is tested
FAMOUS SAYING IN AGILE
Definition of “Done”
Acceptance criteria

27. Agile tester through release planning
 A big picture view
 Sizing stories from the testing perspective
 Testing considerations while prioritizing
 What is in scope?

28. Sample big picture
Web Svc
SIEBEL SVCEIM LOAD CTI
UCM
REMEDY TOOLS
IVR
ESP
Portal
Siebel
UI
Alarm Etc
Web Interaction
Agent Ticket Application
Phone
Phone
W/S
Test
Perf
Load
Tech
UCM
Ubiquity
Portal
Web Interaction
GDC
US

29. Test planning
 Type of testing
 Infrastructure
 Test environments
 Test data
 Test results

30. Lightweight test plans
 Lightweight documents
 Test matrix
 Test spreadsheet
 A white board
 Online tools

31. Sample test plan

32. Sample test plan

33. Preparation for visibility
 Tracking test tasks and status
 Communicating testing results/bugs
 Release metrics
 Test status
 Traceability/coverage
 Defect status

34. Consideration of Bugs
 Is it a bug or is it a new feature?
 Use a Defect Tracking System?
 Which bugs need to be logged?
 When to fix bugs?
 Rules for dealing with bugs

35. Team commitment
 Definition of “Done”
 Ensure enough involvement
 Communication channels
 Agreement for visibility
 Collaboration in a distribution environment

36. Testing activities
in Sprint N - 1 and
Sprint
 Sprint planning
 Driving Requirement
 Driving development
 Collaborate with developers
 Facilitate communication
 Completing testing tasks
 Regression Tests
 Sprint Metrics
 Demo
 Retrospective

37. Sprint N-1
Are the requirements for Sprint N ready or
not?

38. What will testers do during Sprint Planning?
 Learn more details
 Considering all viewpoints
 Think about testability
 Write task cards for testing
 Ensure testing effort is considered in estimation

39. Samples of task cards for testing

40. Samples of task cards for testing

41. Samples of testers sprint plan

42. Driving requirement
 High level tests and examples
Flowchart diagram
Test/verification points
 Test driven requirements
Q&A with customer/representative
High level tests review

43. Sample for high level flowchart
Places Bid
Purchases
Item
Creates
Account
Searches
listings for
item
Creates an
Auction
Ships Item
Buyer
Seller
User

44. Samples of Verification Points

45. Q&A and review
 Collaboration tools to track Q&A and review
comments
 Online and offline review with
customer/business representative

46. Driving Development
 Start Simple
 Add complexity
 Risk analysis
 Coding and testing progress together
 Focus on one story at a time

47. Collaborate with developers
 Pair Testing
 “Show Me”

48. Facilitate communication
 Daily Stand-up
 Bridge the gap between business and
development
 Collaboration tools for distributed teams
 Understand and sacrifice a little time

49. Completing Testing Tasks
 Proactive - test as early as possible
 Address the testing crunch
 Anyone can do testing
 Non-functional testing

50. Regression Tests
 Automation will be preferred
 Keep the build “Green”
 Keep the build quick
 Building a Regression Suite
 Checking the “Big Picture”

51. Sprint metrics
 Measuring progress
 Burn Down Chart
 Is testing done?
 Coverage
 Unit Test
 Code
 Test coverage
 Defect Status

52. Samples for metrics – Test Status

53. Samples for metrics –
Traceability/Coverage
Category
Priority
TC Covered Defects TC NON-Covered Defects Total
# % # %
4-Business Critical 25 60.98% 16 39.02% 41
3-High 34 58.62% 24 41.38% 58
2-Intermediate 28 62.22% 17 37.78% 45
1-Low 19 76.00% 6 24.00% 25
Total 106 62.72% 63 37.28% 169

54. Samples for metrics – Defect Status
0
50
100
150
200
250
300
350
400
450
500
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8
Total Bugs
Total Fixed Bugs Ignore Reopen Situation(Forcast)
Incremeantal Won't Fixed Bugs
Total Fixed Bugs With Reopen Situation(Forcast)

55. Sprint Review (demo)
 Conduct the demo
 Listen to the customers
 Ask questions if possible
 Do not forget the “big picture”

56. Sprint Retrospective
 Start, stop, continue
 Identify test-related obstacles
 Focus on a few action items for the next sprint

57. Clean Up and
Delivery
 Planning enough time for testing
 Testing release candidate
 Testing on a staging environment
 Final non-functional testing
 Integration with external applications
 Product data conversion and load
 Installation testing
 Communication
 Support customer testing

58. System Testing
 Planning in advance
 Test environment(QA/testing and staging)
 Final testing including non-functional testing
 Cooperation with third-party teams
 ETL/data conversion
 Communication constantly
 Installation testing
 Defect trend

59. What if it’s not ready?
 Was the build broken in the last day?
 Any show stopper bugs not detected until final
functional testing?
 Any new non-functional issues found?
 Any test data problems?
 Integration issue?

60. User Acceptance Testing
 High level plan and scope
 Support and collaborate with customer
 Get feedback from customer
 Understand how customers view products

61. Deliverables
 External Customer
 Internal Customer
 Team and Organization Assets

62. Transition from
traditional tester
to an agile one

63. Know yourself
 Change your testing thinking
 Values are more important than process and
practice
 Existing testing skills are useful
 Build you own credibility

64. Agile Testing Quadrants
Functional Tests
Examples
Story Tests
Prototype
Simulations
Exploratory Testing
Scenarios
Usability testing
User Acceptance Testing
Alpha/Beta
Unit Tests
Component Tests
Performance & Load testing
Security Testing
“ility” testing
Application
under test
Business View
Technology View
Supportingtheteam
Critiqueproduct

65. Business supporting test
 Support Team
 Right tools to elicit requirements
 From big picture to details
 Driving development with business – facing testing
 Critique product
 Demonstrate applications to stakeholders to get early
feedback
 Use scenarios and workflow to test from end to end

66. Business supporting testers’ role
 As a bridge between business and development
 Required skills and capability
 Domain knowledge
 Solid testing skills and technical skills as supplement
 Critical and logic thinking
 Soft skills
 Personalities
 Automation skills

67. Ashely and Mina's story

68. Technical Supporting Test
 Support Team
 Unit tests
 Component Tests
 Continuous build
 Critique product
 Performance/load testing
 Security testing
 “ility” testing

69. Technical supporting testers’ role
 Driving development and non-functional needs
 Required skills and capability
 Solid testing and technical skills
 Architecture level understanding as preferred
 Developing and Programming skills
 Requirement and results analysis skills

70. Alex and Colin’s Story