IMT Sharing Knowledge: Agile Testing Introduction

2. Agenda
 Agile Recap
 Agile Testing Overview
 Agile Testing Game
 Agile QA Common Approach
 Test Strategy in Agile Environment
 Advanced Problems For Discussion
 Summary Activity
IMT Confidential
2

3. Agile Recap
IMT Confidential
3

4. Agile Recap
IMT Confidential
4

5. Testing terminology
– Unit testing
• Automatic test on code level run every night or after new code is added – to verify no bugs
introduced
– Integration testing
• Testing interactions between different modules of the system
– Feature testing
• Performed by a QA when a feature is finished implemented
– System testing
• To verify whether the system as a whole meets the specified requirements
– ExploratoryTesting
• Exploratory testing is simultaneous learning, test design, and test execution.
– Regression Testing
• Test to detect side effects from changes to the system, i.e after bug fix
– Retesting
• Verification of bug fix
– Compatibility testing
• Verification of compatibility between AMX applications and the different OS, office applications, browsers
and databases etc that we support
5 thinking levels of Tester.
-----------------------------------
You don't believe it until you see it.
You don't believe everything that you see.

6. Agile Testing Overview
• Agile testing is a software testing practice that
follows the principles of agile software development
and involves all members of a cross-functional agile
team.
• It is a set of activities intended to ensure that
products satisfy customer requirements in a
systematic, reliable fashion.
• In SCRUM (agile) QA is the responsibility of
everyone in R&D, not only the Test team. QA is all
the activities we do to ensure correct quality during
development of new products.
IMT Confidential
6

7. Agile Testing Overview
• Agile testing focuses
on testing iteratively,
as often as stable
code base is
available, until quality
is achieved from end
customer’s
perspective.
IMT Confidential
7

8. Agile Testing Overview
11/19/2015
IMT Confidential
8

9. Agile Testing Overview
Agile Testing Traditional SQA
Agile Testers are integrated, full-fledged
members of dev team and participate in
planning, estimating and all team
activities.
Testing is performed by a separate
organization or group reporting to SQA
Manager.
Agile testers work hand-in-hand with
development and product manager.
SQA teams may work with other groups
but with limited interaction and well-
establishment boundaries.
Business requirements are written a
piece at a time to accommodate
changing business needs.
SQA receive detailed business
requirements and schedules up front.
Throughout the release additional test
scenarios are uncovered through
exploratory testing.
When development is presumed
complete, the application is delivered to
SQA for testing
IMT Confidential
9

10. Agile Testing Overview
Agile Testing Traditional SQA
Agile testing is continuous and
feedback is provided during all stages
of development.
SQA executes a complete cycle of test
and reports defects back to
development.
Agile testing follows a fluid, continuous
process with defects fixed as they are
found.
When testing is complete,
development fixes defects and
delivers another release to SQA
Agile tester are expected to radiate
information and provide complete
visibility into all test-related activities for
the benefit of the larger group.
Traditional SQA teams often keep test
design, implementation and progress
specifics within the SQA site.
Quality software is always ready to
be delivered.
The process of test and fix repeats
until time runs out.
IMT Confidential
10

11. Agile Testing Overview
IMT Confidential
11
• Agile Testing Mindset

12. Agile Testing Overview
IMT Confidential
12
• Exploratory Testing in Agile Environment

13. Agile Testing Overview
IMT Confidential
13
• Exploratory Testing in Agile Environment
Factor Scripted Exploratory
Domain Knowledge Lack of domain knowledge
can be overcome during test
design
Requiring sufficient domain
knowledge
System Complexity Allows for careful design of
test scripts
Relies on skill set of tester
Level of Doc Require good doc to be
supplied
Just assumes good domain
knowledge
Timeframes and Dealines Significant lead-in time
required
Little or no lead-in time
required
Efficiency ? ?
Coverage ? ?
Verification ? ?
Acceptable Risk Level ? ?
Reproducticibility ? ?

14. Agile Testing Overview
IMT Confidential
14
• Exploratory Testing in Agile Environment

15. Agile Testing Overview
• Challenges in Agile Testing
– Short Iterations & Releases
• 1-week, 2-week, 3-4 week release cycles, even shorter feedback cycles
• Validate new “user stories”, ensure old functionality continues to work, communicate bugs, fix
them and validate again.
– Less time for documentation
• Need tools/formats for documentation that cover essence of the test but are lightweight
– Dealing with ‘Slacker’ and ‘Asker’
– Is the number of defects reported a good metric to measure testing
effectiveness?
– Is automation required only when regression testing is needed?
– How to manage Time zone difference?
– How to cope and deal with changes of requirement
IMT Confidential
15

16. Agile QA Game
IMT Confidential
16

17. Agile Testing Game
• Building IMT Agile Testing Manifesto
IMT Confidential
17
Candidate Final
Duplicate Not Clear

18. Agile QA Common Approach
IMT Confidential
18

19. Agile QA Common Approach
Responsible for Test Preparation
 Take part in requirement elaboration
 Attend Sprint planning meeting
 Collaborate well with Product Owner to clarify user
stories/PBI and the concept of tests
 Prioritize all tasks
 Estimate time for testing features/backlogs
 Identify and mitigate risks
19

20. Agile QA Common Approach
Responsible for Test Execution
 Attend Daily Stand up Meeting
 Design Test cases
 Automate acceptance test cases wherever possible
 Perform regression, exploratory & acceptance testing
 Write/update & execute acceptance criteria
 Identify the gap between what the customer expects
and what the team delivers
20

21. Agile QA Common Approach
Responsible for Test Evaluation
 Be part of Sprint review & Retrospective meeting
 Responsible for evaluating test deliverables
 Provide continuous feedback to development team
21

22. Agile Quality Assurance Strategy
• Test strategy in agile development
IMT Confidential
22

23. Test Strategy in agile
• We start our QA activities at Release planning stage
• Release planning
– Create test plans
• High level test planning
• budgeting enough time
– Participate in story sizing
– Typical Test plan will include
• Scope of testing
• New functionalities which are being tested
• Types/ level of testing based on the complexity of the features being tested
• Performance and load testing
• Infrastructure consideration (Test environment/software)
• Risks/ mitigation plans
• Resourcing
• Milestones and deliverables

24. Test Strategy in agile…
• Each Sprint
– Participate in sprint planning
– Estimate tasks(QA input)
– Write test cases using stories(QA)
• High level test cases before coding begins- guide dev
• Detailed test cases written during coding
– Unit/ Integration tests(Dev)
• Automatic test on code level run every night or after new code is added. Bugs found are fixed immediately
– Pair tests(Dev + QA)
• Performed when a part of a feature is finished implemented. Bugs found are fixed immediately
– Feature tests(QA)
• Performed when a feature is finished implemented. Errors found are logged in main project in bug system
and prioritized during next sprint planning. After all prioritized bugs are fixed, the feature is ready for System
Test at the end of the project
– Reviews (stories, req, test cases with customer, programmer)
• Increase collaboration and communications

25. Test Strategy in agile…
• Each Sprint contd…
– Test automation
• Automate new functional test cases
• Run automated regression tests
– Perform non functional testing (load, security, usability etc)
– Demo to the stakeholders

26. Test Strategy in agile…
• Hardening sprint
– A hardening sprint can be used for bug fixes in previous sprints. Bugs that are
prioritized will be considered here
– The end game(System test)
Every member in the team is involved in this.
• Mock deploy on staging
• Smoke test on staging
• Perform non functional testing (load, security, usability etc)
• Complete regression test
• Perform UAT tests
• Other tests
– Compatibility(DB upgrade etc)
– Installation
– Localization etc

27. Test Strategy in agile…
• Release
– Define Exit criteria
• Testing activities (bug severity, open bugs, coverage levels, metrics
etc)
• Product Artifacts (user guide, installation guides, other necessary
documentation are in place)
– Participate in retrospectives
• Start
• Stop
• Continue

28. Iteration metrics
To enhance predictability…
– Measuring progress
• Burn down charts
• Estimated vs. Actual time for tasks
– Defect metrics
• Root cause analysis
• Results from defect tracking system
– Traceability metrics
• Test coverage (story vs. test cases)
• Code coverage

29. Advanced Problems
• Agile Testing Interview Questions
– What is Agile Testing and how is it different to traditional
waterfall or the V model?
– What is your approach when requirements change
continuously?
– What are good characteristics of an Agile Tester / QA?
– What are the two key factors when working as a QA in an
Agile team?
– What are the three main roles in Scrum?
IMT Confidential
29

30. Advanced Problems For Discussion
IMT Confidential
30

31. Advanced Problems For Discussion
• Impact Analysis
IMT Confidential
31

32. Advanced Problems For Discussion
• IMT Projects lesson learnt
– Xfinity Home Security
– Precyse
– Precyse Automation
– TabSquare
– …………………………………
IMT Confidential
32

33. Summary Activity
• Using mind map to draw the map like below
to summary your knowlegde of Agile Testing
IMT Confidential
33

34. Summary Activity
• An example of mind map about Agile Testing Day
IMT Confidential
34

35. Conclusion By Image
11/19/2015
IMT Confidential
35

36. Main References
IMT Confidential
36
http://www.istqb.org/
An Le, Tester Mindset
in Agile Scrum Model
http://jp.imt-soft.com/Blogs/Agile-Scrum/Tester-Mindset-in-
Agile-Scrum-Model

37. Main References
IMT Confidential
37
Nguyen Vu Hung, Tran Son Hai and et al, Tài
liệu Hướng dẫn Scrum dịch từ bản gốc được
cung cấp bởi Ken Schwaber và Jeff Sutherland,

38. Main References
IMT Confidential
38

39. Thank You
IMT Confidential
39

40. Questions
IMT Confidential
40
• Lazada Story Question:
• Recap Question: List the
pros and cons of Agile
Testing in Outsource
Project?