Belgium Testing Days - Making Test Automation Work in Agile Projects

Making Test
Automation Work in
Agile Projects

Belgium Testing Days 2012
Lisa Crispin
With Material from Janet Gregory

1

Introductions: Experience, Goals
 Form groups:
 New to automation
 One to two years experience w/ automation
 More than two years experience
 Talk with at least two other people
 Tell each other your learning goals for today
 Note the most interesting one you hear

2

About me…
Programmer, tester, agile team member

3
Copyright 2012: Lisa Crispin

Introduction - You
 Main role on team?
 Programming, automation experience?
 Using agile approach?
 What have you automated? (Test,
CI,deployment...)

4

Takeaways
Foundation for successful test automation
 “Whole Team” approach
 What/when to automate
 Identifying, overcoming barriers
 Choosing, implementing tools
 First steps
We won’t do any hands-on automation, but will demo
some examples

5

Exercise: Your Learning Goals
 Write one interesting learning goal you heard
from another participant at the start of the
class on a sticky note.
 Write your number one learning goal for
today on a sticky note.
 Put the sticky notes on your table group
“learning goals” sheet
 Group similar ones together

6

Let‟s start by defining “agile”

Agile teams:
 Deliver business value
frequently
 at a sustainable pace
 while adapting to the changing
needs of the business

Source: Elisabeth Hendrickson

7

The key is “sustainable pace”

Technical debt slows us down

8

High technical debt +
insufficient automation = even
less time

Barriers to Test Automation
What‟s holding you back?

11

Exercise: Your barriers
Individually and silently – write one barrier
hindering your team per sticky note
Put these on the “impediments” wall chart
for your table group
Talk to your fellow group members – do you
see any patterns?

12

Questions?

13

The Whole-Team Approach
 Automated tests are code
 Respecting the tests
 Collaborating
 Commitment to quality
 Return on investment

14

Automated tests are code
public class CalculatorFixture extends
ColumnFixture {
public String startDate;
public String endDate;
public double startBalance;
public double endBalance;
public String irrTarget;
private Calculator calculator = new
Calculator();
private Double irr;
15

Test code deserves the same respect as
production code!
(Rodney
Dangerfield, a
comedian whose
tag line was “I
don’t get no
respect… )

Source: Gojko Adzic, StarEast 2011 keynote

Testers are especially good at…
 Eliciting examples

 Turning them into tests

 Ensuring the right testing
gets done
 Exploratory testing

19

Give testers time to do what we do best

How?
20

Experiment: Iteration 1
 Pair up: one will be tester, one programmer
 Sit back to back so you face away from each
other
 Tester gets a drawing which needs to be
replicated
 Tester tells the programmer what to draw, all at
one time.
 Programmer draws the shapes based on what
the tester explained.
 No talking during „coding‟!
 Tester “tests” the drawing, reports “bugs” on
index cards
 Programmer fixes the “bugs”
 How long did it take? Will the customer be 22

happy? Copyright 2012: Lisa Crispin

Experiment: Iteration 2
 Collaborate!
 Tester tells programmer what to draw, watches
the programmer draw, answers questions, points
out „defects‟ for programmer to fix immediately
 (Don‟t show the programmer the drawing, that
makes it too easy, we‟re trying to simulate real
coding)
 How long did it take? Will the customer be
happy?

Thanks to the members of the agile-games group and
Kane Mar for ideas & pictures for this game

23

Ways to collaborate

Pair

24

Team responsibility
Automate all regression tests

25

26

Commitment to quality
What‟s your team‟s commitment?
The best possible software product?

27

Meaningful
Commitmen
t

Experimen
t

Learn
28

Under-commit
Plan less work than you think you can do
Including all test automation


Learn to write maintainable tests
 Get over the “hump of pain”
From Gerard Meszaros’
XUnit Test Patterns

30

Whole-team software development
Create a Expand
user tests –
story Story
Tests
Automate
Write Q2 Tests Pair,
Customer Start “Show
(Q2) thinking Me”
Tests how to
code
TDD

Exploratory
testing
Product owner
Product owner/ Tester
Tester
Tester/Programmer
Programmer Customer
User
Acceptance

31

Exercise

•Think of an experiment to get your
whole team engaged in automating
tests
•Share with your table group
•Pick two to share with the class


Getting Over the Hump
 The test automation pyramid
 The agile testing quadrants
 What should be automated
 What shouldn't
 Difficult areas

33

Test Automation Pyramid

34

Agile Testing Quadrants

35

Shared Understanding
 Start with tests
 Collaborate

36

What Should We Automate?
 Quadrant 1 tests
 Unit, component, TDD
 API, service-level
 Load, performance, stress
 Quadrant 3 tests?
 Leverage automation where useful

37

What Shouldn‟t We Automate?
 Wizard of Oz, prototyping
 Usability, UAT, ET
 Tests that will never fail?
 Assess risk
 ROI not enough
 One-off tests

38

Where Should We Be Careful?
 GUI tests
 Watch ROI
 End-to-End tests
 Push testing down to lowest level
 Remember the Pyramid

39

Hard to Automate?
 Legacy code
 Hard to automate, or just lack of skill?
 “Working Effectively with Legacy Code” –
Feathers
 “Strangling” – Fowler, Thomas

40

Exercise: Low-Hanging Fruit

41

Agile Automation Strategy
 What hurts the most
 Layered approach
 Applying agile principles
 Small chunks/thin slices
 Smart test design
 Choosing the right tools

42

What Hurts the Most
 Use retrospectives
 Keep an impediment backlog

43

Multi-Layered Approach
Example:
 Learn TDD at unit level
 Automate GUI smoke tests

44

Simplicity
 Address one or two needs at a time
 Understand the problem first
 Try simplest approach first
 Work in small chunks, thin slices
 Incremental & iterative

45

Automate a Slice at a Time
Example: 4-step UI to validate, upload profit
sharing contribution data
• Thread 1: All four pages with navigation
• Thread 2: Select year, enter description on page
1, display on page 2, browse and upload file on
page 2
• Thread 3: Validate data in file, display on page 3
• Thread 4: Persist data, display „success‟
message on page 4

46

Thin Slice Example

47

Exercise: Thin Slices
Here‟s our user story:
As an internet shopper, I want to create an
account so that I do not have to enter my address
and billing information each time I make a
purchase
Draw a mind map for this story on a big sheet of
paper
Identify a basic end-to-end slice of functionality
that can be coded, tested, and automated.
If you have time, identify additional slices.

48

Test Design Patterns/Principles
 Code design patterns
 One clear purpose
 Don‟t Repeat Yourself

49

Demo

50

Iterative Feedback
 Spike two different approaches
 Pick one to try for N # of iterations
 Use retrospectives to evaluate

51

Learn by Doing
 Courage – don‟t be afraid to fail
 Production code practices for test code
 Incremental, thin slices
 Experiment

52

Questions About Automation Strategy?

53

Choosing Tools
 Team effort
 Time
 Requirements
 Focus on goals, problems, not tools.
 Experiment

54

Understand the Purpose
 Who‟s using the tests? What for?
 What‟s being automated?
 Existing tools, environment
 Who‟s doing what for automating?

55

What Fits Your Situation
• Existing skills
• Language of application under test
• Collaboration needs
• What‟s being automated
• Life span, future use of tests

56

Test Drivers/Frameworks
 Automation layer
 Fit for everyone on team
 Try out more than one
 Let‟s look at more examples

57

Where To Find Tools
 www.softwareqatest.com/qattls1.html
 www.testingfaqs.org
 www.opensourcetesting.org
 awta.wikispaces.com/2009ToolsList
 groups.yahoo.com/group/agile-testing
 http://bit.ly/AgileTestTools - aa-ftt spreadsheet

58

Example: My Team‟s Tool Choices
• IntelliJ Idea
• Jenkins, ant, Maven
• JUnit
• FitNesse
• Canoo WebTest
• Watir
• JMeter
• Selenium 2.0 / WebDriver with Geb framework

59

Exercise: Tools

60

Making Test Automation Work

 Time to do it right
 Learning culture
 Testable architecture
 Test data
 Managing tests

61

Time To Do It Right

 Limit scope, don‟t over-commit
 Write automation task cards
 Quality must be team goal
 Long-term, will let you go faster

62

Learning Culture

 OK to make mistakes
 Lots of small experiments
 Slack
 Evolve right design

63

Testable Architecture
• Layered architecture
• eg. UI, business logic, data access
• Ports and Adapters pattern
• App can work without UI or database
• Ports accept outside events
• Adapters convert for human or automated
users

64

Test Data

 Avoid database access when possible
 Setup/Teardown
 Independent, rerunnable tests
 Canonical data
 Refresh before each test run
 Customizable data for ET
 Production-like data
 Get customers to provide example data

65

Managing Automated Tests

 Tests as Documentation
 Continuous Integration
 Reporting results
 Metrics

66

Tests as Documentation
 Understandable
 Who will really use them?
 Once passing, must always pass

67

Any Example Can Become a Test

68

Given/Then/When Example
Scenario: Valid name search returns results

GIVEN that Kant is a supervisor with employees
AND Kant has an employee named Smith
WHEN Kant navigates to the employee name
search page
AND enters the value “S”
THEN Kant will see a search result that includes
Smith

69

Continuous integration/testing…
…short feedback loop

70

Keep tests passing
Stop the line to fix problems

71

Test Management Tools
 Manage tests, code together
 Some tools have own management
 What problem are you trying to solve?

72

Exercise: Tests as Documentation

73

Key Success Factors

Thin Slices

Specificatio
n by
Whole Team Example
Approach
Short
Feedback
Loops

Experiments
74

Exercise: Breaking Barriers

75

Remember
 It‟s a team problem!
 Tackle automation problems with diversity
 Try small experiments
 Baby steps – start simple

76

Questions? “Aha” Moments?

77

Agile Testing: A Practical Guide for Testers and Agile
Teams

By Lisa Crispin and Janet Gregory

www.agiletester.ca

78

Experiences of Test Automation
Dorothy Graham and Mark Fewster

79

Beautiful Testing: Leading Professionals Reveal How
They Improve Software
Edited by Tim Riley, Adam Goucher

Includes chapter by yours truly

80

Test Patterns

Xunit Test Patterns: Refactoring Test Code
By Gerard Meszaros

81

Specification by Example
How successful teams deliver the right
software

Gojko Adzic

Case studies from > 50 teams

82
Copyright 2008 Janet Gregory, DragonFire

Agile Test Automation Resources
dhemery.com/pdf/writing_maintainable_automated_accep
tance_tests.pdf
lisacrispin.com
janetgregory.ca
gokjo.net
exampler.com
agile-testing@yahoogroups.com
testobsessed.com
testingreflections.com
pairwith.us

83

Belgium Testing Days - Making Test Automation Work in Agile Projects

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (19)

Semelhante a Belgium Testing Days - Making Test Automation Work in Agile Projects

Semelhante a Belgium Testing Days - Making Test Automation Work in Agile Projects (20)

Mais de lisacrispin

Mais de lisacrispin (14)

Último

Último (20)

Belgium Testing Days - Making Test Automation Work in Agile Projects