Automated Acceptance Testing and Continuous Delivery using Thucydides and Jenkins.

1. Continuous Integration
Continuous Quality
Continuous Delivery

2. Who is this guy, anyway?
Consulta
nt
Trainer
Mentor
Author
Speaker
Coder
John Fer
guson S
mar t

3. Training and Consulting Services in
TDD, BDD and ATDD
Automated Acceptance and Regression Testing
Build Automation and Continuous Delivery
Clean Coding Practices

4. Continuous Delivery
Getting value to the business.
Fast.
Often.
Continuous Delivery: everyone wants it
4

5. But Continuous Delivery is impossible
without Continuous Quality
Quality must be built in to the process
5

6. So how do you deliver?

7. Goals
Capabilities
Features
Stories
Examples
Acceptance Criteria

8. Successful projects start with a shared vision
“We are going to build an online classifieds website”

9. You define goals to achieve your vision
“Let’s increase advertising
revenue by letting sellers
post their classified ads
“Let’s get more sales for our advertisers by
making the ads easier to find online.”
Goals that will deliver business value

10. Features help deliver these goals
“Let’s get more sales for our advertisers by
making the ads easier to find online.”
Notify potential buyers about new items
In Search for online of advertised articles
order to increase sales ads
As aorder to increase sales of advertised articles
In seller
I want previous buyers to know about new items
As a seller
that theybuyers be interested in buying ads for
I want might to be able to easily find
articles they want to buy

11. We use examples and stories to explore the features
Search for online ads
“Searching by category”
“Searching by keyword and category”
“Searching by keyword and location”
“Filter ads by price”

12. We use examples and stories to explore the features
Search for online ads
Searching by keyword and location
Given
Sally
wants
to
buy
a
puppy
for
her
son
When
she
looks
for
ads
in
the
‘Pets
and
Animals’
category
containing
the
word
‘puppy’
in
New
South
Wales
Then
she
should
obtain
a
list
of
ads
for
puppies
in
New
South
Wales.
Acceptance Criteria illustrate and validate the stories

13. We automate the acceptance criteria
search_by_keyword_and_location.story
Scenario: Searching by keyword and location
Given Sally wants to buy a puppy for her son
When she looks for ads in the ‘Pets and Animals’ category containing
the word ‘puppy’ in New South Wales
Then she should obtain a list of ads for puppies in New South Wales.

14. And we implement the acceptance criteria
search_by_keyword_and_location.story
Scenario: Searching by keyword and location
Given Sally wants to buy a puppy for her son
When she looks for ads in the ‘Pets and Animals’ category containing
the word ‘puppy’ in New South Wales
Then she should obtain a list of ads for puppies in New South Wales.

15. We can now execute our automated acceptance tests
“Searching by keyword and
location” is now a pending test

16. ...and implement the application code for that feature
When it is implemented, it will
be displayed as a passing test

17. ...and implement the application code for that feature
And the corresponding feature
is now partially completed

18. What’s so important about
automating the Acceptance Criteria?

19. So why automate your acceptance criteria?
They define your goals

20. So why automate your acceptance criteria?
They keep you on track

21. So why automate your acceptance criteria?
They provide better visibility

22. So why automate your acceptance criteria?
They allow faster release cycles

23. So why automate your acceptance criteria?
Reduce Risk

24. So why automate your acceptance criteria?
Reduce Costs

25. Jenkins - it’s how we automate stuff, boss.
Compile and Test
Quality Gateways
Release Candidates
Reporting and Documentation
Automated Deployment
Continuous Delivery

26. Build Pipelines
T EST UAT
PROD
Getting your code into production

27. A simple build pipeline
Build
and
Fast
Tests
Slower
Tests
Acceptance
Tests
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
Each step is a quality gateway UAT
TEST

28. Build and initial tests
Build
and
Fast
Tests
Fail fast
Slower
Tests Let people know
Acceptance
Tests Never ignore a broken build
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

29. Build and initial tests
Build
and
Fast
Tests
Fail fast
Slower
Tests Let people know
Acceptance
Tests Never ignore a broken build
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

30. Build and initial tests
Build
and
Fast
Tests
Fail fast
Slower
Tests Let people know
Acceptance
Tests Never ignore a broken build
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

31. Build and initial tests
Build
and
Fast
Tests
Fail fast
Slower
Tests Let people know
Acceptance
Tests Never ignore a broken build
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Instant Messaging Plugin Deploy
to
ProducLon
Email-ext Plugin
Hudson Sounds Plugin
... UAT
TEST

32. Automated Acceptance Tests
Build
and
Fast
Tests
Progress reports
Slower
Tests Living Documentation
Acceptance
Tests Help for testers
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

33. Automated Acceptance Tests
Build
and
Fast
Tests
Progress reports
Slower
Tests Living Documentation
MakeAcceptance
Tests
them fast Help for testers
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
Distributed builds
Selenium Grid
SauceLabs
UAT
...
TEST

34. Automated Acceptance Tests
Build
and
Fast
Tests
Progress reports
Slower
Tests Living Documentation
Make them public for testers
Acceptance
Tests Help
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
Notifications
HTML Publisher Plugin
... UAT
TEST

35. Automated Acceptance Tests
Build
and
Fast
Tests
Progress reports
Slower
Tests Living Documentation
Make them public for testers
Acceptance
Tests Help
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
HTML Publisher Plugin TEST

36. Checking Code Quality
Build
and
Fast
Tests
If a metric matters, fail the build
Slower
Tests Proactive Code Quality
Acceptance
Tests Complements Sonar
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

37. Checking Code Quality
Build
and
Fast
Tests
If a metric matters, fail the build
Slower
Tests Proactive Code Quality
Acceptance
Tests Complements Sonar
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

38. Checking Code Quality
Build
and
Fast
Tests
If a metric matters, fail the build
Slower
Tests Proactive Code Quality
Acceptance
Tests Complements Sonar
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

39. Checking Code Quality
Build
and
Fast
Tests
If a metric matters, fail the build
Slower
Tests Proactive Code Quality
Acceptance
Tests Complements Sonar
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UEnforcing code quality
AT
Deploy
to
ProducLon
Cobertura Plugin
Checkstyle Plugin
FindBugs Plugin UAT
... TEST

40. Generating a release candidate
Build
and
Fast
Tests
Build a release candidate early
Slower
Tests Reuse, don’t rebuild
Acceptance
Tests
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

41. Generating a release candidate
Build
and
Fast
Tests
Build a release candidate early
Slower
Tests Reuse, don’t rebuild
Acceptance
Tests
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
Choose a release
strategy UAT
TEST

42. Generating a release candidate
SCM
commit
SCM server
Build
and
Fast
Tests
Slower
Tests
SNAPSHOT
Acceptance
Tests
Code
Quality
Metrics
Release
Candidate
1.0.1

43. Generating a release candidate
What about maven?
SCM
commit
SCM server
Build
and
Fast
Tests
Slower
Tests
SNAPSHOT
Acceptance
Tests
Code
Quality
Metrics
Release
Candidate
1.0.1

44. Generating a release candidate
What about maven?
Strategy 1 : The Maven Release Plugin
but not as we know it...
1.0.0-SNAPSHOT
mvn
release
1.0.65

45. Generating a release candidate
What about maven?
Strategy 2 : Just set the version
1.0.0-SNAPSHOT
mvn
versions:set
mvn
install
Artifactory Plugin
Nexus Plugin
1.0.65

46. Generating a release candidate
Build
and
Fast
Tests
Build a release candidate early
Slower
Tests Reuse, don’t rebuild
Acceptance
Tests
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
Choose an archiving
strategy UAT
TEST

47. Generating a release candidate
Build
and
Fast
Tests
Build a release candidate early
Slower
Tests Reuse, don’t rebuild
Acceptance
Tests
Code
Quality
Metrics maven
Release
Candidate
1.0.1
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Copy Artifacts Plugin Deploy
to
ProducLon
Artifactory Plugin
Nexus Plugin
... UAT
TEST

48. Automated Deployment
Build
and
Fast
Tests
Automate your deployments
Slower
Tests Reuse deployment processes
Acceptance
Tests Deploy your environments too
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

49. Automated Deployment
Build
and
Fast
Tests
Automate your deployments
Slower
Tests Reuse deployment processes
Acceptance
Tests Deploy your environments too
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
Deployer Plugin
SSH Plugin
... UAT
TEST

50. Visualizing the build process
Build
and
Fast
Tests
Slower
Tests
Acceptance
Tests
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

51. Build
and
Fast
Tests
Slower
Tests
Acceptance
Tests
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

52. Visualizing the build process
Build
and
Fast
Tests
Slower
Tests
Acceptance
Tests
Code
Quality
Metrics
Release
Candidate
Deploy
to
Test 1.0.1
PROD
Deploy
to
UAT
Deploy
to
ProducLon
UAT
TEST

53. So how do you deliver?
Rigorous testing and reporting gives
us confidence in our product...

54. So how do you deliver?
And a Jenkins build pipeline makes
automated delivery possible.

55. どうもありがとうございまし
Thank
You
John
Ferguson
Smart
Email:
john.smart@wakaleo.com
Web:
hGp://www.wakaleo.com
TwiGer:
wakaleo

56. Training and Consulting Services in
TDD, BDD and ATDD
Automated Acceptance and Regression Testing
Build Automation and Continuous Delivery
Clean Coding Practices