This talk was given in March at the CzechTest conference in Prague and at the Testing and Finance conference in London in May.

1. The Future of Testing in
Finance
Paul Gerrard
paul@gerrardconsulting.com
Twitter: @paul_gerrard
Web: gerrardconsulting.com
Download from businessstorymethod.com/
Intelligent Testing, Improvement and Assurance Slide 1

2. Thank-you
• Thank-you to Dr Richard Sykes and Susan
Windsor for allowing me to pick their brains
• This PowerPoint can be downloaded from:
gerrardconsulting.com
Intelligent Testing, Improvement and Assurance Slide 3

3. There have been some negative
waves recently
• Testing is Dead
• Death to the Testing Phase
• Testing has contributed little to quality
improvement in the last ten years
• Being a tester is a „bad thing‟
• Is testing really DEAD?
Intelligent Testing, Improvement and Assurance Slide 4

4. My crystal ball
• Changes in our business
demand a rethink of
testing
• Ten five-year predictions
• Some test lead/manager
career suggestions
• It‟s personal, partial and
fallible!
Intelligent Testing, Improvement and Assurance Slide 5

5. “You can fool some of the people all the time and
you can fool all the people some of the time but
you can’t fool all the people all the time.”
Abraham Lincoln
“If you are right, you’ll be hailed as a visionary.
If you are wrong, no one will remember”
Bill Hetzel
“How did I get myself into this?”
Me
Intelligent Testing, Improvement and Assurance Slide 6

6. Theme:
Changes in our business
Software product and
demand a re-think of
service companies and user
where testingaffected
companies are all fits in
systems projects
Intelligent Testing, Improvement and Assurance Slide 7

7. Technology, Hype and
How Innovations
Become Mainstream
Intelligent Testing, Improvement and Assurance Slide 8

8. Intelligent Testing, Improvement and Assurance Slide 9

9. Not all innovations
make it across the
“chasm”
Intelligent Testing, Improvement and Assurance Slide 10

10. The Hype Cycle
Agile, The Cloud and Mobile
computing have made it across
the Chasm
Where are YOU in the
Adoption Cycle?
Intelligent Testing, Improvement and Assurance Slide 11

11. The Cloud
Intelligent Testing, Improvement and Assurance Slide 12

12. Software as a
Service dominates
the growth of public
cloud markets
Intelligent Testing, Improvement and Assurance Slide 13

13. The industry changes its mind –
again
• Moving to Infrastructure as a Service will save
some money
• The IT Director can glory in the permanent
cost savings for a year – and then what?
• The “Business” will want to take advantage of
the flexibility that the move to the cloud
offers.
Intelligent Testing, Improvement and Assurance Slide 14

14. Massive re-architecting of systems
and companies – work in progress
• Many (most?) public websites will be replaced
by mobile apps – depends on who you talk to
• SaaS will represent 26% of all packaged
software (so what?)
– To re-architect an installed base of software and
create new systems will be a monumental effort
– The challenges of integration, security,
performance and reliability will be daunting.
http://forrester.com/rb/Research/sizing_cloud/q/id/58161/t/2
Intelligent Testing, Improvement and Assurance Slide 15

15. The Effect on Suppliers
Intelligent Testing, Improvement and Assurance Slide 16

16. Out with the old…
• “Old model” hardware suppliers
– Motivated to sell more hardware; not motivated
to utilise; requires high labour support
– Motivated to sell new licences (e.g. operating
systems); not motivated to sell SaaS; open source.
– Commercial model and company culture relies on
this mindset
• Can they change quickly enough? Or at all?
• Where are their new markets?
Intelligent Testing, Improvement and Assurance Slide 17

17. In with the new?
• New model cloud suppliers
– Motivated to sell infrastructure as a service
– Have tools in place to manage set-up and tear down with
very little manual intervention; very low labour costs
– Rely on high capacity, utilisation, open source
– Motivated to sell flexibility
• Small number of very highly skilled people
• A cheaper business model, they can reduce current
costs by 75-90% and still make a profit
• Expect prices to fall as competition hots up, but can
the oldies, being late to market, compete?
Intelligent Testing, Improvement and Assurance Slide 18

18. “Bottoms on seats” service models
• Traditional IT service suppliers - applies to all
IT, not just testing
• Operational costs are HIGH (people)
• Low technology – high human support
• No incentive to be efficient
• High utilisation equals high profits
• Moving towards …
Intelligent Testing, Improvement and Assurance Slide 19

19. Brains, not bottoms
• High technology – fewer resources
• Moore‟s law applies – smart people doing more
with less
• Companies are used to business process
outsourcing
• Development and test is becoming a commodity
supported by lots of tools
• Companies may outsource to Service Factories
• Their incentive is always to be efficient
• Niche specialist skills always in demand.
Intelligent Testing, Improvement and Assurance Slide 20

20. Impact on selecting and managing
supplier relationships?
• Even our contracts and purchasing processes
have to change
• Testing has an important role to play of
course.
Intelligent Testing, Improvement and Assurance Slide 21

21. A Methodological Shift
From staged, structured projects to
iterative and Agile and now, towards
„continuous delivery‟
Intelligent Testing, Improvement and Assurance Slide 22

22. Changes ahead
• System users and service vendors moving to
new business models based on new
architectures
• SaaS is an enabler for very rapid deployment
of new functionality onto a range of devices
• SaaS and maturing Agile approaches mean the
promise of Continuous Delivery can be
achieved.
Intelligent Testing, Improvement and Assurance Slide 23

23. And Continuous
Delivery is...
From requirement to deployment in
24 hours… every 24 hours?
Intelligent Testing, Improvement and Assurance Slide 24

24. From traditional delivery…
… to Continuous Delivery
Intelligent Testing, Improvement and Assurance Slide 25

25. The Deployment Pipeline
• Automated Unit
tests
• Automated
Acceptance Tests
• Manual User Tests
Intelligent Testing, Improvement and Assurance Slide 26

26. Structured … Agile?
We are in transition
• A re-think is in progress
• Adopting new architectures and a more streamlined
approach to development
• Requirements half-life of weeks or months
• Agile approaches are homing in on continuous delivery
• Kanban-Ichiwichi-Hokikoki-Kamikaze-Driven
Development – Whatever!
• A hybrid Agile-Structured process:
– Structured Analysis, Agile Delivery?
– That would be SAAD!
Intelligent Testing, Improvement and Assurance Slide 27

27. What does this mean for
practitioners?
• Re-distributed functional testing
• A focus on continuous test and assurance
• Service integration replaces system integration
• Non-functional skills, design and test
particularly UX, security, privacy, performance
are in demand („twas always so)
• The ability to balance competing non-
functional requirements in demand too.
Intelligent Testing, Improvement and Assurance Slide 28

28. The real squeeze on testing
• From the „low-value‟ unskilled, downstream
A Redistribution of
test role to upstream, business-savvy,
workflow-oriented, UX-aware testing
Testing and checking –its tools
specialists with new tools
– Developers absorb the
not with
–death - is inevitable.
Business analysis owns acceptance
– New focus on requirements management
– Under-skilled testers, relying on clerical skills to
get by will be squeezed out.
Intelligent Testing, Improvement and Assurance Slide 29

29. Board Level Messages
Is testing a Board-Level concern?
Intelligent Testing, Improvement and Assurance Slide 30

30. The New Testing: A Wider
Resource Orchestration
• The creation & delivery of systems with
assurance in this new environment calls for
Testing Through
new approaches to development and testing
• Assurance is required across the full life cycle,
the Lifecycle
in an increasingly virtualised environment
• This requires orchestration of people, process
& technology throughout the lifecycle
increasingly in the virtual.
Intelligent Testing, Improvement and Assurance Slide 31

31. The assurance of outcomes, not
just deliverables
• Testing assures the quality of deliverables, as ever
We Test
• But stakeholders commission systems to achieve
business outcomes
Systems, not
• Confidence in the delivery of outcomes is, for a
business, an operational and strategic strength
Just Software
• Testing should, above all, be about delivering
confidence in outcomes.
Intelligent Testing, Improvement and Assurance Slide 32

32. My Five-Year
Predictions
Intelligent Testing, Improvement and Assurance Slide 33

33. #1
50% of in-house testers
will be reassigned or
outsourced
Intelligent Testing, Improvement and Assurance Slide 34

34. #2
Business test planning
will become part of up-
front analysis and design
(where it should be)
Intelligent Testing, Improvement and Assurance Slide 35

35. #3
Specification by Example
will become the new
buzzword (includes
ATDD, BDD, TDD etc.)
Intelligent Testing, Improvement and Assurance Slide 36

36. #4
Developers will adopt
behaviour‐driven
development and new
tools
Intelligent Testing, Improvement and Assurance Slide 37

37. #5
Some system tests and
most acceptance tests
will be business model-
driven
Intelligent Testing, Improvement and Assurance Slide 38

38. #6
Business models plus
stories will increasingly
become „contractual‟
Intelligent Testing, Improvement and Assurance Slide 39

39. #7
System Tests will be
generated from stories
or will be outsourced
Intelligent Testing, Improvement and Assurance Slide 40

40. #8
Manual system test
execution will be
outsourced (and in the
cloud)
Intelligent Testing, Improvement and Assurance Slide 41

41. #9
Maybe half of all
acceptance tests will be
automated in CI…
…forever
Intelligent Testing, Improvement and Assurance Slide 42

42. #10
New tools that manage
the full requirements,
stories, scenarios,
workflows, testing
pipeline will emerge
Intelligent Testing, Improvement and Assurance Slide 43

43. Some Doors Close, but
Other Doors are
Opening
What are the new opportunities for
Test Leads and Managers?
Intelligent Testing, Improvement and Assurance Slide 44

44. #1
Upstream with
stakeholders and
analysis – providing
testing and assurance
skills to business
Intelligent Testing, Improvement and Assurance Slide 45

45. #2
Managing Business
Requirements
knowledge; analysing
business impact of
change
Intelligent Testing, Improvement and Assurance Slide 46

46. #3
Testmaster: Providing an
assurance function to
teams, projects and
business stakeholders
Intelligent Testing, Improvement and Assurance Slide 47

47. #4
Managing the
information flows
to/from the Continuous
Integration process
Intelligent Testing, Improvement and Assurance Slide 48

48. #5
Managing/monitoring
outsourced/offshore
teams who detail and
execute tests
Intelligent Testing, Improvement and Assurance Slide 49

49. Close
Intelligent Testing, Improvement and Assurance Slide 50

50. The phase after development is
REWORK, not Testing
• Managers will never name this phase correctly
• Testing must align with development, not
follow it
• Manual test phase will fade away replaced by:
– Up-front test design mostly automated
– Integrated with code change and software delivery
• Testers must embrace the change, not resist it
• Make that leap across the chasm!
Intelligent Testing, Improvement and Assurance Slide 51

51. Perhaps testing and the
role of testers can at
last mature to what
they should be?
Intelligent Testing, Improvement and Assurance Slide 52

52. Thank-You
Intelligent Testing, Improvement and Assurance Slide 53

53. The Future of Testing
Paul Gerrard
paul@gerrardconsulting.com
Twitter: @paul_gerrard
Web: gerrardconsulting.com
Download from businessstorymethod.com/
Intelligent Testing, Improvement and Assurance Slide 54