Watch the on demand webinar recording - https://opentestingplatform.curiositysoftware.ie/in-sprint-regression-testing-webinar Visit TestModeller.io to start a free trial and work through our free Test Modeller for Perfecto tutorial. Achieving in-sprint regression- convert continuous feedback into rigorous automated testing - September 9th, 2020 Delivering quality systems at speed demands rigorous testing before each and every release. Unfortunately, robust and reliable automated regression testing is difficult to achieve in short iterations. Too often, complex processes force testing far behind development. Teams scramble to update and execute complex test suites, leaving system logic and configurations exposed to costly bugs. Model-based test automation (MBTA) can help. With the new integration between TestModeller.io and Perfecto, you can achieve rigorous test creation, maintenance, and execution at scale. This webinar will present a practical approach to in-sprint regression testing, delivering accurately-built, fully-tested software in short iterations. You will discover how: 1. Visual modelling provides a single source of truth for fast-changing designs and continuous feedback; 2. Model-based test generation creates regression packs that focus on the logic most likely affected by system changes; 3. One click pushes the risk-based tests to Perfecto for cloud-based execution across a full spread of devices and platforms; 4. Feeding advanced test reports back into the central models closes the feedback loop, enabling continuous, in-sprint testing. Sound too good to be true? Join James Walker and Julius Mong to discover how you can get started today! About the Speakers: James Walker is Director of Technology at Curiosity Software and brings years of experience in successful software delivery. James has collaborated closely with a range of organisations to identify and resolve their QA needs and has invented several success Model-Based Testing technologies. Julius Mong has over 20 years of experience in software development, consulting, business development, pre and post-sales operations, and QA across the software, consumer product, digital media, and digital marketing industries. He specializes in helping enterprises optimize their QA strategies and make testing become of value rather than a liability by "shifting testing left.”

1. © Curiosity Software Ireland Ltd. 2020
Achieving In-Sprint Regression
Convert Continuous Feedback Into Rigorous Automated Testing
September 9th 2020
Julius Mong
Senior Solutions Engineer, Perfecto
Jmong@perforce.com
James Walker, PhD
Director, Curiosity Software Ireland
James.Walker@Curiosity.Software

2. © Curiosity Software Ireland Ltd. 2020
Achieving In-Sprint Regression
1. Fireside chat: why isn’t in-sprint regression widespread?
3. Demo: Testing continuously with model-based testing
4. Q&A and discussion
2. Why model-based test automation?

3. © Curiosity Software Ireland Ltd. 2020
Global Reach
• Customers in 80 countries
• Over 20,000 customers worldwide
• 400,000 users strong
• More than 250 of the Fortune 500
• 14 brands covering full SDLC
• 20+ offices and 11 data centers which
give us global vision and reach
• Over 800 employees in 25 countries

4. © Curiosity Software Ireland Ltd. 2020
Automation At Scale
TESTING AT
ENTERPRISE SCALE
FIND BUGS
FASTER
FIX BUGS
FASTER
SHIP STABLE CODE
FASTER
ACHIEVE AUTOMATION
GOALS QUICKLY
OPEN & INTEGRATED
AUTOMATION
TRUSTED DEVOPS
LEADER
REDUCED RISK OF
ESCAPED DEFECTS

5. © Curiosity Software Ireland Ltd. 2020
Curiosity Software Ireland
• In 2017, Curiosity launched with technologies and
expertise built on decades of experience.
• We’ve built test data solutions since 1995, when
our MD founded the first of 5 pioneering vendors.
• Today we’re your specialist vendor in
requirements modelling, test data automation,
and model-based test automation
-Our global presence-
-And trusted partners in every continent-
Curiosity locations

6. © Curiosity Software Ireland Ltd. 2020
1. What Challenges
Are You Seeing?
What’s blocking you from achieving in-sprint regression testing?

7. © Curiosity Software Ireland Ltd. 2020
What Continuous Testing looks like
CONTINUOUS

8. © Curiosity Software Ireland Ltd. 2020
What most DevOps processes look like
2-3 WEEKS 1-3 WEEKS
❌ Unstable
❌ Labor-intensive
❌ Cluttered
❌ Slow

9. © Curiosity Software Ireland Ltd. 2020
Poll: What are the barriers to in-sprint
regression testing at your organisation?

10. © Curiosity Software Ireland Ltd. 2020
Insufficient Test Coverage

11. © Curiosity Software Ireland Ltd. 2020
Insufficient Test Coverage
Are your users testing your app before you do?
Manual testing is boring and costly
What’s stopping you from automating?
Can you test real user journeys?

12. © Curiosity Software Ireland Ltd. 2020
Lab/Environment
To cloud or not to cloud? Is it a real cloud?
What about security?
Can you scale easily?
Public vs private lab
Is it the team that will support you or just a lab?

13. © Curiosity Software Ireland Ltd. 2020
Feedback and Analysis Is Too Slow
• Still crunching through test reports?
• Make quick informed decisions.
• Top failure reasons for your tests?
• False negatives / positives?
• Can you do RCA efficiently?

14. © Curiosity Software Ireland Ltd. 2020
Pre-execution blockers to in-sprint
testing
What’s going wrong pre-execution? 5 common blockers:
1. Test creation remains wholly manual (and slow!)
2. Manual test case design or scripting cannot achieve sufficient
coverage for complex systems, which are released at the risk of
bugs.
3. Test data destabilises automation and creates bottlenecks.
4. Brittle scripts need maintaining manually after every change.
5. Automation code is not easily re-usable, over-working skilled
engineers.

15. © Curiosity Software Ireland Ltd. 2020
Change is a real headache…
Change requests/user
stories flow in
A code change made to a
maze of components
Which tests are
affected? Which need
updating?
What has been
impacted by the
change? What
needs re-testing?
Test Automation Engineer
Automated test maintenance kills the ROI of automation
No formal mapping: code requirements
Code
Commit

16. © Curiosity Software Ireland Ltd. 2020
“Information hops” leak time & quality
Silo
‘Agile’, Continuous Testing, “DevOps”… Or mini-waterfalls?
Wall Information Hop Delay
No feedback
or traceability
Dev Automation EngineersQA/TestBAs
Effort: Repetitious.
“Flat” formats prevent
automation in QA & Dev.
Quality:Most bugs
originate in incomplete,
ambiguous requirements.
Effort: Re-coding after
misunderstood designs
and uncaught bugs.
Quality:Bugs are more
damaging the later they
are discovered.
Effort: Slowly create
and maintain 1000s of
test cases. Wait for data.
Quality:Manually
derived tests hit a
fraction of system logic.
Effort: Repetitious
scripting, growing
maintenance.
Quality: Data clashes,
false positives, false
negatives, low coverage.
Formulate complex requirements
and change requests in
disparate, unconnected formats.
Convert the incomplete designs
into code, trying to fix the impact
of changes on complex systems.
Unsystematically create high
volume, low coverage test cases,
mismatched with system designs.
Choose between creating new
scripts for new logic or
maintaining existing code.

17. © Curiosity Software Ireland Ltd. 2020
2. Testing continuously with
model-based testing
Convert Continuous Feedback Into Rigorous Automated Testing

18. © Curiosity Software Ireland Ltd. 2020
Model-Based Test
Automation
Recorded
tests activity
Test Cases
BPMN
diagrams
BDD
Scenarios
Business
Requirements
Application Scans
Reactive Model
of the system
Find / Make / Allocate Test Data
Run Results
Tests
New
Iteration
 Perfect sets of tests
 Fast Maintenance
 Reusable Components
 Move faster
 Deliver with confidence
Test AutomationTest Automation

19. © Curiosity Software Ireland Ltd. 2020
MBTA busts pre-execution
barriers to in-sprint regression
MBTA works as an accelerator, not a replacement. It allows you to:
• Minimize duplicate effort and avoid repetitious test scripting – engineers focus on
creating what’s needed. Cross-functional can re-use the code!
• Automate maintenance of existing code – focus on newly added/updated logic.
• No test data frustrations – “Just in time” data avoids frustrating data clashes, false
positives, and false negatives.
• Optimise tests for quality - automation finds bugs before they hit production.

20. © Curiosity Software Ireland Ltd. 2020
Create Models of
Requirements
Find / Make / Allocate Test Data
Scan web applications to
create robot resources
Export automated test
scripts
Make / Find Test data
Scripts are
automatically pushed
to source control
management system
Automation Tests
Run results
Embedded Data
Automation Execution
Continuous Delivery Tooling
Product Owner
Tester
Tester
Test execution
Perfecto handles
the rest

21. Demo
© Curiosity Software Ireland Ltd. 2020
1. Test Modeller: In-sprint test design and maintenance
2. Perfecto Analytics: Cross-platform execution, CI dashboards, & heatmaps

22. © Curiosity Software Ireland Ltd. 2020
Fast Feedback Loop
• CI Dashboard
• Heatmaps
• RCA breadcrumbs
• Unique test report URLs
• Single pane of glass for all stakeholders

23. © Curiosity Software Ireland Ltd. 2020
Breaking the barriers to In-
Sprint Regression
Barriers to In-Sprint Regression Model Based Test Automation
✘ Slow and repetitious test creation. ✓ Auto-generate tests from quick-to-build flowcharts.
✘ Low Test Coverage ✓ Apply coverage algorithms to create the smallest set of tests needed for rigorous testing.
✘ Test data creates bottlenecks and
destabilises automation.
✓ Define dynamic data functions that resolve during execution for each automated test, or
use high-speed data allocation to automatically find or make data for every possible test!
✘ Manual test maintenance. ✓ Simply update the easy-to-maintain flowchart models and auto-update tests.
✘ The central team of engineers are
over-burdened.
✓ Custom-created automation recipes are available in a flexible “low code” approach,
allowing anybody to automate their tests. Subflows are also fully re-usable.
✘ Poor device/platform coverage ✓ Cloud-based execution across a complete range of devices, platforms and systems
✘ Hard-to-simulate prod. behaviour ✓ Cloud environments simulate complex production configurations seamlessly
✘ Time lost to run result analysis ✓ Test execution automatically populates granular dashboards

24. Questions and Discussion
© Curiosity Software Ireland Ltd. 2020

25. © Curiosity Software Ireland Ltd. 2020
Carry on the conversation?
Fill out the post-webinar survey – we’d love to hear from
you!
• Visit Testmodeller.io for a free trial. Use our free Perfecto tutorial
to get started with in-sprint regression.
• We’ll email out the slides, a webinar recording and ways you
can get in touch.

26. © Curiosity Software Ireland Ltd. 2020
Thank you for listening
Email us if you have any questions or feedback, or
would like a demo of any of the technology shown
Julius Mong
Senior Solutions Engineer, Perfecto
Jmong@perforce.com
James Walker, PhD
Director, Curiosity Software Ireland
James.Walker@Curiosity.Software