See and hear about EVERFI's journey to generating targeted tests automatically from changing system schemas in this webinar with Applitools, CircleCI, and Cypress. Greg Sypolt, VP of Quality Engineering, and Sneha Viswalingam, Director of Quality Engineering, share the time and productivity savings achieved through this approach, and how adopting shift-left test generation has shortened the QA cycle. * See the Applitools products used, including the Ultrafast Grid, at https://applitools.info/n3o * Read and download the case study at https://applitools.info/gbi

1. HOWEVERFIMOVEDFROMNO
AUTOMATIONTOCONTINUOUS
TESTGENERATIONIN9MONTHS
GREG SNEHA

2. ABOUTUS
@GREGSYPOLT IN/GREGSYPOLT GSYPOLT.GITHUB.IO
GREG SYPOLT
VICE PRESIDENT OF QUALITY ENGINEERING
SNEHA VISWALINGAM
DIRECTOR OF QUALITY ENGINEERING
@SVISWALINGAM IN/SNEHAVISWALINGAM CODEHERSTORIES

3. •Like everything, we have to start at the base to drive positive
change early on. Understand our testing problem.
•How do we build for the future with model-based testing?
How we modeled the system to increase our quality
coverage.
•The technologies driving the transformation. Learn how we
are auto-generating Cypress.io test code and Applitools
visual assertions from Curiosity Software model-based
testing tool solution.
•It's a journey; we are always thinking about how to become
more e
ff
icient, faster, and not the bottleneck.
KEYTAKEAWAYS

4. QUALITYEVOLUTION
THE FUTURE
Targeted Generation
Cache Generated Tests
Improved Branch Strategy
Video and Audio Assertions
Innovation Never Stops
BEGINNING
It calls for change, we are
committed to answering it by
introducing model-based
testing. So we started the journey
so that we could remove manual
testing for 100+ courses with
model-based testing.
MONTH
6 MONTHS
Innovation (Test Generator CLI) and
Best Practices
HOURS
3 MONTHS
Utilizing test generation algorithms
will recreate the entire system from
a collection of models to generate
Cypress and Applitools test code
from those visual flowchart models.
WEEKS
9 MONTHS
Innovation (Model Course
Generation CLI)
52 courses are automated by
generating test code with seconds.
SECONDS

5. WHYMODELBASEDTESTING

6. SETOFPAINPOINTS
•Traditional testing methods
•Not all sections of the page tested
•Not all permutations and combinations of the course were tested
•Need to make tests re-usable
•Quick learning time for folks from non-coding background
•Front end testing is fragile
•Ambiguous test cases

7. Model-Based Testing
FUTURESOLUTION

8. WHATISMODEL-BASEDTESTING(MBT)?
• A model is a description of a system
behavior (input sequences, actions,
conditions, output, etc.)
• Reusable, only de
f
ined once as
digital twin of the system behavior
• Your quality coverage of the system
is documented
• Test scripts automatically generated
once the models are ready.
Model System
Executable
Tests
Is a partial description of
Generation Run tests

9. TECHNOLOGIESDRIVINGTHECHANGE

10. WHYMBT WASTHEBESTCHOICEFOREVERFI
OtherForms
• In a common test script, you would
write a test closure (describe , it),
with a series of
f
ixed commands and
assertions, that makeup scenarios.
• Using page object model typically
leads to duplication.
• Copy and paste.
Model-BasedTesting
• Software behavior speci
f
ied once
• Lower maintenance
• 1:1 relationship between model and web component
• Easier to develop and maintain
• Very high coverage
• Everything inside your model is validated
• Test code from test to test are similar; beforeEach / afterEach hooks and it’s a collection
of reusable functions
• Requires a shift in mindset and culture
• More up front effort and steep learning curve
• Maximizes coverage with the least number of tests. The manual calculation is less
ef
f
icient and less accurate than MBT options.

11. THETECHNOLOGIESDRIVINGALEARNINGTRANSFORMATION
TESTMODELLER
MODELGENERATION
FUNCTIONS
REUSABLEOBJECTS
USERFLOW
TESTGENERATION
CODETEMPLATE
GENERATIONALGORITHMS
TESTCODEARTIFACT
CYPRESS
CYPRESSTEMPLATE
GENERATEDTESTCODE
LOCALANDCIEXECUTION
PARALLELISM
TESTRESULTDASHBOARD
GITHUBINTEGRATION
APPLITOOLS
ULTRAFASTGRID
TESTRESULTDASHBOARD
VISUALASSERTIONS
SOURCEOFTRUTH
CROSSDEVICETESTING
RESOLUTIONVALIDATION
GITHUBINTEGRATION
STORYBOOKTESTING
SLACKINTEGRATION
GITHUBINTEGRATION
CIRCLECI
TESTINGGRID
CYPRESSINTEGRATION
CONTINUOUSTESTING
TRIGGERS

12. COMPONENTTESTGENERATION
Test Modeller Test Engine
JS
Keyboard
JS
A11y
JS
Visual
JS
Mouse
M K A V
M K A V
M K A V
Parallelism: 12
V
V
V
V
V
V
V V
V V
V V
Chrome 1024x768
Chrome 764x768
Safari 1024x768
Safari 764x768
JS
Template
Button
Accordion
Carousel
Course Modeled
Components
Modello

13. COURSETESTGENERATION
Button
Accordion
Carousel
Video
Applitools
80+ Components
Test Generation Sequences
Page 1 Page 2 Page 3 Page 4 Page 5
Page 1 Page 2 Page 3 Page 4
Test Modeller

14. CLOSERLOOKATMODELCODE
const numberOfSlides = Cypress.$('.slick-dots > li').length;
for(var i = 0; i < numberOfSlides-1; i++){
cy.get(`#hp-${i+1}`).click();
if (Cypress.env('APPLITOOLS_ENABLED')) {
cy.eyesOpen({
testName: [testName] + ` ${i}`,
});
cy.eyesCheckWindow({
target:'window',
fully: true,
});
cy.eyesClose();
}
cy.get("BUTTON[aria-label='Next']").click();
}
cy.get(`#hp-${numberOfSlides}`).click();
if (Cypress.env('APPLITOOLS_ENABLED')) {
cy.eyesOpen({
testName: [testName] + ` ${numberOfSlides}`,
});
cy.eyesCheckWindow({
target:'window',
fully: true,
});
cy.eyesClose();
}
Carousel
Button
Modeling Course Page

15. CLOSERLOOKATMODELCODE
cy.get([selector]).click({timeout:[set_timeout]});
Carousel
Button
Modeling Course Page

16. DASHBOARD(CYPRESS)
DEMO VIDEO

17. DASHBOARD(APPLITOOLS)
DEMO VIDEO

18. TAKINGACLOSERVIEWATTEST
MODELLER

19. MANUALLYGENERATECOURSEMODEL
DEMO VIDEO

20. HOWCANWEMAKETHISBETTER?
•Large set of courses to generate automation
•Course map changes
•Course automation maintenance
•Understanding all the reusable models and
f
iguring out where they are
located in the Test Modeler
•How can we shift left

21. INTRODUCTIONTOANEW
IN-HOUSESOLUTION
MODELLO
AUTO-GENERATECOURSE
MODELSFROMCOURSE
SCHEMAMAP
TESTGENERATORAPI

22. HOWEVERFISCALESCOURSE
DEVELOPMENT

23. FORGE-CONTENTMANAGEMENTSYSTEM
DEMO VIDEO

24. HOWMODELLOWORKS
ENDTOEND

25. BUILDANDTESTCOURSEWORKFLOW
COURSE ARTIFACT TEST ENGINE UI TEST VISUAL TEST
TESTING GRID
MODELLO
M odello
COURSE BUILDER

26. MODELLO
Parse and Transform to
Data Structures
Recurse through the
course structures
Course Pages iterated
to create models for
each page
Sequential Map of
Pages and Blocks

27. GENERATECOURSEMODEL
DEMO VIDEO

28. INCREASEDPRODUCTIVITYWITH
MODELLO

29. DATAPOINTSALONGOURJOURNEY
Number of
Courses
Speed of Modeling
System
Quality Coverage
Accelerated
Transformation
Shift Left
Boosted
Productivity
Collaboration
Quality of Delivery
Cost Savings

30. THEFUTUREOFTESTGENERATIONAT
EVERFI

31. THEFUTUREOFEVOLUTION
Targeted Generation Cache Generated Tests Improved Branching Strategy Explore Video and Audio Assertions

32. CREDITS,WHOMADEALLTHISHAPPEN
Other Credits
IMAGES by unsplash.com
LOGOS by
f
latiron.com
Engineering and Product Teams
dell
M
Scott Gunther
Chris Barnes
Subarna Karki
Mosab Nazli
Mark Nalbach
Jennifer Maki
James Moten
Usman Raja
Pallavi Pandey
Juan Pablo Durante
Sam Black
Indu Rajasekaran
Yoli Roca
Sneha Viswalingam
Greg Sypolt
Event Sponsors

33. @GREGSYPOLT IN/GREGSYPOLT GSYPOLT.GITHUB.IO
GREG SYPOLT
VICE PRESIDENT OF QUALITY ENGINEERING
SNEHA VISWALINGAM
DIRECTOR OF QUALITY ENGINEERING
@SVISWALINGAM IN/SNEHAVISWALINGAM CODEHERSTORIES