End-to-End Test Automation Using TestCafé

End-to-End Test Automation using Testcafé
Francisco Mancardi

TESISQUARE® 2019© TESI SpATESISQUARE® 2019© TESI SpA 2
▪ Electronics Engineering Degree, University Buenos Aires.
▪ 30+ experience on development, linux sysadmin,
training, methods & best practices.
▪ working with CFML since 2002
▪ working on testing since 2004 (TestLink)
▪ using open source tools for SDLC since 2002
▪ Methodology Manager at TESISQUARE
COORDINATES:
francisco.mancardi@tesisquare.com
ABOUT ME
Instructor:
* Software Development
* CFML
* Methods

TESISQUARE® 2019© TESI SpATESISQUARE® 2019© TESI SpA2019 © TESI SpA
Years
of Experience
24
2016 2018
+38%
Revenues
Growth rate in
International Business
2X
Branches
The Netherland
France
Spain
Germany
4
Projects
abroad
18%
Countries
37
International
Customers
200
Investments
in innovation
⁓9%
Companies in the
collaborative network
42K
Customer
Retention
99,7%
Customers worldwide
top brands
28%
Millions
Transactions/Years
300
Growth Worldwide Coverage
Facts&Figures
20171995 2000 2005 2010 2015 201920182016 2020
€Millions
0,2
4
12
18
22
31
40
20
27
TESISQUARE® 3
More than 80 CFML Developers
organised in a Product Team
and 4 Delivery Teams

TESISQUARE® 2019© TESI SpATESISQUARE® 2019© TESI SpA
Agenda - Talk - IntoTheBox 2020
▪ Automated User (web) interface Test/Checks
▪ The Page Object Model
▪ Proposed project folder structure
▪ Example #1 MantisHUB
▪ Example #2 TESISQUARE PLATFORM
▪ Test Automation Tools - Software Selection
▪ Generic / Compliance Checks
▪ Hints / Warnings
▪ Other uses of Browser Automation Scripts
▪ Myths and legends
▪ Final Remarks
▪ References 4

Automated User (web)
interface Test/Checks

Automated User (web) interface Test/Checks
▪ the behaviour
▪ the layout
▪ the usability
▪the adherence to your corporate design
6
Executing this kind of test in a manual way is a solution that does not scale well.
With web interfaces there are multiple aspects that you probably want to test
around your UI

Page Object Model (POM)
REFERENCES:
https://martinfowler.com/bliki/PageObject.
html
https://medium.com/tech-tajawal/page-
object-model-pom-design-pattern-
f9588630800b
8
https://martinfowler.com/bliki/PageObject.html

Page Object Model (POM)
9
https://martinfowler.com/bliki/PageObject.html
▪ Using a low level API (HTML) you will write
unwieldy test case scripts.
▪ These scripts will be difficult to maintain, and will
suffer a lot when UX changes will happen.
▪ This approach will be ‘not sexy/not appealing’ for
people with a less technical mindset, then you
will not attract business analysts as potential test
script writers.

Proposed project
folder structure

Project Folder Structure
11
config
one json file for each environment to test.
i.e. Quality Assurance, Test Automation
Pre-production
intothebox.mantishub.io.json
{
"thisFileIs": "configuration for intothebox.mantishub.io",
"baseURL": "https://intothebox.mantishub.io/",
"language": "English"
}

Project Folder Structure: data
12
data/entities
json files with default values to help in entity
creation tests.
data/localization-l10n
json files with messages in different languages.
data/uploads
sample files good and bad, to be used in file
upload tests.
data/entities/user/user-dogbert.json
{
"I am": "dogbert",
"user-username": "dogbert",
"user-realname": "Dogbert Supreme Ruler of Earth",
"user-access-level": "administrator",
"user-enabled" : true,
"email-field": "dogbert@dilbert.com"
}

Project Folder Structure: lib
13
lib/helpers
js files with functions or classes that implement
behaviours like:
- login process
- bootstrap toast object model
- result grid object model
- CSRF object model
behaviours reused in the Page Object Models
lib/util
js files with functions or classes that provides
generic functions like:
* access environment variables
* access configuration present in config files
* logger

Project Folder Structure:
pages
14
here is where all the js files that implements the Page
Object Model of your pages live.

Project Folder Structure: test
15
here is where all the js files that implements the tests are
placed.
Example :
for the application TESISQUARE PLATFORM our choice
was to recreate the folder structure that mirror our Test
Management System Test Suites design.
Design choice: 1 test -> 1 js file

Project Folder Structure: test organization
16
TESISQUARE PLATFORM example

MantisHub - login flow
18
Login Step #1: enter the username
login_page.php Login Step #2: enter the password
login_password_page.php
Then

MantisHub - login flow done!
19
You are in!

MantisHub - Page Object Model
20
login_page.php pages/login_page.js
pages/login_password_page.js
my_view_page.php pages/my_view_page.js

MantisHub - Page Object Model - commons
21
User clicks “View Issues”
User clicks “Report Issue”

MantisHub - Page Object Model - commons
22
common candidate: the navBar
common candidate: the user identity
@@What about the Search Issue Box?

23
login_page.php

24
login_password_page.js

MantisHub - tests
25
Naming convention: 3 letters prefix (MantisHuB) + progressive + something descriptive
Test Script Specification
MHB-001.login.js Validate the login process is OK
checking an specific button in nav bar
MHB-002.loginFailure.nopassword.js Validate the login process failure due to no password
provided, using login helper that will return ok/ko.
MHB-003.loginFailure.checkMsgIsDisplayed.js Validate that a user feedback message is displayed,
because login failed
MHB-004.loginFailure.checkMsgText.js Validate the TEXT of the user feedback message
displayed, because login failed.
MHB-005.bannerUserName.js Validate that after successful login the user name is
displayed in the banner.
https://github.com/fmancardi/intoTheBox2020MantisHUB

The Anatomy of a test script
Navigate
Set / change
data
Get data
Validate
your findings

MantisHub - MHB-001.login.js
27
Common code, that will be present in each test case

28
Request the Page Object Models needed to implement the test
case.

29
The steps that implements the test case.
expect will exit on condition check FAILURE

MantisHub - MHB-004.loginFailure.checkMsgText.js
30

Example #2
TESISQUARE
PLATFORM

Application UX Listing example
32

<cfsavecontent variable=“local.files.listing"><cfoutput>
/**
* @filesource: <filename>
* <pagetitle>
*/
import { Selector } from 'testcafe';
const control = require('@helpers/controls.js');
const lp = require('@/pages/listing.class.js');
/* selectors */
export const searchFields = {
<fields>
}
const resultsFields = {
}
export const manager = new lp.listing();
manager.searchFieldsConfig(searchFields);
Application UX Listing example - POM Template
33

Page Object Model API
getFilters()
setFilters()
validateFilters()
search()
34
selectRowByIndex()
orderGridByColHeader()
orderGridByColIndex()
moveColToIndex()
resizeColByIndex()
hideColByHeader()
getColHeaders()
getColPositions()
getCellByColHeader()
getRowByIndex()
validateRow()
getButtons()
validateButtons()

POM - Helpers - Input Controls API
35
Input Controls
text
combo
multilevel sel
date
checkbox
numeric
hidden
set(value)
get(options)
clear()
validate(expectedValue,msg)
eqlOrMatch(actual, expected)
isReadOnly()
exists()
getFake(prefix)
API

POM - Helpers - Input Controls API
36
Script W/O POM
pure HTML API
pom.view.searchFields.ViewStatus.set(‘public’)
POM using Input Controls API

Live Example #2 Sticky Filters
STEPS A
* Filter the list view
* Modify First item
* Access the Rows TAB
* Display filters
* Pin the row number filter
* Set only the row number filter
* Close the filters
Expected Results
* Only the row number filter
is displayed
STEPS B
* Set the row number filter value
to extract zero results.
* Apply filters
Expected Results
* Only the row number filter
is displayed
* Zero Items displayed.

Test Automation Tool:
The Software Selection

Test Automation Tool Selection Approach
I want to have a tool with Recording Capability
▪ Why?
▪ To reduce time to learn needed by the script developers
▪ To provide developers and business analysts a way
to convert testing effort into a starting point for script development.
I want something not too expensive (< 1000 U$S x seat)
Multibrowser Support is important?
Side effect:
the recorder can be useful
when reporting issues,
reducing operational risks, so
making the maintainers life
easier.
39
There are too many tools!
My Requirements

Why Test Cafe ?
First approach as usual: Try something Open Source
▪ 3 candidates
▪ Selenium
▪ TestCafe
▪ Cypress
Does a recorder exist?
▪ Selenium - yes, Katalon Recorder Browser extension POC done / Katalon Studio (ECLIPSE)
▪ TestCafe - yes, TestCafe Studio (commercial, not expensive IMHO)
▪ Cypress - no
Multibrowser Support?
▪ Selenium - if the WebDriver is available
▪ TestCafe - yes
▪ Cypress - not now - https://github.com/cypress-io/cypress/issues/310
40
Pay attention: final decision is a matter of ‘feeling’ with the tool

Other alternative - CodeceptJS
can it provide you more freedom?
41
Helpers: TestCafe,Protractor,Puppeteer,Nigthmare,WebDriver

gauge/taiko is worth mentioning
42
Limited Browser support: Chromium
Great features
* Read Evaluate Print (REPL)
* Request/Response stubbing and mocking
*Proximity selectors

Generic /
Compliance
Checks
(no application
logic needed)

Generic / Compliance / Blind
UX translations
▪ crawl the application checking the labels. You specify for each
HTML ID the localised string
in different languages
UX Compliance
▪ check that feedback messages of a kind have same layout
▪ check icon
▪ check label
▪ check message structure
44
Check all menus
then all buttons.
Report findings

UX Compliance check example
45

Another UX check example
46

Page content (TABS) driven by configuration
47
Master data management
has several
TABS that changes according
* entity type
* system configuration
* user role

Page content (TABS) driven by configuration
48
User Roles 2
Configuration options 7
Entities 2
Tabs 12
Lot of combinations
Error prone
WATT
What A Tedious Test!!
JSON file with
configuration
to drive
the checks
James Watt

Hint
Generate HTML with test automation in mind
50
In order to automate you will need to be able to access
in a simple way different elements in the User Interface
data-qa-selector attribute

Warning
JS / JQuery Widgets need special attention
51
The use of widgets like Select2 JQuery plugin can created
some minor issues when creating the automation.
The ID can have a dynamic/random components

Hint / Warning
The script has to mimic human behaviour
52
Parallelism (promise.all) can break the application
usage flow.
The abstraction to manage the input fields need
to be designed with attribute to specific fill-in order.

Hint
Move / Shake / resize
53
Sometimes to make that some logic present in the page triggers,
the page resizing is needed. reduce / increase
Bootstrap PopOver issue
Scroll element into view
Sometimes is needed -> testcafè hover()

Warning
You will need time to analyse the results
54
In the same way that if you active Fusion Reactor you need to allocate time to
analyse the logs, if you add test scripts to your CI system, you need to
allocate time to analyse the test scripts failures.

Other uses for
Browser
Automation
Scripts

Reach a milestone
56
Taiko is great to do this (also puppeteer)
1) Login ( user + password + click button)
2) open the top level menu option
3) open the first level menu option
4) finally access the feature
Only now you are ready to test your development
How many minutes do you waste in these 4 steps
during the day?
https://github.com/fmancardi/intoTheBox2020MantisHUB/other/mantisHubAccessViewIssues.js

Issue reporting (by developers)
57
Developers will be (theoretically) more comfortable
writing a small script to describe how to reach the point
where the issue can be reproduced.
Developers can be the Tom Tom Go for the
Application Maintenance people
Advantages
* reduce misunderstanding
* reduce errors
* increases resolution velocity

E2E Test Automation myths and legends
Get a tool with the test design recorder feature, then
say good bye to the script writing task
(and the script developers).
Facts/Drawbacks
* Play back issues when you do not use Plain Vanilla HTML (jQuery Plugins)
* Dynamic/random data.
* You can not use Page Object Model with the recorder.
59
Anyone can automate the tests without
having written test specifications
▪ how to test the unknown?
▪ very low value tests

Final Remarks
Test Automation Development IS SOFTWARE DEVELOPMENT
Test Automation will reduce the regression testing time
Test Automation provides scalability
- multiple environments (ACF2016/18/Lucee)
- multiple browsers
- intensive regression testing
Test Automation can not be a SIDE PROJECT

References
The Practical Test Pyramid
https://martinfowler.com/articles/practical-test-pyramid.html
Page Object Model (POM) | Design Pattern
https://medium.com/tech-tajawal/page-object-model-pom-design-pattern-f9588630800b
Testing and Checking Refined
https://www.satisfice.com/blog/archives/856
Testing Automation University
https://testautomationu.applitools.com/instructors/angie_jones.html
https://testautomationu.applitools.com/setting-a-foundation-for-successful-test-automation/

Copyright© 2019 TESI SpA. The information contained herein is proprietary and confidential; its disclosure would be extremely useful to competitors that offer similar services.
The material in this document includes a description of concepts result of a significant research and development effort. Therefore TESISQUARE® prohibits the use or disclosure of either full or partial information contained
in this document, including attachments, for purposes different from the offer evaluation aimed at the supply assignment.
THANK YOU T +39 02 89202682
www.tesisquare.com
marketing@tesisquare.com
TESI S.p.A.
Via Mendicità Istruita, 24
12042 Bra (Cuneo)
Headquarters
Via Savigliano, 48
12062 Roreto di Cherasco (Cuneo)
Branches
Milan, Rome, Turin, Padua, Genoa, Bologna
TESI International
Milan, Amsterdam, Paris, Barcelona, Munich

End-to-End Test Automation Using TestCafé

End-to-End Test Automation Using TestCafé

Recomendados

Mais conteúdo relacionado

Similar a End-to-End Test Automation Using TestCafé

Similar a End-to-End Test Automation Using TestCafé(20)

Mais de Ortus Solutions, Corp

Mais de Ortus Solutions, Corp(20)

Último

Último(20)

End-to-End Test Automation Using TestCafé

Notas do Editor