How Incremental Test Automation Saves Money for Retailers

How Incremental Test
Automation helps
Retailers Save Money

©1996-2019 Aspire Systems, Inc. NETHERLANDS | POLAND | SINGAPORE | US | UK | ME | IND
Sr. Test Automation Consultant,
Aspire Systems
bhanuprakash.pitchu@aspiresys.com
Speaker
Bhanu Prakash P

About Aspire
Global technology services firm with
core DNA of Software Engineering
Specific areas of expertise around Software
Engineering, Digital Services, Testing and
Infrastructure & Application Support
International headquarters in Singapore
with presence across US, UK, Benelux,
Middle East and India
Recognized 9 consecutive times as “Best
Place to Work for” by GPW Institute
Vertical focus among Independent
Software Vendors and Retail,
Distribution & Consumer Products
2750+ employees; 170+ active
customers
CMMI Maturity Level 3, ISO 9001:2015
and ISO 27001 : 2013 certified
A Great Place to Work - 9
Glorious Wins in a Row
Quality Maturity
Level
Technology Awards
Forrester Rated Aspire one of
the Top Continuous Testing
Service Providers (2017)
UK
India
Middle East
Singapore
US
Poland
Netherlands
Mexico

1 Importance of Cost of Quality (CoQ)
2 How Test Automation Helps Optimize CoQ
3 The Need for Incremental Test Automation
4 Platforms that Facilitate Test Automation for Retailers
5 Recommended Tools and Methodologies
6 How a Retail Chain Benefited from Incremental
Test Automation – Use Case
Agenda

Importance of Quality on Cost
Cost of Quality – A financial measure of Quality Performance in an organization
Prevention Costs
Appraisal Costs
Internal Failure
Costs
External Failure
Costs
Cost of Good
Quality
Cost of Bad
Quality
Cost of Quality

Ever - Increasing Test Coverage
A Decade
ago..
A pure website-based
commerce
Now !!
Customer experience is
being delivered in
multiple view ports on
different OS platforms
What do Retailers Need Now?
 A seamless shopping experience across all the viewports
 All features and changes to be tested and certified on majority of the above
platforms

How Test
Automation Helps
Optimize CoQ

Test Automation for Optimizing CoQ
Test Automation should
be planned right from the
design phase of the
project and should fall
under Preventive Costs
Shift Left Approach will
help in keeping a control
on appraisal costs, as
majority of the test
activity costs lies in the
Appraisal
Delay in introducing Test
Automation into STLC
will lead to increased
Internal and External
Failure Costs
Test Automation allows
testers to concentrate on
validating new features
and changes to system
there by finding critical
bugs early
This leads to an early
Go-No Go decision
which leads to faster
delivery of the features

Test Automation – Do’s & Don’ts
 100% Test Automation and High expectations
on ROI
 Not planning for Test Automation Maintenance
 Automating Applications planned for Major
Redesign or Sun-setting in the near future
 High Flaky or Unstable Functional Tests
 High Critical and Revenue Generating Flows
 Stable Functional Tests
 Labor Intensive Tests
 Unit/Integration Tests
 Build Verification Tests
Factors to be
considered while
planning
Test Automation
Common
Pitfalls to be
avoided while
designing Test
Automation

The Need for Incremental
Test Automation

Incremental Test Automation – Why ?
Evolving Requirements
Inherent to the Retail Industry,
requirements keep changing based on
shopping trends – This translates to more
number of builds for QA team
Test Across Devices
Seamless shopping experience across
multiple shopping platforms is a costly affair.
Incremental and Selective Automation of
revenue generation flows across different
channels keeps the costs to minimum while
reducing internal/external Failures
Scalable Testing
Cloud Platforms enhances Incremental
Test Automation by testing across
different viewports without any added
costs
Early to Market
Quick to market leads to first
mover’s advantage. So retailer
should able to certify the changes
faster. Incremental test automation
helps in this and let QA team give
confidence & faster sign-off
Frequent Testing
Utilizing CI/CD Pipeline helps in
verifying the sanity of builds
frequently through out the day helps
developers detect conflicts in the
code and deliver fixes well before
QA team pickup the build

Incremental Test Automation – How ?
67%
11%
10%
10%
2%
iOS
Android
Macintosh
Windows
Other OS
62%
24%
14%
Safari
Chrome
Other browsers
Device/Revenue Traffic
Below charts show snapshot of traffic by OS and by browser, for a typical retailer.
This should be prioritized based on which view ports should be automated first
Budget allocation
Versatile Automation tools are available in market – both Open Source & Licensed
Based on budget planned and scope of applications being automated, appropriate Automation tools should be selected for
implementation
Open Source Licensed
Incremental Test Automation should be driven by below factors

Incremental Test Automation – How ?
Ease of Use
Ease of use (level of programming ability, installation/setup, language support) will also be a driving factor if
 there are mix of desktop and web apps
 Business users wanting to use
 Compatibility with other tools/infrastructure
Data driven
Framework
Keyword Driven
Framework
CI/CD Integration
CI/CD Integration and Incremental Test Automation go hand-in-hand, as they derive benefits from each other.
Robust and well planned CI/CD pipeline enhances Test automation benefits many fold by
 Helping DEV team to unit test their code for each code check-in and deliver a quality build to test
 QA team then uses it for verifying Regression flows and provide a faster sign-off.

Platforms that Facilitate Test
Automation for Retailers

Cloud Device Platform
Cloud-based Infrastructure solution consisting of pool of Virtual machines and Real Devices
Easy and Remote access to all
infrastructure for all teams distributed
across globe
Enables organizations to provide access
only on need basis to modern and legacy
devices so that an inventory of devices need
not be purchased and maintained on-
premises
Enhances the capability of Incremental
Automation by accelerating QA and there by
ensures faster Time to Market
Full support to most of the CI/CD Tools with
plug-n-play capability with existing setup
Intelligent Reports Dashboard along
with screenshots and playback of
execution
Virtual (Emulated/Simulated) devices as
well as Physical devices of various
configurations – OS, Storage, RAM

Digital Commerce quality automation framework (DCqaf)
E-Commerce
UAT Release
Automation Frame work
Reusable Scripts
Product Specific Scripts
Test Data
Config Files
Object Repository
Selenium
Core Engine
Version
Control
Continuous
Integration
Remote VMs or
Cloud Device Platform
HTML Reports /
Extent Reports
QA Team
Report & Failure
Analysis
On Success

DCqaf – Key Advantages
Technology Scalability
Ease of Use Test Reports
3rd Party and Backend Integrations Reusability
 Tool Stack (eCom) - Selenium Webdriver – Java
and Cucumber – Ruby Framework
 Tool Stack (mCom) – Appium Web driver with Java
and Ruby
 Tool Stack (POS) - Test Complete with Jscript
 Flexible programming language compatibility
 Multi test data execution for a test case
 Low complexity framework with ease of scripting
 Automation testing support for 3rd party e-commerce
Tools – Taxation, Fraud Management, Loyalty,
Customer Interaction Touchpoints
 Integrated with TestRail and Jenkins
 Built-in Integration with Sauce Labs
 Regression Testing support for multiple
browsers/versions
 Support for mobile devices - Web and Native App
 Multi-brand, Multi-site Support
 Test reports customization
 Easy to interpret
 Shared with stakeholders after execution
 100+ Generic methods to support most of e-
commerce Website components
 Product specific reusable scripts like for
Demandware and Magento

How DCqaf Helped a Leading Retail Chain
ChallengesAbout the Retailer
 An American retail chain specializing in
counterculture-related clothing and
accessories.
 Has more than 600 stores and online portal
for selling apparel and branded merchandise.
Also has two co-brands with online-only
presence
 Longer regression cycles due to absence of effective
automation
 High execution time of existing test automation suite
 No concrete Test automation process and no CI
implementation
 In need of Parallel execution implementation for
Automation suite
Incremental Test Automation
Solution
 Revisited existing regression suite to first eliminate test cases which are duplicate or has similar verification points
 Rationalized the test suite and consolidated it to keep it lean(reduced it by 25%) while having the same test
coverage
 Implemented DCqaf using Selenium & Ruby-Cucumber.
 With in-built reusable and platform-specific(SFCC in this case) methods and back end verification scripts, Test script
Development started with minimal setup time.
 With DCqaf’s in-built features, new co-brands were easily added to existing suite with around 60% reduction in
development time. This resulted in high ROI for new brands [ < 6 months]
 Incrementally automated the new scripts for each sprint and revenue drawing view ports were prioritized for
development
 Sauce Labs Cloud platform was setup while using Jenkins as CI Pipeline

How DCqaf Helped a Leading Retail Chain
Tools
 DCqaf with Selenium Web driver with Ruby + Cucumber Framework
 Cloud Platform: Sauce Labs
 Continuous Integration: Jenkins
Results
0
10
20
30
40
50
60 Reduction in Regression suite size by
Rationalization
Reduction in development time for newly
added co-brands
Reduction in turn around time for
Regression cycle with help of Jenkins CI &
Sauce Labs25%
60%
50%

QnA
maria.josephine@aspiresys.com

Thank You
maria.josephine@aspiresys.com

How Incremental Test Automation Saves Money for Retailers

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a How Incremental Test Automation Saves Money for Retailers

Semelhante a How Incremental Test Automation Saves Money for Retailers (20)

Mais de Aspire Systems

Mais de Aspire Systems (20)

Último

Último (20)

How Incremental Test Automation Saves Money for Retailers