RaaS
REGRESSION AS A
SERVICE
RAAS
Self-aware
automation
Ragavan Ambighananthan
@ragsambi
Expedia
London Test Automation In Devops Meetup - July 2018
SOLVING MICRO-
SERVICES AND ONE SITE
PROBLEM USING RAAS
Part I
Part II

On-boarding a whitelist partner takes few man-weeks of testing
Slow automation development for new Partners/Partner POS
Custom feature requirements from different partners
This sometimes varies even among the same partner POSes
PROBLEM DESCRIPTION

RaaS design driven by json based meta data
Single git repo and no code duplicity
Self-aware automation design and code re-use
This RaaS framework itself is re-usable
no need to change automation code for new partners
SOLUTION(S)

DEFINITION OF RAAS 1/5
RaaS Automation Self-Aware Design
• Complex pattern of feature requirements captured as blue print
in json file
• New partner sites can be on-boarded just by adding them to json
file, without having to change any code
• Intelligent tagging and meta data allows tests to be run only if its
applicable for a POS and at an individual test case level
HOLISTIC TEST AUTOMATION
THAT IS SELF-AWARE

BIG PICTURE
RAAS API
PARTNER
AUTOMATI
ON TESTS

DEFINITION OF RAAS 2/5
Self Service
• Allows teams to create pre-configured CI/CD pipeline
• Allows users to view the blue print, update and add new
Partners
SELF-SERVICE PORTAL FOR CREATING CI/CD
PIPELINE AND MANAGING THE RAAS
BLUEPRINT

SELF SERVICE CI PIPELINE

DEFINITION OF RAAS 3/5
EWE Scalatest Framework
• Provides reusable, necessary nuts and bolts to write UI test
• Integrated with other RaaS tools
TEST FRAMEWORK WHICH PROVIDES CORE
TEST AUTOMATION FUNCTIONALITY WITH
REUSABLE COMPONENTS

DEFINITION OF RAAS 4/5
Distributed Automation
• Provides cloud based test automation infrastructure which
provides scalable and concurrent automation execution
system
ABILITY TO STORE AUTOMATION ARTIFACTS
LIKE AND MAKE THEM AVAILABLE VIA A UI
RICH DASHBOARD

DEFINITION OF RAAS 5/5
DA Dashboard
• Easy to view trend of test automation results
• Dashboard helps to debug and find the root cause faster
PROVIDES CLOUD BASED TEST
INFRASTRUCTURE WHICH PROVIDES SCALABLE
AND CONCURRENT AUTOMATION EXECUTION
SYSTEM

RAAS

ADVANTAGES OF RAAS
• When a new partner is launched, no need to read the
automation code and add tags to matching test scenarios
• Imagine doing this for 100 tests?!
• Update new Partner Point of Sale details in the json file
and use the self service to create jenkins jobs
• No need to change automation code

SOLVING “MICROSERVICES AND ONE SITE”
PROBLEM USING RAAS
Maintaining the sanity of a site powered by micro
services with RaaS
Ragavan Ambighananthan
@ragsambi
Expedia Group
London Test Automation In Devops Meetup - July 2018

Y
O
U
R
S
I
T
E
.
COM
Micro services release pattern
app1-flex-web
release time Day 1 Day 2 Day 3 Day 4 Day 5 Day 6
app2-web
app3-web
app4-web
app5-api-web
app60-web
app200-web
app250-shopping-web
app399-shopping-
web
app400-web

Problem Statement
• Release of one micro service among hundreds can
potentially impact the site
• A commit moving into production is going to be fast
• Amount of commits moving into production is going to
be high as well, so is the problem caused by them.
• Above problem multiplied by the number of micro
services

Problem Statement
• How to maintain the independence
of CI/CD of all micro service and
still maintain the sanctity of your
master site?

Solution 1:
Git submodule

Git submodule
Submodules
It often happens that while working on one project, you
need to use another project from within it. Perhaps it’s
a library that a third party developed or that you’re
developing separately and using in multiple parent
projects. A common issue arises in these scenarios:
you want to be able to treat the two projects as
separate yet still be able to use one from within the
other.

Git submodule
Disadvantages
• Different micro services can use different
test framework
• Even if they use same framework, they
can use different versions
• Even with all same, they could be run
differently like using Maven, Gradle, etc
• Hence it is not possible to run as a single
jenkins job

Relationships: Based on knowledge,
a change to one page might affect
the previous or the next page in the
flow
Dependents: Based on fact, a
change to one service-api will
affect it’s dependents
Solution 2:
RaaS and Contracts
https://your-internal-git.repo/app2-web
For design purpose, json file is
shown. Ideally the dependency tree
will be dynamically created from
some form of discovery mechanism

Solution 2 (cont.)
RaaS and Contracts
• Use a discovery service to find the
dependency of a micro-service
• In the absence of a discovery service,
use a static json file
• Map the ‘relationship’ part as well

RaaS and publish contracts
• Each app publishes a contract on how to run it’s UI tests
• This contracts includes are common job to run the app’s test, parameters
• to pass to this job like selenium hub IP,etc
• https://your-internal-git.repo/app2-web/contract.json
{
"params":{
"jobParameters":{
“env”:”test",
“test_tags”: “tier1”
“hub”: “1.1.1.1”,
“ branch”: “master”
},
“token”:"raasapp2web"
},
"name":"run automation tests on test environment",
"action":{
“name”:”name_of_your_masterjenkins”
},
“actionId”:”app2-web-common-critical-tests”,
"disabled":false,
"description":"run critical UI tests for app2-web“
}
THIS COMMON JOB CAN BE USED BY ANY
PROJECT AND ‘CONCURRENT RUN” IS ENABLED

app1-flex-web
Automating the solution
• A self service app will read the json file(or use a discovery service)
and get the git repos of ‘relationship’ and ‘dependencies’ apps
• Then it will read each app’s contract.json file from their own git repo
and create a master automation job which includes the common
jobs.
https://expweb.builds.exp-tools.net/job/app2-web-common-critical-ui-tests
https://expweb.builds.exp-tools.net/job/app1-flex-web-ui-tests
https://expweb.builds.exp-tools.net/job/app50-web-common-critical-api-tests
CI BUILD
DEPLOYMEN
T JOB
https://expweb.builds.exp-tools.net/job/app40-web-common-critical-ui-tests
TEST
ENV
Jenkins jobs
"jobParameters":{
“env”:”test",
“test_tags”: “tier1”
“hub”: “1.1.1.1”
“ branch”: “master”
}
“ branch”: “master” “ branch”: “master”
https://expweb.builds.exp-tools.net/job/app1-flex-web-master-common

app2-web
Automating the solution
https://expweb.builds.exp-tools.net/job/app2-web-common-critical-ui-tests
https://expweb.builds.exp-tools.net/job/app1-flex-web-ui-tests
https://expweb.builds.exp-tools.net/job/app5-api-common-critical-api-tests
CI BUILD
DEPLOYMEN
T JOB
https://expweb.builds.exp-tools.net/job/app40-web-common-critical-ui-tests
TEST
ENV
Jenkins jobs
“ branch”: “branch1” “ branch”: “branch1”
app1-flex-web
https://expweb.builds.exp-tools.net/job/app2-web-common-critical-ui-tests
https://expweb.builds.exp-tools.net/job/app1-flex-web-ui-tests
https://expweb.builds.exp-tools.net/job/app50-web-common-critical-api-tests
CI BUILD
DEPLOYMEN
T JOB
https://expweb.builds.exp-tools.net/job/app40-web-common-critical-ui-tests
TEST
ENV
“ branch”: “master” “ branch”: “master”
"jobParameters":{
“env”:”test",
“test_tags”: “tier1”
“hub”: “1.1.1.1”
“ branch”: “master”
}
https://expweb.builds.exp-tools.net/job/app1-flex-web-master-common
https://expweb.builds.exp-tools.net/job/app2-web-master-common

Big Picture - Self Service
https://expweb.builds.exp-tools.net/job/app2-web-common-critical-ui-tests
https://expweb.builds.exp-tools.net/job/app1-flex-web-ui-tests
https://expweb.builds.exp-tools.net/job/app50-web-common-critical-ui-tests
https://expweb.builds.exp-tools.net/job/app40-web-common-critical-ui-tests
DISCOVERY
SERVICE
SELF SERVICE
https://your-internal-git.repo/*/contract.json
reads contract.json
of all relationships and dependents
https://expweb.builds.exp-tools.net/job/app1-flex-web-master-common
and add it to the master job of the app

Big Picture - Pipeline
app2-web
CI BUILD
DEPLOYMEN
T JOB
TEST
ENV
“ branch”: “branch1” “ branch”: “branch1”
"jobParameters":{
“env”:”test",
“test_tags”: “tier1”
“hub”: “1.1.1.1”
“ branch”: “branch1”
}
https://expweb.builds.exp-tools.net/job/app2-web-common-critical-ui-tests
https://expweb.builds.exp-tools.net/job/app1-flex-web-ui-tests
https://expweb.builds.exp-tools.net/job/app50-web-common-critical-ui-tests
https://expweb.builds.exp-tools.net/job/app40-web-common-critical-ui-tests
https://expweb.builds.exp-tools.net/job/app1-flex-web-master-common
• All the above jobs run in parallel
• All the tests within the job runs in parallel pointing to the
hub “1.1.1.1”
• Each app will have its own DA hub
• The ‘*-common-critical-ui-tests’ jobs will only run the tests
tagged ‘tier1’

Questions?
https://github.com/ambirag/microservices-and-onesite-problem
https://twitter.com/ragsambi