Integra(on
Tes(ng
in
Enterprises
using
TaaS
Anand
Bagmar
So#ware
Quality
Evangelist
Principal
Consultant,
@BagmarAnand

Reality
–
OrganizaBons
are
spread
across
the
World

OrganizaBon
ObjecBve
Time
Quality
Money

What
is
a
pracBce
that
makes
Teams
successful?
Test
Automa(on

What
is
a
pracBce
that
makes
Teams
unsuccessful?
Test
Automa(on

Is
Test
AutomaBon
treated
as
1st
class
ciBzen
in
your
organizaBon?
• Value
• Quality
Where
is
Test
AutomaBon
placed
in
the
Food
Chain
of
SDLC?

Principles
&
PracBces
for
Building
a
Test
AutomaBon
Framework

Principles
for
building
an
Automa(on
Framework
Code
Quality
Design
PaBerns
Abstrac(on
Layers
Refactor
Pairing
Evolve
Extend

Prac(ces
for
building
an
Automa(on
Framework
Configurable
Con(nuous
Integra(on
(CI)
Screenshots,
Video
Recording
Logging
Tools
&
U(li(es
Test
Data
No
Copy-­‐Paste

Case
Study

Outlook
Outlook
(Windows)
Outlook
(Mac)
Outlook
Web
Access
(OWA)
(Browser)
Outlook
Sync
(Android)
…

How
will
you
Automate
the
TesBng
for
Outlook?

Outlook
for
Windows
• White
/
QTP
Outlook
for
Mac
• Automator
Outlook
Web
Access
(OWA)
• Selenium
/
Ruby
Outlook
Sync
for
Android
• RoboBum
/
Java

For
a
user
who
has
installed
and
configured
Outlook
on
– Windows
OS
–
desktop
version,
and,
– Android
device
–
naBve
mobile
app
– Mac
OS
–
desktop
version
– …
An
email
draQed
in
one
product
is
reflected
in
the
other
products
How
will
you
Automate
the
IntegraBon
Test
….

End-­‐2-­‐End
Integra-on
Tests
(Test
Framework
for)
Outlook
on
Windows
(Test
Framework
for)
Outlook
Sync
on
Android
Create
&
Save
as
DraE
1
Verify
DraE
4
Verify
DraE
present
2
Modify
DraE
3
Orchestrator

Can
you
use
the
same
technology
stack
for
automa(ng
the
tes(ng
of
each
variant
of
Outlook?
The
Problem

Outlook
for
Windows
• White
/
QTP
Outlook
for
Mac
• Automator
Outlook
Web
Access
(OWA)
• Selenium
/
Ruby
Outlook
Sync
for
Android
• RoboBum
/
Java

My
Original
Problem

The
Solu(on

What
is
TaaS?
“TaaS”
is
a
product
that
allows
you
to
validate
the
integra-on
across
a
variety
of
products
via
Test
Automa-on
–
the
“correct”
way

What
is
TaaS?
Agnos-c
Pla^orm
/
OS
ImplementaBon
Language

What
is
TaaS?
Contract
Decoupling
of
technology
barriers
Timeout

What
is
TaaS?
Simple
Parameters
Input
&
Output

What
is
TaaS?
Developed
in
Ruby
using
Sinatra

TaaS
is
Not
a
…
“Tool”
for
integraBon
different
products
-­‐ Lack
of
Security
-­‐ Probably
not
as
robust

TaaS
is
Not
a
…
Load
TesBng
Tool

How
does
TaaS
work?

End-­‐2-­‐End
Integra-on
Tests
Framework
TaaS
Client
Test
Framework
for
Outlook
on
Windows
TaaS
Server
Test
Framework
for
Outlook
Sync
on
Android
TaaS
Server
Create
&
Save
as
DraE
1
Verify
DraE
4
Verify
DraE
present
2
Modify
DraE
3
Orchestrator
Service
Providers

TaaS
Server
ImplementaBon

What
is
a
Contract?
Specified
in
a
simple
yml
file

Contract
DefiniBon

Contract
DefiniBon

Implement
the
Contract
• Update
tests
to
-­‐
– Handle
input
parameters
• From
Environment
variables
– Execute
the
“test”

Implement
the
Contract

Parse
Input
Parameters

Return
the
Results
• Output
parameters
– As
console
output
between
special
markers
• Console
logs
• ExcepBons
(if
any)
• All
return
values
are
in
“json”
format

Collect
Output
Parameter
Values

Return
Output
Parameter
Values

Run
TaaS
Server
• Dedicated
machine
• Provide
IP
address
/
host
name
+
port
to
potenBal
clients

TaaS
Client
ImplementaBon

Setup
TaaS
Client

Invoke
TaaS
Contract

TaaS
Technical
Summary

TaaS
Server
• Specify
contract
details
• Implement
contracts
• Return
the
results
• Run
TaaS
Server
(REST
service)
• Serve
TaaS
Client
requests
– As
separate
processes

TaaS
Client
• Collect
the
input
parameters
• Invoke
the
contract
• Consume
the
result
• ConBnue
with
test
orchestraBon

Summary
of
Features
• Contract
– Decoupling
of
technology
barriers
– Timeout
• Passing
of
input
parameters
as
Environment
variables
• Result
as
json
– Output
parameters
– Console
logs
and
errors
– ExcepBons
• Developed
in
Ruby
using
Sinatra

Why
is
this
a
Good
Idea?
• Automate
the
last-­‐mile
• No
code
duplicaBon
• ImplementaBon
of
contract
lies
with
the
framework
tesBng
that
product
– Evolves
with
product
changes
• Decoupling
of
technologies

Why
is
this
a
Good
Idea?
• Helps
in
Manual
TesBng
(setup
of
data)
• Anyone
can
use
it
• Each
product
is
tested
in
the
“best”
possible
way
• Its
–
providing
Test
as
a
Service!!!

How
can
you
use
TaaS?
• Open
source
(apache2
license)
• Available
on
github!
• Available
on
rubygems.org
!

Steps
to
Follow
• NegoBate
contract
details
• Setup
&
configure
TaaS
Server
– Implement
contracts
– Return
the
results
– Run
TaaS
Server
(REST
service)
• Configure
TaaS
Client

Does
it
really
work?
>550
downloads

Before
you
use
TaaS
…
IdenBfy
–
• Is
TaaS
the
right
thing
for
you?

• Error
codes
for
the
service
• Service
Discovery
• Security
• Create
a
Java
/
.Net
version
of
TaaS
Server
• Asynchronous
requests
How
can
You
help
TaaS?

What
TaaS
did
for
me?
• Be
creaBve
• Find
InnovaBve
soluBon
to
the
problem
• Another
open
source
contribuBon

abagmar@gmail.com
@BagmarAnand
hip://about.me/anand.bagmar
Thank
you