Typically in organizations, there are multiple projects / products. These products may be of implemented using tech-stacks over many years. Yet - they interact with each other in some way. To manage the complexity around Test Automation, many organizations prefer to have a common Test Automation solution across these products in an effort to build, standardize and maintain the framework. However, this is not a good idea! With this approach one potentially ends up having to compromise on the quality of automation that can be done for each product, limited by the toolset. The better approach would be to use the tools and technologies that are "right" for each product. This does have other disadvantages, but you would ensure each product is well tested! The only missing piece that remains is that these different products talk with each other. You need to test the integration between them in an automated way to verify all is well. "TaaS" is an open-source product solution that allows you do achieve the "correct" way of doing integration testing across a variety of products via Test Automation. Example: For one set of products, Selenium-based toolset may be the right choice, where as for legacy reasons, QTP may be used for some other product. With TaaS - you will be able to automate the Integration Testing between these products, by re-using the tests already implemented in the individual product suites.

1. INTEGRATION TESTING
IN ENTERPRISES USING
TAAS
Anand Bagmar
Test Practice Lead

2. @BagmarAnand
about.me/anand.bagmar
ABOUT ME

3. Money / Value
Time
Quality
ORGANIZATION OBJECTIVE

4. DISTRIBUTION / FRAGMENTATION

5. AUDIENCE SURVEY
How many of you are part of a large
organization?

6. AUDIENCE SURVEY
What are the average number of
products / projects in your
organization’s portfolio?
… large organization?

7. AUDIENCE SURVEY
What is the typical technology stack of
these products?
… large organization?
… number of products and projects?

8. AUDIENCE SURVEY
How many of you are part of merged /
acquired companies?
Do you have the same technology across
the “new” products?
… large organization?
… number of products and projects?
… same technology stack?

9. How many of you work in distributed
teams?
… large organization?
… number of products and projects?
… same technology stack?
… merged companies – same technology stack?
AUDIENCE SURVEY

10. Have you heard of a
Common Test Automation framework?
Good? Bad?
Easy? Difficult?
… large organization?
… number of products and projects?
… same technology stack?
… merged companies – same technology stack?
… distributed teams?
AUDIENCE SURVEY

11. If a product technology stack cannot be
unified in an organization, then WHY
FORCE the Test Automation technology
stack to be the same across the
organization?

12. Case Study

13. Outlook
Outlook
(Windows)
Outlook (Mac)
Outlook Web
Access (OWA)
(Browser)
Outlook Sync
(Android)
…

14. How will you
Automate the
Testing for Outlook?

15. Outlook for
Windows
• White / QTP
Outlook for
Mac
• Automator
Outlook
Web Access
(OWA)
• Selenium /
Ruby
Outlook
Sync for
Android
• Appium /
Espresso

16. How will you
Automate the
Integration Tests for
Outlook?

17. For a user who has installed and configured
Outlook on
• Windows OS – desktop version, and,
• Android device – native mobile app
• Mac OS – desktop version
• …
An email drafted in one product is
reflected in the other products

18. End-­‐2-­‐End
Integra-on
Tests
(Test
Framework
for)
Outlook
on
Windows
(Test
Framework
for)
Outlook
Sync
on
Android
Create & Save
as Draft
1
Verify Draft4
Verify Draft
present2
Modify Draft
3
Orchestrator

19. Can you use the same technology
stack for automating the testing
of each variant of Outlook?
THE PROBLEM

20. Outlook for
Windows
• White / QTP
Outlook for
Mac
• Automator
Outlook
Web Access
(OWA)
• Selenium /
Ruby
Outlook
Sync for
Android
• Appium /
Espresso

21. MY ORIGINAL PROBLEM

22. THE SOLUTION

23. How does TaaS
work?

24. End-2-End
Integration
Tests
Framework
TaaS
Client
Test
Framework
for
Outlook on
Windows
TaaS
Server
Test
Framework
for Outlook
Sync on
Android
TaaS
Server
Create & Save
as Draft
1
Verify Draft4
Verify Draft
present
2
Modify Draft
3
Orchestrator
Service Providers

25. TaaS Server
Implementation

26. WHAT IS A CONTRACT?
Specified in a simple yml file

27. CONTRACT DEFINITION

28. CONTRACT DEFINITION

29. IMPLEMENT THE CONTRACT
¨ Update tests to -
¨ Handle input parameters
¨ From Environment variables
¨ Execute the “test”

30. IMPLEMENT THE CONTRACT

31. PARSE INPUT PARAMETERS

32. RETURN THE RESULTS
¨ Output parameters
¨ As console output between special markers
¨ Console logs
¨ Exceptions (if any)
¨ All return values are in “json” format

33. COLLECT OUTPUT PARAMETER VALUES

34. RETURN OUTPUT PARAMETER VALUES

35. RUN TAAS SERVER
¨ Dedicated machine
¨ Provide IP address / host name + port to potential
clients

36. TaaS Client
Implementation

37. SETUP TAAS CLIENT

38. INVOKE TAAS CONTRACT

39. What is TaaS?

40. WHAT IS TAAS?
“TaaS” is a product that allows you to validate
the integration across a variety of products via
Test Automation – the “correct” way

41. WHAT IS TAAS?
¨ Agnostic
¨ Platform
¨ OS
¨ Implementation language
¨ Developed in Ruby using Sinatra

42. WHAT IS TAAS?
¨ Contract
¨ Simple parameters
¨ Input & Output
¨ Timeout
¨ Decouples the technology barriers

43. WHAT IS TAAS?
¨ Result as json
¨ Output parameters
¨ Console logs & errors
¨ Exceptions

44. TaaS is NOT a …

45. TAAS IS NOT A ….
¨ Tool for integrating different products
¨ Lack of security
¨ Probably not as robust
¨ Load Testing Tool

46. TaaS Technical
Summary

47. TAAS SERVER
¨ Specify contract details
¨ Implement contracts
¨ Return the results
¨ Run TaaS Server (REST service)
¨ Serve TaaS Client requests
¨ As separate processes

48. TAAS CLIENT
¨ Collect the input parameters
¨ Invoke the contract
¨ Consume the result
¨ Continue with test orchestration

49. WHY IS THIS A GOOD IDEA?
¨ Automate the last-mile
¨ No code duplication
¨ Implementation of contract lies with the framework
testing that product
¨ Evolves with product changes
¨ Decoupling of technologies

50. WHY IS THIS A GOOD IDEA?
¨ Helps in Manual Testing (setup of data)
¨ Anyone can use it
¨ Each product is tested in the “best” possible way
¨ Its – providing Test as a Service!!!

51. HOW CAN YOU USE TAAS?
¨ Open source (apache2 license)
¨ Available on github!
¨ Available on rubygems.org !
¨ Read about it from my blog

52. STEPS TO FOLLOW
¨ Negotiate contract details
¨ Setup & configure TaaS Server
¨ Implement contracts
¨ Return the results
¨ Run TaaS Server (REST service)
¨ Configure TaaS Client

53. DOES IT REALLY WORK?
>2500 downloads

54. BEFORE YOU USE TAAS …
Identify –
¨ Is TaaS the right thing for you?

55. HOW CAN YOU HELP?
¨ Error codes for the service
¨ Service Discovery
¨ Security
¨ Create a Java / .Net version of TaaS Server
¨ Asynchronous requests

56. WHAT TAAS DID FOR ME?
¨ Be creative
¨ Find Innovative solution to the problem
¨ Another open source contribution

57. abagmar@gmail.com
Anand.Bagmar@thoughtworks.com
@BagmarAnand
about.me/anand.bagmar
THANK YOU