In the not-too-distant past, the only viable options for testing complicated web-based applications were commercial (i.e., expensive) tools. These tools were well designed but difficult to scale without significant investment in both human capital and licensing costs. Now, a number of open-source tools are available, allowing for rich, robust, expressive testing against applications as complicated as any in the world—and they’re free. However, your savings in licensing and support fees can potentially be eclipsed by the cost of maintaining a team of developers to support these "free" tools. But as the tools progress, the industry is starting to shift toward open-source test frameworks to help manage these tools, allowing their support and maintenance to be done through the “community.” This leaves your team with only the challenge of modeling your application. Marcus Merrell presents one such framework, which allows for quick modeling and implementation of a robust, low-maintenance test suite that requires minimal Java skills.

1.
Session
Presented by:
Marcus
R
Brought to you by:
340 Corporate Way, Suite Orange Park, FL 32073
888‐2
W11
Concurrent
4/9/2014
2:00 PM
“Getting Started with Open Source
Testing Tools”
Merrell
etailMeNot, Inc.
300,
68‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Marcus Merrell
RetailMeNot, Inc
A test architect at RetailMeNot, Inc., Marcus Merrell has written UI and API test
frameworks for several products since 2001. Marcus is obsessed with code design and
holds sacred the philosophy that test frameworks should be approached, developed,
and tested just as carefully as the production software they seek to examine. The
successful demonstration of these principles has led to an intense focus on
collaboration between testers and developers, and to a deep appreciation for code
that is at once simple and sophisticated―an API for testers which is reusable,
interface-agnostic, and universal to many problem sets.

3. Getting Started
with
Open Source
Testing Tools
Marcus Merrell,
RetailMeNot, inc
@mmerrell

4. What You Need in a
Framework
Multiple browsers
Ability to build a “model”
Simple execution
CI
Command-line
Robust reporting
Extensibility
Beware of “codeless” solutions

5. Choices
Ruby (RSpec, Riot)
Python (pytest, nose)
Java (JUnit, TestNG)
PHP (PHPUnit)
… or the thing I’m going to talk about

6. Our Framework
Contains all the tools above, plus several
more
Proxy
Database interaction
Analytics
Not open source just yet
We’re looking for partners

7. Tech Stack
Maven
Spring
TestNG
Selenium
Hibernate
CQMS

8. Features
Scalability
Autoscaling Grid!
Data-driven tests
Analytics
Outclicks
Site Catalyst
Database
Validation
Test data creation

9. Hierarchy
Pages
Elements
Tests
…let’s just look at some code

10. Execution
TeamCity kicks off
Maven job, which executes
TestNG tests

11. Reporting
TeamCity
Database
TestRails
…extensible, can be used with anything

12. Statistics
500 RetailMeNot core tests
300 test for mobile site
450 internal admin tests
100 tests for Native iOS app
800 headless API tests

13. Downside
Community supported, but this can be an
upside in many cases
Can be difficult to measure costs
Can be difficult to get started
(that’s where we want to help!)
Requires higher skill level to navigate

14. Conclusion
Open source = robust, well-supported,
tight control
Scalable, particularly from a license point
of view
If you choose well-supported projects,
you’ll get what you need