1. Using Selenium to Improve a Team's
Development Cycle
Selenium Conference 2012
Mike Davis - Google

2. Background

3. Background
What is a Software Engineer in Test?
"Engineer who does everything and anything to improve
productivity and quality of Google products."

4. How Can Selenium Improve a Team's
Development Cycle?
The Objectives
● Catch bugs before submission
● Provide high confidence that system is working
● Release faster & more often
● Catch bugs not likely to be caught by manual QA

5. How Can Selenium Improve a Team's
Development Cycle?
The Strategy
● Tests are written at the same time as features
● Tests are run before every check-in
● Code that breaks a test is not submitted
● Tests run at all stages of the development process

6. How Can Selenium Improve a Team's
Development Cycle?
Some Corollaries
● Tests must be run as much as possible
● Developers must be involved in writing and maintaining tests
● We need really good tests

7. What Makes Good Functional Tests?
● Fast & Stable
● Easy to run
● Easy to read
● Easy to debug
● Easy to write

8. What Makes Good Functional Tests?
Tests Must Run From a Single Command (Or IDE)
● No "follow steps in this wiki"
● No "first install this software"
● No "Add yourself to this unix group"
● No "First visit website and sign up a new user"
● No command line flags
Run this command & everything "just works"

9. What Makes Good Functional Tests?
Hermetic Tests
Hermetic:
● Self contained
● No external dependencies
Managing dependencies:
● Run them in-memory as well
● Look for in-memory implementations of data-stores
● Fake out dependencies that can't be brought up

10. What Makes Good Functional Tests?
Keep Tests Small / Limit Number of Tests
Limit the number of tests:
● Do you really need this test?
● Is this already covered by unit tests?
● Can you write a unit test for this?
Keep tests small:
● Can you make it 2 tests?

11. What Makes Good Functional Tests?
Tests Set Up All Data
public void testUserHasOnePurchase() {
AccountPage accountPage = loginPage.login("testUser1", "testPassword");
assertEquals(1, accountPage.getPurchaseCount());
}
Example Failure 1:
Exception: Expected <1> got <2>
Example Failure 2:
Exception: Login Failed

12. What Makes Good Functional Tests?
Errors Are Easy to Track Down
● Assertions have useful error messages
Exception : False line:225
VS
Exception : Expected User to Have Purchase
● Supply Screenshots of Error States
Screenshot:
Exception: Timed out locating element
"By.id = SearchButton". A screenshot has
been saved at http://screenshot/_Ghktyc

13. What Makes Good Functional Tests?
Errors Are Easy to Track Down
● Misleading exceptions should be
wrapped.
Exception: Timed out locating element "By.id = BuyButton"
Exception: Error page detected locating element "By.id = BuyButton"
● Grab server side logs
Exception: Error page detected locating element "By.id = BuyButton"
ServerSide: NullPointerException @Purchase.java line 225

14. What Makes Good Functional Tests?
Example
public class OrdersTest extends TestCase {
...
public void setUp() {
server = new LocalRunningServer();
server.initialize();
pathFinder = new PathFinder(server.getAddress());
}
public void testOrdersShowInAccount() {
User user = userCreator.createNewUser();
purchaser.makePurchaseForUser(user);
HomePage homePage = pathFinder.getHomePage();
LoginPage loginPage = homePage.navigateToLoginPage();
UserDashboard userDashboard = loginPage.loginAs(user);
OrdersPage ordersPage = userDashboard.navigateToMyOrders();
assertEquals(1, ordersPage.getNumberOfOrders());
}
}

15. What Makes Good Functional Tests?
Counter Example
Bad Better
public class OrdersPage implements PageObject { public class OrdersPage implements PageObject {
... ...
public WebElement getOrder() { public String getOrderId() {
if (!hasOrders()) { assert(hasOrders(),
createOrder(); "Expected orders but none found") ;
} return driver.findElement("order").getText();
return driver.findElement("order"); }
}
public List<String> getOrderIds() {
public List<WebElement> getOrders() { ...
return driver.findElements(By.name("order")); }
} }
}

16. Going Beyond

17. Going Beyond
Testing In-Memory Servers vs Deployed Servers
Objective:
● Run tests at all stages of development cycle
Problem:
● Priorities for Release Candidate are different
● Already have good suite of tests, don't want to
write another
Server
Local Remote
Server Server

18. Going Beyond
Profiling Tests
Browser Ajax Server
TestLoadHomePage() - Call Profile at Every Commit
Key:
3
No. of GetHomePageData
Calls
GetSpecialOffers
2
GetUserData
AddToBasket
1
History of Project

19. Summary
Using Selenium to Improve a Team's Development Cycle:
● Run tests as often as possible
● Get dev involved
● Good tests
○ Fast & Stable
○ Easy to run
○ Easy to read
○ Easy to debug
○ Easy to write
There are loads of ways to extend a good suite of tests.

20. Questions?