1. Lessons Learned in Testability
Scott McMaster
Google
Kirkland, Washington USA

2. About Me
• Software Design Engineer @ Google.
– Building high-traffic web frontends and services in Java.
– AdWords, Google Code
• Ph.D. in Computer Science, U. of Maryland.
• Formerly of Amazon.com (2 years), Lockheed Martin (2 years), Microsoft
(7 years), and some small startups.
• Frequent adjunct professor @ Seattle University, teaching software
design, architecture, and OO programming.
• Author of technical blog at http://www.scottmcmaster365.com.

3. Testing and Me
• Doing automated testing since 1995.
• Ph.D. work in test coverage and test suite maintenance.
• Champion of numerous unit, system, and performance
testing tools and techniques.
• Co-founder of WebTestingExplorer open-source automated
web testing framework (www.webtestingexplorer.org).

4. Agenda
• What is Testability?
• Testability Sins
– Statics and singletons
– Mixing Business and Presentation Logic
– Breaking the Law of Demeter
• Testability Solutions
– Removing singletons.
– Asking for Dependencies
– Dependency Injection
– Mocks and Fakes
– Refactoring to UI Patterns

5. Testability: Formal Definition
• Wikipedia: “the degree to which a software
artifact (i.e. a software system, software
module, requirements- or design document)
supports testing in a given test context.”
http://en.wikipedia.org/wiki/Software_testability

6. Some Aspects of Testability
• Controllable: We can put the software in a state to begin
testing.
• Observable: We can see things going right (or wrong).
• Isolatable: We can test classes/modules/systems apart from
others.
• Automatable: We can write or generate automated tests.
– Requires each of the previous three to some degree.

7. •
Testability: your testingPractical
Testability is a function of
More goals.
•
Definition automated tests.
Our primary goal is to write or generate
• Therefore, testability is the ease with which we can write:
– Unit tests
– System tests
– End-to-end tests

8. Testers and Testability
• At Google, test engineers:
– Help ensure that developers build testable
software.
– Provide guidance to developers on best practices
for unit and end-to-end testing.
– May participate in refactoring production code for
testability.

9. Example: Weather App

10. Weather App Architecture
Rich Browser UI (GWT)
Frontend Server (GWT RPC
servlet)
Remote Web
Service (XML-over- User Database
HTTP)

11. Original Weather App Design

12. Testability Problems?
• Can’t test without calling the cloud service.
– Slow to run, unstable.
• Can’t test any client-side components without
loading a browser or browser emulator.
– Slow to develop, slow to run, perhaps unstable.

13. Mission #1: Unit Tests for
• Problem:WeatherServiceImplto
Uses static singleton reference
GlobalWeatherService, can’t be tested in isolation.
• Solution:
– Eliminate the static singleton.
– Pass a mock or stub to the WeatherServiceImpl
constructor at test-time.

14. WeatherServiceImpl: Before
private static GlobalWeatherService service = new GlobalWeatherService();
GlobalWeatherService();
public List<String> getCitiesForCountry(String countryName) {
getCitiesForCountry(String countryName)
try {
if (countryName == null || countryName.isEmpty()) {
countryName.isEmpty())
return new ArrayList<String>();
ArrayList<String>();
}
return service.getCitiesForCountry(countryName);
service.getCitiesForCountry(countryName)
} catch (Exception e) {
throw new RuntimeException(e);
RuntimeException(e);
}
}
What if we try to test this in its current form?
1. GlobalWeatherService gets loaded at classload-time.
1. This itself could be slow or unstable depending on the implementation.
2. When we call getCititesForCountry(“China”), a remote web service call gets made.
3. This remote web service call may:
1. Fail.
2. Be really slow.
3. Not return predictable results.
� Any of these things can make our test “flaky”.

15. Proposed Solution
• First we need to get rid of the static singleton.
• Then we need something that:
– Behaves like GlobalWebService.
– Is fast and predictable.
– Can be inserted into WeatherServiceImpl at test-
time.

16. A Word About Static Methods and
• Never use them! Singletons
• They are basically global variables (and we’ve all
been taught to avoid those).
• They are hard to replace with alternative
implementations, mocks, and stubs/fakes.
– They make automated unit testing extremely difficult.

17. Scott’s Rules About Static
Methods and Singletons
1. Avoid static methods.
2. For classes that are logically “singleton”, make
them non-singleton instances and manage
them in a dependency injection container (more
on this shortly).

18. Singleton Removal
public class WeatherServiceImpl extends RemoteServiceServlet implements
WeatherService {
private final GlobalWeatherService service;
public WeatherServiceImpl(GlobalWeatherService service) {
WeatherServiceImpl(
this.service = service;
}
...
• Also, make GlobalWeatherService into an interface.
• Now we can pass in a special implementation for unit
testing.
• But we have a big problem…

19. We’ve Broken Our Service!
• The servlet container does not understand
how to create WeatherServiceImpl anymore.
– Its constructor takes a funny parameter.
• The solution?

20. Dependency Injection
• Can be a little complicated, but here is what you
need to know here:
• Accept your dependencies, don’t ask for them.
– Then your dependencies can be replaced (generally, with
simpler implementations) at test time.
– In production, your dependencies get inserted by a
dependency injection container.
• In Java, this is usually Spring or Google Guice.

21. • When properly set up, it will create your objects and pass
them to the appropriate constructors at runtime, freeing you
Dependency Injection with Google Guice
up to do other things with the constructors at test-time.
• Setting up Guice is outside the scope of this talk.
– This will get you started: http://code.google.com/p/google-
guice/wiki/Servlets

22. Fixing WeatherServiceImpl (1)
•Configure our servlet to use Guice and tell it about our objects:
public class WeatherAppModule extends AbstractModule {
@Override
protected void configure() {
bind(WeatherServiceImpl.class);
class
class);
bind(GlobalWeatherService.class).to(GlobalWeatherServiceImpl.class);
class ).to(GlobalWeatherServiceImpl.class
class).to( GlobalWeatherServiceImpl.class);
}
•When someone asks for a “GlobalWeatherService”, Guice will
}
give it an instance of GlobalWeatherServiceImpl.

23. Fixing WeatherServiceImpl (2)
• At runtime, Guice will create our servlet and the object(s) it
needs:
@Singleton
public class WeatherServiceImpl extends RemoteServiceServlet implements
WeatherService {
private final GlobalWeatherService service;
@Inject
public WeatherServiceImpl(GlobalWeatherService service) {
WeatherServiceImpl(
this.service = service;
}
...
The “@Inject” constructor parameters is how we ask Guice for
instances.

24. After Testability Refactoring #1

25. Finally! We Can Test!
• But how?
• We want to test WeatherServiceImpl in
isolation.
⇒For GlobalWeatherService, we only care about
how it interacts with WeatherServiceImpl.
⇒To create the proper interactions, a mock object
is ideal

26. • The mock object framework verifies these interactions occur as expected.
– A useful consequence of this: If appropriate, you can verify that an application
is not making more remote calls than expected.
Mock Object Testing
– Another useful consequence: Mocks make it easy to test exception handling.
• Common mocking frameworks (for Java):
– Mockito
– EasyMock
• I will use this.

27. Using Mock Objects
1. Create a mock object.
2. Set up expectations:
1. How we expect the class-under-test to call it.
2. What we want it to return.
3. “Replay” the mock.
4. Invoke the class-under-test.
5. “Verify” the mock interactions were as-expected.

28. Testing with a Mock Object
private GlobalWeatherService globalWeatherService;
globalWeatherService;
private WeatherServiceImpl weatherService;
weatherService;
@Before
public void setUp() {
setUp()
globalWeatherService = EasyMock.createMock(GlobalWeatherService.class );
EasyMock.createMock( GlobalWeatherService.class);
weatherService = new WeatherServiceImpl(globalWeatherService);
WeatherServiceImpl( globalWeatherService);
}
@Test
public void testGetCitiesForCountry_nonEmpty() throws Exception {
testGetCitiesForCountry_nonEmpty()
EasyMock.expect(globalWeatherService.getCitiesForCountry ("china"))
EasyMock.expect( globalWeatherService.getCitiesForCountry("china"))
.andReturn(ImmutableList.of("beijing", "shanghai"));
.andReturn ImmutableList.of(" beijing",
andReturn( ("beijing
EasyMock.replay(globalWeatherService);
EasyMock.replay( globalWeatherService);
List<String> cities = weatherService.getCitiesForCountry ("china");
List<String> weatherService.getCitiesForCountry("china");
assertEquals(2, cities.size());
assertEquals(2, cities.size());
assertTrue(cities.contains("beijing"));
assertTrue( cities.contains(" beijing"));
("beijing
assertTrue(cities.contains("shanghai"));
assertTrue( cities.contains("shanghai"));
EasyMock.verify(globalWeatherService);
EasyMock.verify( globalWeatherService);
}
Observe:
• How we take advantage of the new WeatherServiceImpl constructor.
• How we use the mock GlobalWeatherService.

29. • Problem: Talks to external web service, does non-trivial XML
Mission #2: Unit Tests for
processing that we want to test.
• Solution:GlobalWeatherService
– Split the remote call from the XML processing.
– Wrap external web service in an object with a known interface.
– Pass an instance to the GlobalWeatherServiceImpl constructor.
– Use dependency injection to create the real object at runtime, use a
fake at test-time.

30. After Testability Refactoring #2

31. – Contains a simplified implementation of the real thing (perhaps using static data, an in-memory
database, etc.).
– Implementation usually generated by hand.
– Often reusable across test cases and test suites if carefully designed.
• Mocks and Fakes
–
Fakes vs. Mocks
Either can often be used in a given situation.
– But some situations lend themselves to one more than the other.

32. Use a Fake, or Use a Mock?
• Problem: Setting up a mock object for
GlobalWeatherDataAccess that returns static XML is
possible, but ugly (and perhaps not very reusable).
• Idea: Create a fake implementation of
GlobalWeatherDataAccess.
– We can give the fake object the capability to return
different XML in different test circumstances.

33. Implementing the Fake Object
public class FakeGlobalWeatherDataAccess implements GlobalWeatherDataAccess {
// Try http://www.htmlescape.net/javaescape_tool.html to generate these.
// http://www.htmlescape.net
www.htmlescape.net/
private static final String CHINA_CITIES = "<?xml version="1.0" encoding="utf-8"?>n<string xml
private static final String BEIJING_WEATHER = "<?xml version="1.0" encoding="utf-8"?>n<string
&lt;Visibility&gt; greater than 7 mile(s):0&lt;/ Visibility&gt;n &lt;Temperature&gt; 39 F (4 C)&lt;/
lt;Visibility&gt; mile(s):0&lt;/Visibility&gt
Visibility&gt;n &lt;Temperature&gt
lt;Temperature&gt; C)&lt
lt;/
private static final String NO_CITIES = "<?xml version="1.0" encoding="utf-8"?>n<string xmlns=
xmlns=
@Override
@Override
public String getCitiesForCountryXml(String countryName) throws Exception {
getCitiesForCountryXml(String countryName)
if ("china".equals(countryName.toLowerCase())) {
("china".equals countryName.toLowerCase()))
china".equals(
return CHINA_CITIES;
}
return NO_CITIES;
}
@Override
@Override
public String getWeatherForCityXml(String countryName, String cityName)
getWeatherForCityXml(String countryName, cityName)
throws Exception {
return BEIJING_WEATHER;
}
}

34. Testing with a Fake Object
private GlobalWeatherServiceImpl globalWeatherService;
globalWeatherService;
private FakeGlobalWeatherDataAccess dataAccess;
dataAccess;
@Before
public void setUp() {
setUp()
dataAccess = new FakeGlobalWeatherDataAccess();
FakeGlobalWeatherDataAccess();
globalWeatherService = new GlobalWeatherServiceImpl(dataAccess);
GlobalWeatherServiceImpl(dataAccess);
}
@Test
public void testGetCitiesForCountry_nonEmpty () throws Exception {
testGetCitiesForCountry_nonEmpty()
List<String> cities = globalWeatherService.getCitiesForCountry("china");
assertEquals(2, cities.size());
assertTrue(cities.contains("beijing"));
assertTrue(cities.contains("shanghai"));
}
@Test
public void testGetCitiesForCountry_empty () throws Exception {
testGetCitiesForCountry_empty()
List<String> cities = globalWeatherService.getCitiesForCountry("nowhere");
assertTrue(cities.isEmpty());
}
The fake keeps the tests short, simple, and to-the-point!

35. • Problem: UI and business logic / service calls all mixed
Mission #3: Unit Tests for
together.
WeatherHome
– The view layer is difficult and slow to instantiate at unit test-time.
– But we need to unit test the business logic.
• Solution:
– Refactor to patterns -- Model-View-Presenter (MVP).
– Write unit tests for the Presenter using a mock or stub View.

36. Mixing Business and Presentation
@UiHandler("login")
void onLogin(ClickEvent e) {
onLogin(
weatherService.getWeatherForUser(userName.getText(),
new AsyncCallback<Weather>() {
AsyncCallback<Weather>()
@Override
public void onFailure(Throwable caught) {
onFailure(
Window.alert("oops");
}
@Override
public void onSuccess(Weather weather) {
onSuccess( weather)
if (weather != null) {
fillWeather(weather);
false);
unknownUser.setVisible(false);
} else {
true);
unknownUser.setVisible(true);
}
}
});
}
How NOT to write a UI event handler for maximum testability:
• Have tight coupling between the UI event, processing a remote service call, and
updating the UI.

37. Model-View-Presenter (MVP)
• UI pattern that separates business and
presentation logic.
• Makes the View easier to modify.
• Makes the business logic easier to test by
isolating it in the Presenter.

38. Model-View-Presenter Overview

39. Model-View-Presenter
Responsibilities
• Presenter uses the View interface to
manipulate the UI.
• View delegates UI event handling back to the
Presenter via an event bus or an interface.
• Presenter handles all service calls and
reading/updating of the Model.

40. •
Passive Viewthe View is
A particular style of MVP where
MVP
completely passive, only defining and layout and
exposing its widgets for manipulation by the
controller.
– In practice, you sometimes don’t quite get here, but this is
the goal.
• Especially if you use this style, you can skip testing
the View altogether.

41. After Testability Refactoring #3

42. Presenter Unit Test Using
EasyMock
@Test
public void testOnLogin_unknownUser() {
testOnLogin_unknownUser()
weatherService.expectGetWeatherForUser("unknown");
EasyMock.expect(weatherView.getUserName()).andReturn("unknown");
true);
weatherView.setUnknownUserVisible(true);
EasyMock.expectLastCall();
weatherView.setEventHandler(EasyMock.anyObject(WeatherViewEventHandler.class));
class
class));
EasyMock.expectLastCall();
EasyMock.replay(weatherView);
WeatherHomePresenter presenter = new WeatherHomePresenter(weatherService, weatherView);
WeatherHomePresenter( weatherService, weatherView);
presenter.onLogin();
EasyMock.verify(weatherView);
weatherService.verify();
}
This test uses a manually created mock to make handling the async callback easier.

43. Question
• Why does the View make the Presenter do
this:
;
weatherView.setUnknownUserVisible(true);
• Instead of this:
weatherView.getUnknownUser().setVisible(true)

44. Answer
weatherView.getUnknownUser().setVisible(true)
• Is hard to test because it is hard to mock:
–To mock this, we would have to mock not only the
WeatherView, but also the UnknownUser Label inside
of it.
• The above code is “talking to strangers”.

45. • Also known as the “Don’t talk to strangers” rule.
• It says:
Law of friends. Demeter
–Only have knowledge of closely collaborating objects.
–Only make calls on immediate
• Look out for long chained “get”-style calls (and don’t do them:
a.getB().getC().getD()
• Your system will be more testable (and maintainable,
because you have to rework calling objects less often).

46. • To write good unit tests, we need to be able to insert mocks and fakes
into the code.
What’s the Point?
• Some things that help us do that:
– Eliminating static methods and singletons.
– Asking for dependencies instead of creating them.
– Using design patterns that promote loose coupling, especially between
business and presentation logic.
– Obeying the Law of Demeter.
• Code that does not do these things will often have poor test coverage.

47. What Can I Do?
• Developers:
– Follow these practices!
• Testers:
– Educate your developers.
– Jump into the code and drive testability improvements.
• A good way to motivate this is to track test coverage metrics.

48. Questions?
Scott McMaster
Google
Kirkland, Washington USA
scott.d.mcmaster (at) gmail.com

49. Bonus Slides

50. Model-View-Controller (MVC)

51. Model-View-Controller (MVC)
• View directly accesses the Model and fires
events to the Controller.
• Controller performs operations on the Model.
• Controller doesn’t really know about the View
other than selecting the View to render.

52. Which to Use?
• Many web MVC frameworks exist (Struts,
Rails, ASP.NET MVC).
• But these days, we work more with MVP.

53. Manually Created Mock
public class MockWeatherServiceAsync implements WeatherServiceAsync {
private List<String> expectGetWeatherForUserCalls = Lists.newArrayList();
newArrayList();
private List<String> observeGetWeatherForUserCalls = Lists.newArrayList();
newArrayList();
// More @Overrides not shown on the slide.
@Override
public void getWeatherForUser(String userName, AsyncCallback<Weather> callback) {
getWeatherForUser(String userName, AsyncCallback<Weather>
observeGetWeatherForUserCalls.add(userName);
if ("scott".equals(userName)) {
("scott ".equals(userName
scott".equals( userName))
new
callback.onSuccess(new Weather());
} else {
callback.onSuccess(null);
null
null);
}
}
public void expectGetWeatherForUser(String userName) {
expectGetWeatherForUser( userName)
expectGetWeatherForUserCalls.add(userName);
}
public void verify() {
verify()
assertEquals(expectGetWeatherForUserCalls, observeGetWeatherForUserCalls);
expectGetWeatherForUserCalls.clear();
observeGetWeatherForUserCalls.clear();
}
}

54. 2012-12-20