1. @iordanis_g @geeky_android
#shazam_testing
KEEPING 100+ MILLION USERS HAPPY
HOW WE TEST SHAZAM ON ANDROID
Iordanis Giannakakis
@iordanis_g
Savvas Dalkitsis
@geeky_android

2. @iordanis_g @geeky_android
#shazam_testing
How Shazam works

3. @iordanis_g @geeky_android
#shazam_testing
100+ million users

4. @iordanis_g @geeky_android
#shazam_testing
Happy users
Even developers?

5. @iordanis_g @geeky_android
#shazam_testing
Android testing

6. @iordanis_g @geeky_android
#shazam_testing
Faster release cycles

7. @iordanis_g @geeky_android
#shazam_testing
Better code

8. @iordanis_g @geeky_android
#shazam_testing
Cheaper
http://testdroid.com/testdroid/5851

9. @iordanis_g @geeky_android
#shazam_testing
Easy

10. @iordanis_g @geeky_android
#shazam_testing
A user walks into a bar

11. @iordanis_g @geeky_android
#shazam_testing
Doing it the test driven way
Write UI
test
Implement
Refactor
Write unit
test
BDD
TDD

12. @iordanis_g @geeky_android
#shazam_testing
“To TDD or not to TDD?
...That is not the question”
Seb Rose
http://claysnow.co.uk/to-tdd-or-not-to-tdd/

13. @iordanis_g @geeky_android
#shazam_testing
Test first
• Generally easier
• Eliminates external dependencies
• Easily repeatable

14. @iordanis_g @geeky_android
#shazam_testing
The Acceptance tests cycle
Write UI
test
Implement
Refactor
Write unit
test
BDD
TDD

15. @iordanis_g @geeky_android
#shazam_testing
Acceptance criteria
Given : arrange
When : act
Then : assert

16. @iordanis_g @geeky_android
#shazam_testing
Example
Given a user is near a music venue
And the server always returns a known result
When the user Shazams
Then the user can check-in their discovery

17. @iordanis_g @geeky_android
#shazam_testing
given(user).isNear(lexington());
given(server).returns(lustForLife());
when(user).shazams();
then(user).canCheckIn(lustForLife(), lexington());
gwen
https://github.com/shazam/gwen

18. @iordanis_g @geeky_android
#shazam_testing
Acceptance test technologies
• Instrumentation
• JUnit 3
• Robotium & Espresso
• gwen
• HamMock Server

19. @iordanis_g @geeky_android
#shazam_testing
The Unit tests cycle
Write UI
test
Implement
Refactor
Write unit
test
BDD
TDD

20. @iordanis_g @geeky_android
#shazam_testing
What we don’t test
Activities & Fragments
https://github.com/xxv/android-lifecycle

21. @iordanis_g @geeky_android
#shazam_testing
What we don’t test

22. @iordanis_g @geeky_android
#shazam_testing
What we do unit test
• Presentation logic
• Business logic
• That’s all!

23. @iordanis_g @geeky_android
#shazam_testing
The MVP pattern
Presenter
Model View

24. @iordanis_g @geeky_android
#shazam_testing
The MVP pattern for Android
Presenter
Model
View
start
stop
Android

25. @iordanis_g @geeky_android
#shazam_testing
The MVP pattern
• Makes presentation logic testable
• No need to test the “dummy” view
• Avoid Android dependencies

26. @iordanis_g @geeky_android
#shazam_testing
Dependency Injection, Yourself (DIY)
• Break hardcoded dependencies
• Behaviour vs Implementation
• Swappable dependencies for test and production
runtimes

27. @iordanis_g @geeky_android
#shazam_testing
Hardcoded dependencies
Client
Feature XFeature X

28. @iordanis_g @geeky_android
#shazam_testing
Dependency Injection
Client
Injector
Feature X
Interface

29. @iordanis_g @geeky_android
#shazam_testing
public
interface
VenueRetriever
{
void
findClosestVenue(VenueFoundCallback
callback);
}
public
class
NetworkVenueRetriever
implements
VenueRetriever
{
public
void
findClosestVenue(VenueFoundCallback
callback)
{
//
Some
slow
networking
}
}
public
class
LocalVenueRetriever
implements
VenueRetriever
{
public
void
findClosestVenue(VenueFoundCallback
callback)
{
//
DB
look-­‐up
/
caching
layer,
perhaps?
}
}
Model

30. @iordanis_g @geeky_android
#shazam_testing
public
class
ResultActivity
extends
Activity
implements
ResultView
{
private
final
VenueRetriever
venueRetriever;
private
ResultPresenter
resultPresenter;
public
ResultActivity()
{
venueRetriever
=
venueRetriever();
}
public
void
onCreate(Bundle
savedInstanceState)
{
//
TODO:
Setup
layouts
&
views
Result
result
=
resultToDisplay();
resultPresenter
=
new
ResultPresenter(this,
venueRetriever,
result);
}
public
void
onStart()
{
resultPresenter.startPresenting();
}
}
Activity

31. @iordanis_g @geeky_android
#shazam_testing
public
class
ResultPresenter
{
public
ResultPresenter(ResultView
resultView,
VenueRetriever
venueRetriever,
Result
result)
{
this.resultView
=
resultView;
this.venueRetriever
=
venueRetriever;
this.result
=
result;
}
public
void
startPresenting()
{
resultView.showResult(result);
venueRetriever.findClosestVenue(new
VenueFoundCallback()
{
public
void
venueFound(Venue
venue)
{
resultView.showCheckInPrompt(venue);
}
});
}
}
Presenter

32. @iordanis_g @geeky_android
#shazam_testing
public
interface
ResultView
{
void
showResult(Result
track);
void
showCheckInPrompt(Venue
venue);
}
View

33. @iordanis_g @geeky_android
#shazam_testing
public
class
ResultActivity
extends
Activity
implements
ResultView
{
public
void
showResult(Result
result)
{
//TODO
show
the
result
screen
&
bind
result
data
}
public
void
showCheckInPrompt(Venue
venue)
{
//TODO
bind
the
venue
with
check-­‐in
prompt
view
}
}
Activity

34. @iordanis_g @geeky_android
#shazam_testing
Unit tests technologies
• JUnit 4
• Robolectric
java.lang.RuntimeException: Stub!
• Hamcrest
• JMock

35. @iordanis_g @geeky_android
#shazam_testing
Test execution

36. @iordanis_g @geeky_android
#shazam_testing
Continuous Integration

37. @iordanis_g @geeky_android
#shazam_testing
Speed (Hint: slow)

38. @iordanis_g @geeky_android
#shazam_testing
Usual execution
Test Suite

39. @iordanis_g @geeky_android
#shazam_testing
Spoon
http://square.github.io/spoon

40. @iordanis_g @geeky_android
#shazam_testing
Fork
Test Suite

41. @iordanis_g @geeky_android
#shazam_testing
Fork
• Inspired by Spoon, but faster
• Infinitely scalable
• Current setup 1 test / 2 seconds
• Pooled execution

42. @iordanis_g @geeky_android
#shazam_testing
Flakiness monitor

43. @iordanis_g @geeky_android
#shazam_testing
ADB Remote
https://github.com/sleekweasel/CgiAdbRemote

44. @iordanis_g @geeky_android
#shazam_testing
If all else fails…

45. @iordanis_g @geeky_android
#shazam_testing
If all else fails…

46. @iordanis_g @geeky_android
#shazam_testing
If all else fails…

47. @YourTwitterHandle
#DVXFR14{session hashtag}
@iordanis_g @geeky_android
#shazam_testing
• Testing is easier than you may think
• Practice, practice, practice
• Toolset is limited
• Ship it!

48. @YourTwitterHandle
#DVXFR14{session hashtag}
@iordanis_g @geeky_android
#shazam_testing
Hiring...