Brief explanation on mobile applications, specific mobile applications testing considerations. Suggestions on when to do automation for your project, and share some commonly used mobile application automation test frameworks and tools.

1. Automating
Mobile
Applications
April
Luk
Mobile
Applica0ons
QA
Developer,
Decoder
april@decoderhq.com

2. Agenda
• Tes0ng
Mobile
Applica0ons
considera0ons
• When
to
do
automa0on?
• Common
Mobile
Applica0on
Test
Frameworks
/
Tools

3. What
is
Mobile
Applications
• A
mobile
app
is
soFware
that
runs
on
a
handheld
device
(phone,
tablet,
iPod
…
etc)
that
can
connect
to
Wi-­‐Fi
or
wireless
carrier
networks
• Na0ve
app
• Na0ve
to
one
type
of
mobile
opera0ng
system
with
the
plaOorm
SDK,
tools
and
languages,
usually
download
from
app
stores
such
as
Apple
app
store,
or
Android
Market.
App
is
stored
on
the
users
mobile
device)
• Can
interface
with
device’s
na0ve
features,
info
and
hardware
(camera,
address
book,
etc..
)
• Mobile
Web
app
• Programs
run
within
a
mobile
device’s
web
browser,
the
app
resides
on
server
and
is
accessed
via
the
internet
(i.e.
they
are
not
separate
programs
that
get
stored
on
the
users
mobile
device)
• Hybrid
• Like
na0ve
apps,
run
on
the
device
and
are
wriTen
with
web
technologies.
Runs
insides
a
na0ve
container,
and
leverage
the
device’s
browser
engine
(but
not
the
browser)
to
render
HTML
and
process
the
JavaScript
locally

4. Testing
Mobile
Applications
considerations
• Hardware
• Phones
vs.
Tablets,
manufacturers,
features
(camera,
phone)
• SoFware
• OSs,
browsers,
app
updates,
back
grounding,
analy0cs,
gestures
• User
Interface
• Keyboard,
no0fica0ons,
redirec0ons,
social
media
• Opera0ons
• Connec0vity,
Wireless
vs
Wi-­‐Fi,
Online
vs
Offline,
interrup0ons
• Data
• Synchroniza0ons,
social
media
dependencies,
content
pushing

6. When
to
do
Automation?
• Team
understands
the
of
automa0on
• Improve
product
quality
and
team
produc0vity
• Focus
on
regression
and
not
feature
tests,
keep
team
mo0vated
• Don’t
bother
if
…
• Team
believes
we
can
automate
tes0ng
so
that
we
can
reduce
QA
staff
• Team
jus0fies
the
value
of
automa0on
test
based
solely
on
how
many
defects
it
catches
• Weak
communica0on
between
QA
and
Dev
team

7. When
to
do
Automation?
• Team
understand
the
of
automa0on
• Seeking
required
skill
set
• Time
to
evaluate
automa0on
feasibility
• Effort
to
build
and
maintain
automa0on
tests
• Impact
to
project
development
process
• Don’t
bother
if
…
• QA
has
no
technical
background,
or
not
mo0vated
to
implement
automa0on
• Team
aims
to
automate
everything
• QA
is
expected
to
automate
tests
aFer
manual
tes0ng
is
complete
(i.e.
automa0on
becomes
a
separate
project)
• Team
engages
QA
too
late
in
development
cycle

8. Common
Native
Mobile
Application
Test
Frameworks
/
Tools
• Mobile
applica0ons
automa0on
can
include
both
backend
and
UI-­‐client
tests
• Backend
automa0ons
• Expected
API
requests/responses
• Database
interac0ons
• UI-­‐client
automa0ons
• Gestures
(touches,
swipes,
long-­‐pressed,
pull
…
etc)
• Content
display
(table,
list,
webviews,
alerts,
no0fica0ons
…
etc)
• Executed
on
simulators/emulators
vs.
devices

9. Common
Native
Mobile
App
Test
Frameworks
-­‐
Cucumber
• Cucumber
• Ruby
based
BDD
test
tool,
user
describes
system
features,
iden0fy
test
scenarios
and
create
test
steps
in
plain
English
with
some
specific
keywords
(Given-­‐When-­‐Then)
• Step
defini0ons
are
wriTen
in
Ruby,
explaining
how
test
steps
are
executed
• Some
commonly
used
ruby
gems
for
backend
automa0on
• Json_spec,
rexml
• Rest-­‐client,
net/hTp,
curb
• Mongo,
bson_ext,
postgres

10. Common
Native
Mobile
App
Test
Frameworks
-­‐
iOS
• UIAutoma0on
• Apple’s
solu0on
for
automated
tes0ng.
It
runs
tests
wriTen
in
Javascript
through
the
Instruments
applica0on
that
comes
with
SDK
• UISpec
• Test
framework
that
drives
the
iOS
UI
by
using
UIQuery
to
traverse
the
view
hierarchy.
Users
create
tests
using
UIScript
/
Objec0ve-­‐C
• KIF
• Test
framework
that
links
directly
to
iOS
app,
test
steps
traverse
the
view
stack
using
UIView
addi0ons.
User
creates
tests
using
Objec0ve-­‐C

11. Common
Native
Mobile
App
Test
Frameworks
-­‐
iOS
• Frank
• 2
key
components:
(1)
Frank
server
compiled
into
the
iOS
app,
(2)
Frank-­‐cucumber
driver
from
within
test
scripts
• User
creates
tests
using
cucumber
and
Ruby
• Originally
uses
UISpec
to
perform
view
selec0on
(UIQuery)
and
to
simulate
interac0ons
(touches,
gestures,
etc)
• Current
version
updated
to
use
a
query
library
called
Shelley
for
view
selec0on
and
KIF’s
UIView
addi0ons
to
simulate
interac0ons
• Includes
a
live
introspec0on
tool
called
Symbiote,
enter
a
view
selector
and
it
will
highlight
all
the
views
it
would
select

12. Common
Native
Mobile
App
Test
Frameworks
-­‐
Android
• android.test
Package
• Test
framework
comes
with
Android
SDK,
use
Ac0vityInstrumenta0onTestCase2
class
to
facilitate
func0onal
tes0ng
of
a
single
ac0vity
• Test
cases
crea0on
becomes
complicated
with
an
app
includes
mul0ple
ac0vi0es,
self-­‐defined
intents,
views
with
scrollable
lists
…
etc
• Robo0um
• Test
framework
that
extends
the
Ac0vityInstrumenta0onTestCase2
class,
simplify
android.test
framework
interface
• Test
cases
are
more
declara0ve
and
less
procedural
• Full
support
for
Ac0vi0es,
Dialogs,
Toasts,
Menus,
and
Context
Menus

13. Common
Native
Mobile
App
Test
Frameworks
-­‐
Android
• Calabash
• Test
framework
that
uses
Robo0um
as
a
library,
Instrumenta0on
test
server
installed
on
the
app
and
cucumber
driver
from
within
test
scripts
• Robolectric
• More
focused
on
Unit
Test
• Tests
are
executed
on
a
desktop
worksta0ons/server
JVM
(avoid
indexing,
packaging,
deploying
and
running
app
on
emulator/device

14. References
to
Resources
Mobile
Tes0ng
Mind
Map:
hTp://www.ministryoFes0ng.com/2012/05/mobile-­‐tes0ng-­‐course-­‐
pictures-­‐and-­‐a-­‐mindmap/
Cucumber:
hTps://github.com/cucumber/cucumber/wiki
UISpec:
hTp://code.google.com/p/uispec/
KIF:
hTps://github.com/square/KIF
Frank:
hTps://github.com/moredip/Frank
Robo0um:
hTp://code.google.com/p/robo0um/
Calabash:
hTps://github.com/calabash/calabash-­‐android

15. Thank
you!
Q&A