Different testing requirements that mobile applications have, challenges and solutions Challenges 1. Complex mobile testing matrix, Expensive test environment 2. Repetitive testing 3. Mobile testing for devices located at various locations Solutions: • Risk Based Testing approach • Using Mobile device emulators • Use of Automation tools • Leveraging external services

1. Mobile Application Testing : Challenges &
Solutions
Ashwini Phalle

2. 1. Mobile Testing – The
Need
2. Mobile Application
overview
3. Mobile Application
Testing
Requirements
4. Testing Challenges
5. Possible Solutions
2

3. The Mobile Testing – The Need
Brand new devices – smart phones, tablets being released every week
Analysts at Forrester Research predicted within a few years, the mobile
channel will displace the online channel for day-to-day banking tasks
Mobile apps have limited computing resources – still need to be agile and
reliable
Complex mobile environment: Devices, OS, Carries
3

4. Mobile Application
Test Requirements
4

5. Mobile Application – Test Requirements
Functional & UI related testing
• UI functional Testing
• System Testing
• Regression Testing
Usability Testing
• Usability of applications functionality
• Usability with various devices
External factor (Device related) testing
• Device Related Testing: Battery, SD card interactions,
• Voice Call Handling, SMS Handling, Notifications
• External incoming communications Infrared etc
Performance Testing
• Application response time
• Use of resources
• Application loading speed
Security Testing
• Security Testing for financial apps
• Authentication checks
5

6. Mobile Testing
Challenges
6

7. Mobile Test Environment Elements
7

8. Mobile Testing Matrix
1. Increased
Testing
Cost
2. Increased
Testing
Efforts
Features
3. Devices at
multiple
locations –
Not
Feasible
Locations
Device Models
OS & Browsers
8

9. Possible Solutions
9

10. Test Effort Optimization Through Prioritization
Risk Based Testing approach for optimized Testing Effort :
• With weighted Device Platform Matrix
• Testing Requirement and application types matrix
• Test Cases Business priority matrix
Device
Relative
Importanc OS1 OS2 OS3 OS4 OS5
e
OS
Relative
Importance
1 2 3 4 5 App
Relative
Importanc Financial Native Webapp App4 App5
Device1 e
1 Testing
Type
Device2
2
Relative
1 2 3 4 5
Device3 Importance
3
Functional 1
Device4
4
Usability 2
External
3
Factors
Performan
4
ce
Security
10

11. Mobile Emulators
• Mobile devices with different resolutions, screen sizes and browser implementations
• Good for finding browserplatform specific bugs
• Test the apps on device emulators, interact with OS, access in browser simulators
Note :
• Not all activities can be realistically
emulated
• Mobile apps can exhibit different behavior
on device due to
• Due to lower device power and memory,
• Different DPI resolution
• Not for Usability Testing
11

12. Outsourcing
Crowd-sourced Mobile Testing
Community that serves as a virtual
testing team or as an on-demand
extension for in-house QA departments
Mobile Could
Remote device access service, gives
online access to numerous real devices
on various networks
Perfectomobile.com
DeviceAnywhere.com
12

13. Best Practices
13

14. Best Practices
Browser metrics: Subject to change - depends on changes in technology, popularity of
the browser, performance issues
Device popularity: Analysis on the popular devices in the market, metrics are going to
change frequently
Metrics for regions, carriers: Analysis for device/carrier popularity in terms of web
traffic from devices in particular countries and regions
Test on desktop browser first for early bug detection
Testing at an early stage for the priority devices from the metrics
Test on emulator then on device for device specific bugs
Early Usability testing – on multiple devices
Performance Testing on different device and carrier mix based on country and
region
14

15. Q&A
15