Preparing your QA team for mobile testing
•Transferir como PPTX, PDF•
8 gostaram•4,257 visualizações
Rewriting your Master Testing Strategy is in order when you look to tackle your first mobile initiative. This includes getting your test environments ready for testing. Often test environments have been on closed internal networks that are not accessible from the internet. Delivering and testing a mobile application on a 3G or 4G network changes all that. Coming up with a supported platform strategy is also paramount to success. One needs to take network speed, device features, screen size and resolution as well as multi-tasking life-cycles into effect when testing. QA testers will need to become familiar on how to service the devices they test with as well. Additionally the team will need to incorporate field testing prior to delivery. This presentation will organize the challenges a modern QA team has to contend with, and make some strong suggestions on how to craft a respectable Mobile Test Strategy.
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Destaque
Destaque (14)
Semelhante a Preparing your QA team for mobile testing
Semelhante a Preparing your QA team for mobile testing (20)
Último
Último (20)
Preparing your QA team for mobile testing
- 1. Preparing Your QA Team for Mobile Testing Where Reality and New Beginnings Meet
- 2. @ggeoffreggeoffre.com 20 years of experience in the development, marketing, sales, and leadership of computer software A C T S
- 3. Where did the term "bug" come from? 66 years ago on September 9th 1947, operators of the Mark II Aiken Relay Computer being tested at Harvard University, found something curious trapped between points at Relay #70, Panel F.
- 8. Bugs not Defects! Outside the scope of most acceptance or unit testing Tend to design around them not to them Defensive programming techniques employed More likely hardware based than software based Environment plays a major role discovering them Magnified greatly with mobile based development
- 10. Three Eras of Computing mainframe personal ubiquitous
- 11. Three Human to Computer Ratios many one one one one many : : :
- 12. Leaving Other Eras Behind Single, closed computing systems Three "major" user upgradable operating systems Maybe five “major” browsers Virtualization possible to contain testing options
- 19. Mobile Testing Challenge Device Manufactures and Device Models Carrier Networks and Network Speeds OS Versions and API Updates Different Features for Different Phones
- 20. Android Fragmentation 11,868 Distinct Android devices seen this year 3,997 Distinct Android devices seen last year 682,000 Devices surveyed for this report 47.5% Samsung's share of those devices 8 Android versions still in use 37.9% Android users on Jelly Bean http://opensignal.com/reports/fragmentation-2013/
- 21. Android Fragmentation 18,796 Distinct Android devices seen this year 11,868 Distinct Android devices seen last year 682,000 Devices surveyed for this report 43% Samsung's share of those devices 7 Android versions still in use 46% Android users on Jelly Bean http://opensignal.com/reports/2014/android-fragmentation/
- 36. iOS Omneity 12* Distinct iOS devices seen this year 8 Distinct iOS devices seen last year 10M Events/hr surveyed for this report 100% Apple's share of those devices 3 iOS versions still in use 80.7% iOS users on iOS 7 (97.1% iOS 6+7) http://www.fiksu.com/iOS-7-iPhone-5s-5c-Usage-Tracker
- 37. iOS Omneity 17* Distinct iOS devices seen this year 12 Distinct iOS devices seen last year 10M Events/hr surveyed for this report 100% Apple's share of those devices 3 iOS versions still in use 80.7% iOS users on iOS 7 (97.1% iOS 6+7) http://www.fiksu.com/iOS-7-iPhone-5s-5c-Usage-Tracker
- 47. 700 Million iOS Devices
- 48. 800 Million iOS Devices
- 49. 900 Million Android Devices
- 53. Planning for the Experience
- 54. NFRs of Mobile Immutable requirements set before the project even begins Dictated to you by many different outside organizations Typically misunderstood during the design phase Often ignored by functional based acceptance testing Not typically caught by code quality unit testing Highly improbable if not impossible to avoid absolutely Things that every mobile user deals with every single day
- 55. Examples of Mobile NFRs Wireless Network Access, Latency and Speed Screen Size, Aspect Ratio, Resolution and Orientation Multi-tasking Interruptions, Time Limits and Unexpected Shutdowns User Specified Configurations and Privacy Settings Processor Performance and Availability of On-Device Storage
- 56. Examples of Mobile NFRs GPS Speed, Accuracy and Availability Photo Resolution, Focal Length, Lowlight Ability Platform Versions, Releases and Updates Code Signing, Packaging and Distribution Methods
- 58. Network Reachability Degraded performance depending on network latency Slows down performance Some areas are not connected at all Stops functionality cold Bandwidth utilization and data plans Frustrates users and impacts runtime costs Users are actually Mobile and moving around! Don’t always access the network from the same WiFi
- 62. Screen Size and Orientation Layout of Information Can all of the information be viewed Scrolling To Access Information Usability of hidden navigational components Logic Triggered by Switching Refresh screen data with each rotation Deployment to Tablet Devices
- 64. Multitasking Interruptions Interruption from other Apps Incoming Phone Calls Tweets, Posts, Email etc… Background Operations Downloads, Tracking, Music External Connectivity Bluetooth, WiFi Screen Sharing
- 66. Battery Life Using lots of device features GPS, Network, Video, etc... User disabling device features to prolong battery life Affects use cases and therefore should affect testing scenarios User deletes Apps to find out which one is causing problems
- 68. Accessing Online APIs Keep in mind that users do not have to update Loosely coupled APIs with optional fields helps with flexibility Layers of services may simplify end user support, but complicates operations (Seperate API Layer for each User Group or Platform) Large APIs accounting for all needs get updated more frequently than specialized smaller APIs for a single Apps needs.
- 70. Rewrite Master Testing Strategy Connected and disconnected states Internet facing test environments Multiple sized devices for each platform version Multitasking scenarios per platform Track battery life in different use case scenarios Plan for live field testing to gauge responsiveness Review instrumentation results for memory use
- 71. Impacted Testing Areas Levels Unit Integration System Acceptance Methods Static Dynamic White-Box Black-Box Gray-Box Types Installation Compatibility Smoke Regression Usability Accessibility Security Internationalization and Localization Performance
- 72. Primary Test Cases Rotate the device (on every single screen) Receive a phone call (at the worst possible times) Switch to/from another app (at the worst possible times) Resolution Specific Content and Graphics Disconnect from the network entirely (airplane mode) 1
- 73. Edge Test Cases Switch between wi-fi and wireless networks (device specific) Battery dies (device shuts off instantly) Apply an update (with data change) 2
- 74. Special Test Cases Background processing of developed app App to app communications Specific interaction with device hardware Notification or event based triggering Specific interaction with platform APIs 3
- 75. Frustration comes from… Tester can no longer reproduce the error Tester can reproduce on physical device, but developer cannot in the simulator Developer requires physical access to testers device in order to reproduce and diagnose the error API environment is down or inaccessible App needs to be removed and re-installed Device needs to be reset (power, network, settings, full)
- 76. Learning How Mobile Works Shutdown Apps for a Fresh Start Hard Reset to Recover from Catastrophic Failures Back-up and Restore the Device to Factory Settings Delete and Re-Install Apps How to Create Multi-Tasking Interruptions Understanding of the Platform’s Notification System Privacy and Accessibility Settings
- 77. Some things that can help Version number of build, traceable back to the snapshot of the code branch, located within the deployed app on the test device Screen shots and even video of error occurring on the test device Simplistic API Test Interface to verify if it is the backend or the app Adequate supply of physical test devices that can be checked in/out by all team members
- 78. Some things that can help Add validation criterial to every story card Make version control and change control information of APIs available to all team members Create an API up/down status page
- 79. Important Communications Report the Device Type - Hardware and OS Versions Capture the complete software version information at time of failure App version traceable back to code branch Fresh install or update from prior version? API Environment and Version Perform a Standardized Smoke Test First Steps to Reproduce Reproducible on second like device Reproducible on second un-like device Reproducible within Simulator/Emulator
- 80. Agile Considerations introduction of non-functional story cards addition of non-functional tests to story cards SCRUM Considerations co-location and field testing is not practical together everyone plays every role (anyone can play any role) personal devices may not be fair game test devices
- 81. What about beta testers? They are dynamic and certainly fall into the system based compatibility testing space, but… are more often than not selective function based testers with their own agenda non-dedicated resources that have higher priorities in life outside of testing your app in a controlling position as some of your most valuable customers
- 82. Manual is too expensive… On the day of the launch, purchase the app from the store and… ensure that all of the last minute content changes were applied correctly access the system and log on to a real account download the updated version from the store to see if all of your previously stored data is still accessible