Testing in the New Internet Era
•Transferir como KEY, PDF•
3 gostaram•1,344 visualizações
The Internet is not the same as you used it 5 years ago. The rise of Web Properties, Mobile, and Cloud services has changed how we use the Web today. Come and listen to how Mozilla is approaching the new Internet World, and get an overview how we Test the products build for it. High level overview of the QA process, as well as opportunities how you can get involved with testing.
Recomendados
Mais conteúdo relacionado
Último
Último (20)
Destaque
Destaque (20)
Testing in the New Internet Era
- 1. Testing in the New Internet Era Tony Chung QA Manager
- 3. A Trip down Memory Lane...
- 4. A Trip down Memory Lane...
- 5. The Internet of Yesterday
- 6. The Internet of Yesterday
- 7. The Internet of Yesterday
- 8. The Internet of Yesterday
- 9. The Internet of Yesterday
- 10. The Internet of Yesterday
- 11. The Internet of Yesterday
- 12. The Internet of Today
- 13. The Internet of Today
- 14. The Internet of Today
- 15. The Internet of Today
- 16. The Internet of Today
- 17. The Internet of Today
- 18. The Internet of Today
- 19. The Internet of Today
- 21. What Will we Cover Today?
- 22. What Will we Cover Today? • My Top 4 Internet Themes • Why, Mozilla? • Testing Approaches • Ways to Get involved!
- 23. Mozilla Mission
- 28. Top 4 Internet Themes
- 31. #4: Desktop
- 33. #4: Desktop Browsing • Despite slowing growth on PC usage, still the top way to access internet • Users still averaging ~7.7 hours a day usage* • In Southeast Asia, average 14-25 hours a week (source: Nielsen Wire, Nov 2011)* • Users still spend 1/3 of time online using Email, Search, Social, Gaming, and video • Security and Privacy even more important than ever
- 34. #4: Desktop Browsing • Despite slowing growth on PC usage, still the top way to access internet • Users still averaging ~7.7 hours a day usage* • In Southeast Asia, average 14-25 hours a week (source: Nielsen Wire, Nov 2011)* • Users still spend 1/3 of time online using Email, Search, Social, Gaming, and video • Security and Privacy even more important than ever * Source: http://blog.nielsen.com/nielsenwire/global/surging-internet-usage-in-southeast-asia-reshaping-the-media-landscape/
- 36. #4: Why, Mozilla? • Firefox is still our Flagship gateway to the Internet • Always pushing for Open Standards • 18-week release support (Nightly, Aurora, Beta) • Accelerated deployments has introduced new headaches, so still lots of work to improve on
- 38. #4: Testing Approach • Over 30,000 automated nightly tests ran, multiple platforms • Test Execution every week against Nightly, Aurora, and Beta channels • Automated regression tests via Mozmill Test Framework • Keeping up with new features and Testplans • Cross platform coverage, Localization, Security releases, Support • Web Compatibility and Exploration through Crowdsourcing
- 41. #3: Cloud
- 42. #3: Cloud
- 43. #3: Cloud • Online Storage: Access your data from Any device • Online computing: Relying less power on native apps • Apple, Google, Amazon, Netflix, etc.. investing hundreds of millions into Cloud storage and infrastructure • Storing music, video, apps, photos, docs • Broadband and Virtualization brings speed up, costs down
- 45. #3: Why, Mozilla? • Cloud Services are important building blocks to provide better User experiences • Built out 3 co-locations to support 1.6M Sync users • Secure: Ease of setup, yet Encrypted Desktop to Phone data syncing • building reusable libraries into a framework for Developers using REST apis
- 47. #3: Testing Approach • Weekly test coverage on Client & Server deployment trains • Functional feature coverage on Dev and Staging environments while generating balanced load • New coverage on upcoming services products like BrowserID and Notifications • Better ways to support new Service Apps, RESTful APIs
- 50. #2: Web Apps
- 51. #2: Web Apps
- 52. #2: Web Apps • Applications moving to Cloud, less relying on native OS • Mobile applications is redefining browsing and usage • Light storage, rapid application, Ubiquitous • Distribution model through Web stores and marketplaces
- 54. #2: Why, Mozilla? • Web Developers want an Appstore model for the web to showcase their work without proprietary silos • An OpenWebApp store to promote Apps written in HTML5, CSS, and Javascript • Uphold high security guidelines while respecting user privacy
- 55. Preview of OWA
- 56. Preview of OWA Click
- 58. #2: Test Approach • Cross Functional teams, lots of collaboration • Manual and automated testing approach • Project management challenge: Team dependencies, Mix of Code repositories, Independent deadlines, mixture of tools and bug tracking systems
- 61. #1: Mobile
- 62. #1: Mobile
- 63. #1: Mobile • Mobile is Everywhere! Now world’s most widely distributed computing system • Smartphones and Tablets are flooding the market • iOS: 27.3%, Android: 43.7%, Others: 30%* • Has led to explosion of mobile Applications • Affordable and Plentiful • “The mobile revolution is moving from Ear to Hand” - Jussi Hinkkanen, Head of Policy, Nokia Africa
- 64. #1: Mobile • Mobile is Everywhere! Now world’s most widely distributed computing system • Smartphones and Tablets are flooding the market • iOS: 27.3%, Android: 43.7%, Others: 30%* • Has led to explosion of mobile Applications • Affordable and Plentiful • “The mobile revolution is moving from Ear to Hand” - Jussi Hinkkanen, Head of Policy, Nokia Africa * comScore, Inc, US household, Aug 2011
- 71. * Source: Mobile Usage Statistics, Stats & Facts 2011 | Digital Buzz Blog
- 73. #1: Why, Mozilla? • We need to go where the internet is going, and uphold our mission doing it. • Firefox on Android. from XUL to Native • Web APIs support to mobile platform • Large investment in Boot 2 Gecko OS • Apps need to be on a common platform, not walled-garden approaches
- 74. B2G and WebAPI
- 75. B2G and WebAPI
- 77. #1: Test Approach • Lots of challenges: regional, hardware limitations, distribution channels • Firefox Android follows 18-week progression cycle • Manual and Automated testing of features and behavior • Execute litmus tests, building automated test harness • Web and Device compatibility coverage, More Crowdsourcing!
- 80. Who can help with Testing?
- 81. You Can!
- 82. You Can! • Do you have a computer or Android phone? Yes! • Do you want to produce quality products? Yes! • Do you support the Open Web? Yes! • Here’s how you can help...
- 83. Desktop Testing
- 84. Desktop Testing • Download a Firefox build on Nightly, Aurora, or Beta • Run Litmus tests on a localized build • File bugs in bugzilla! • Join our Firefox Testdays for real-time interaction with the community
- 86. Cloud Services Testing • Set Up Sync on Firefox! • Sync your bookmarks and history • Visit or Create a site using Browser ID and report bugs • File bugs in bugzilla! • Join our Sync or BrowserID Testdays for real-time interaction with the community
- 87. Open Web Apps Testing
- 88. Open Web Apps Testing • Download a Web App through the App Marketplace (coming soon) • Create a Web App and upload one! • File bugs in bugzilla! • Join our WebApp Testdays for real-time interaction with the community
- 89. Mobile Testing
- 90. Mobile Testing • Android phone users -- install Firefox for Android • Visit your favorite websites on different devices • Develop automated java tests in Robotium against Firefox Android • File bugs in bugzilla! • Join our Mobile Testdays for real-time interaction with the community
- 92. • Get started by visiting • http://quality.mozilla.org • Chat with us real-time on • irc.mozilla.org, channel #qa • File and Verify Bugs!
- 93. Join Us!
- 94. Join Us!
Notas do Editor
- my name is Tony Chung, and i'm a QA manager at Mozilla. I've been a mozilla employee for 5 years, starting with the day that Firefox 2.0 shipped. \n\nWe're here primarily to Pursue Quality, push for user voice, and ship quality products. \n\nI've titled this talk Testing in the Internet Era, because we are no longer one firefox product for Desktop product anymore. Mitchell gave us a preview in her keynote this morning on what this new Internet Era is, and we'll focus the next 30 minutes on what those are, and how our Quality approach is like. \n
- \n
- \n
- modems--IE vs Netscape -- One Platform, --Proprietary Web Properties (active X, flash) --Service Providers controlling content (AIM, prodigy) --Expensive Option\n
- modems--IE vs Netscape -- One Platform, --Proprietary Web Properties (active X, flash) --Service Providers controlling content (AIM, prodigy) --Expensive Option\n
- modems--IE vs Netscape -- One Platform, --Proprietary Web Properties (active X, flash) --Service Providers controlling content (AIM, prodigy) --Expensive Option\n
- modems--IE vs Netscape -- One Platform, --Proprietary Web Properties (active X, flash) --Service Providers controlling content (AIM, prodigy) --Expensive Option\n
- modems--IE vs Netscape -- One Platform, --Proprietary Web Properties (active X, flash) --Service Providers controlling content (AIM, prodigy) --Expensive Option\n
- modems--IE vs Netscape -- One Platform, --Proprietary Web Properties (active X, flash) --Service Providers controlling content (AIM, prodigy) --Expensive Option\n
- Broadband Everywhere, -- Multiple Platforms --Speed, Speed, Speed,---Security and User Privacy,--The norm on how we communicate\n
- Broadband Everywhere, -- Multiple Platforms --Speed, Speed, Speed,---Security and User Privacy,--The norm on how we communicate\n
- Broadband Everywhere, -- Multiple Platforms --Speed, Speed, Speed,---Security and User Privacy,--The norm on how we communicate\n
- Broadband Everywhere, -- Multiple Platforms --Speed, Speed, Speed,---Security and User Privacy,--The norm on how we communicate\n
- Broadband Everywhere, -- Multiple Platforms --Speed, Speed, Speed,---Security and User Privacy,--The norm on how we communicate\n
- Broadband Everywhere, -- Multiple Platforms --Speed, Speed, Speed,---Security and User Privacy,--The norm on how we communicate\n
- Broadband Everywhere, -- Multiple Platforms --Speed, Speed, Speed,---Security and User Privacy,--The norm on how we communicate\n
- \n
- \n
- \n
- Mitchell, Chibi, Mary, and everyone else covered this already, but i’ll reiterate why each role plays a big part of the mission. As a Quality Assurance person, we need to continue to appealing to our users, First. Our reputation is about quality, “standing up, taking a moment, and file a bug” \n\nhttp://www.mozilla.org/about/mission.html\nOur Mission: Mozilla's mission is to promote openness, innovation and opportunity on the web.\nWhat we do - We do this by creating great software, like the Firefox browser, and building movements, like Drumbeat, that give people tools to take control of their online lives.\nWhat we strive for - As a non-profit organization, we define success in terms of building communities and enriching people’s lives instead of benefiting shareholders. We believe in the power and potential of the Internet and want to see it thrive for everyone, everywhere.\nWho we work with - Building a better Internet is an ambitious goal, but we believe that it is possible when people who share our passion get involved. Coders, artists, writers, testers, surfers, students, grandparents—anyone who cares about the web can help make it even better. Find out how you can help.\nWhat drives us - Read the Mozilla Manifesto to learn more about the values and principles that guide the pursuit of our mission.\n\n\n
- Mitchell, Chibi, Mary, and everyone else covered this already, but i’ll reiterate why each role plays a big part of the mission. As a Quality Assurance person, we need to continue to appealing to our users, First. Our reputation is about quality, “standing up, taking a moment, and file a bug” \n\nhttp://www.mozilla.org/about/mission.html\nOur Mission: Mozilla's mission is to promote openness, innovation and opportunity on the web.\nWhat we do - We do this by creating great software, like the Firefox browser, and building movements, like Drumbeat, that give people tools to take control of their online lives.\nWhat we strive for - As a non-profit organization, we define success in terms of building communities and enriching people’s lives instead of benefiting shareholders. We believe in the power and potential of the Internet and want to see it thrive for everyone, everywhere.\nWho we work with - Building a better Internet is an ambitious goal, but we believe that it is possible when people who share our passion get involved. Coders, artists, writers, testers, surfers, students, grandparents—anyone who cares about the web can help make it even better. Find out how you can help.\nWhat drives us - Read the Mozilla Manifesto to learn more about the values and principles that guide the pursuit of our mission.\n\n\n
- Mitchell, Chibi, Mary, and everyone else covered this already, but i’ll reiterate why each role plays a big part of the mission. As a Quality Assurance person, we need to continue to appealing to our users, First. Our reputation is about quality, “standing up, taking a moment, and file a bug” \n\nhttp://www.mozilla.org/about/mission.html\nOur Mission: Mozilla's mission is to promote openness, innovation and opportunity on the web.\nWhat we do - We do this by creating great software, like the Firefox browser, and building movements, like Drumbeat, that give people tools to take control of their online lives.\nWhat we strive for - As a non-profit organization, we define success in terms of building communities and enriching people’s lives instead of benefiting shareholders. We believe in the power and potential of the Internet and want to see it thrive for everyone, everywhere.\nWho we work with - Building a better Internet is an ambitious goal, but we believe that it is possible when people who share our passion get involved. Coders, artists, writers, testers, surfers, students, grandparents—anyone who cares about the web can help make it even better. Find out how you can help.\nWhat drives us - Read the Mozilla Manifesto to learn more about the values and principles that guide the pursuit of our mission.\n\n\n
- \n
- \n
- \n
- The internet is still widely associated with using your computer. Desktop Applications and storage are moving to the cloud, and less of us are carrying around disks and USB drives around. \n\nHardware support for faster processors, more memory, Bettery battery, Large screensizes, built in networking, hardware acceleration support, complements performance on powerful web content usage\n\n“Nielsen’s new Southeast Asia Digital Consumer Report examined the digital media habits and attitudes of consumers in Singapore, Thailand, Indonesia, Malaysia, Vietnam and the Philippines. Singaporeans led the region in online usage, spending more than a day (25 hours) online each week, while Filipinos and Malaysians came close behind, spending 21.5 hours and 19.8 hours a week online, respectively. Indonesians trailed the region, spending an average of 14 hours per week.”\nSource: http://blog.nielsen.com/nielsenwire/global/surging-internet-usage-in-southeast-asia-reshaping-the-media-landscape/\n
- The internet is still widely associated with using your computer. Desktop Applications and storage are moving to the cloud, and less of us are carrying around disks and USB drives around. \n\nHardware support for faster processors, more memory, Bettery battery, Large screensizes, built in networking, hardware acceleration support, complements performance on powerful web content usage\n\n“Nielsen’s new Southeast Asia Digital Consumer Report examined the digital media habits and attitudes of consumers in Singapore, Thailand, Indonesia, Malaysia, Vietnam and the Philippines. Singaporeans led the region in online usage, spending more than a day (25 hours) online each week, while Filipinos and Malaysians came close behind, spending 21.5 hours and 19.8 hours a week online, respectively. Indonesians trailed the region, spending an average of 14 hours per week.”\nSource: http://blog.nielsen.com/nielsenwire/global/surging-internet-usage-in-southeast-asia-reshaping-the-media-landscape/\n
- Marketshare dipped, but still about 22% in the US. \n\nInvesting heavily into HTML5, we believe JS, HTML, and CSS creates optimal and open web development. \n- And many browsers are adopting that too\n\n- in 2010, we realized we needed to move faster. Thus, 6 week cycles began, on Nightly, Aurora, and Beta channels\n\n- Many improvements landed, but so did headaches. \n** Extensions broke, update fatigue\n
- \n
- \n
- \n
- The general idea of the “cloud” is to store your media on the internet so you can access it from any device anywhere, as opposed to leaving it on a hard drive. Now with cloud services, we can juggle around our data between multiple gadgets.\n\nCloud computing is a marketing term for technologies that provide computation, software, data access, and storage services that do not require end-user knowledge of the physical location and configuration of the system that delivers the services. \nCloud computing describes a new supplement, consumption, and delivery model for IT services based on Internet protocols, and it typically involves provisioning of dynamically scalable and often virtualised resources\nCloud computing providers deliver applications via the internet, which are accessed from web browsers and desktop and mobile apps, while the business software and data are stored on servers at a remote location. In some cases, legacy applications (line of business applications that until now have been prevalent in thin client Windows computing) are delivered via a screen-sharing technology, while the computing resources are consolidated at a remote data center location; in other cases, entire business applications have been coded using web-based technologies such as AJAX.\n\nGoogle - Cloud storage, App Engine\nAmazon, Apple, Sync, itunes (store your video and photos), purchase your store\n
- \n* We’ve heavily invested in Sync for 2 years now. Setting bookmarks and history up once, then leaving it in the backgound, and using your other machines or phone to retrieve data has been a automatic\n\n* Secure: \nMobile and Desktop complete the two roundtrips of J-PAKE messages to agree upon a strong secret K\nA 256 bit key is derived from K using HMAC-SHA256 using a fixed extraction key.\nThe encryption and HMAC keys are derived from that 256 bit key using HMAC-SHA256.\nKey Derivation: The AES encryption key T(1) and the HMAC key T(2) will be derived from J-PAKE's strong secret K as follows:\nSagrada - As we build more services at Mozilla, we will be able to identify further shared functionality and look to make a new components, but the ones we are choosing to focus on initially are:\n\nSauropod is a secure storage system for user data. It employs end-to-end encryption and secure key storage to enable least-privilege access, fine-grain user permissioning, and a controlled and auditable process for administrative and automated data access. \n- key-value storage API, where each user has a completely independent universe of keys.\n\n\nSagrada link: https://wiki.mozilla.org/Services/Sagrada\n
- BrowserID providing single sign on capability to the browser, targeting smaller websites\n* dont have to cave into big bad hairy facebook and google to worry about sharing data\n* APIs make it easy to authentic on your email address only, and establish the handshake\n\nPlans for a beta program provides shared APIs to service app hackers to put smaller projects into an extension and throw them into a train, with low obligations but a release test model.\n
- \n
- \n
- \n
- Web developers are expressing interest in an app store model for the Web that would enable them to get paid for their efforts without having to abandon Web development in exchange for proprietary silos, each with their own programming language and SDK, variable and sometimes opaque review processes, and limited reach.\nSupporting the needs of Web developers in their efforts to develop websites and apps that aren’t bound to a specific browser and work across the Web is core to Mozilla’s public benefit mission.\nAnd so we’ve been actively exploring what an Open Web App Store would need to look like to ensure the long-term health and vitality of the Web as an incredibly open and accessible platform for innovation.\nWhat does it mean to be an Open Web App Store? As a starting point, we are proposing a set of high-level principles.\nAn Open Web App Store should:\nexclusively host web applications based upon HTML5, CSS, Javascript and other widely-implemented open standards in modern web browsers — to avoid interoperability, portability and lock-in issues\nensure that discovery, distribution and fulfillment works across all modern browsers, wherever they run (including on mobile devices)\nset forth editorial, security and quality review guidelines and processes that are transparent and provide for a level playing field\nrespect individual privacy by not profiling and tracking individual user behavior beyond what’s strictly necessary for distribution and fulfillment\nbe open and accessible to all app producers and app consumers\n\n
- \n
- OWA Market - Web QA\nDashboard - All\nFirefox Extension - Desktop QA\nHTML5 Shim - API automation QA\nMobile (Soap) - Mobile QA\nBrowserID - Services\nScalability - WebQA and Services\n\n
- \n
- \n
- ask the audience to raise their smartphone, get a gauge how many people that is\nMulti - purpose: No longer just dialing your girlfriend or mother. Web browsing, gaming, social, watching videos, trading stocks, sending Texts, everything you can do on a PC and more., \nin some developing countries, a mobile device is the only way to get on the internet now. many manufacturers in \nEven in poor countries about two-thirds of people have access to one (see chart 1). As a result, such devices and their networks, though mainly still much simpler than in the rich world, have become a platform on which many other services can be built. This boosts innovation—just as smartphones and faster wireless data networks have led to an explosion of mobile applications (“apps”).\n\n
- ask the audience to raise their smartphone, get a gauge how many people that is\nMulti - purpose: No longer just dialing your girlfriend or mother. Web browsing, gaming, social, watching videos, trading stocks, sending Texts, everything you can do on a PC and more., \nin some developing countries, a mobile device is the only way to get on the internet now. many manufacturers in \nEven in poor countries about two-thirds of people have access to one (see chart 1). As a result, such devices and their networks, though mainly still much simpler than in the rich world, have become a platform on which many other services can be built. This boosts innovation—just as smartphones and faster wireless data networks have led to an explosion of mobile applications (“apps”).\n\n
- \n
- \n
- \n
- \n
- \n
- \n
- Last summit theme was Be Mobile. \n\n2008, Started with Nokia Maemo platform, tried out Win mo, killed winmo, back to Nokia, and now Android. \n\nB2G - stripped down to android kernal\n\nMozilla believes that the web can displace proprietary, single-vendor stacks for application development. To make open web technologies a better basis for future applications on mobile and desktop alike, we need to keep pushing the envelope of the web to include --- and in places exceed --- the capabilities of the competing stacks in question.\nWebAPI - New web APIs: build prototype APIs for exposing device and OS capabilities to content (Telephony, SMS, Camera, USB, Bluetooth, NFC, etc.)\n\n
- \n
- Lots of work. since mobile is new, we need to evangelise and push hard on Android. Challenges:\n* regional limitation of devices. iOS has 5 devices, 1 manufacturer. Android has 6 major manufacturers, over 250 known brands and climbing\n* many are running low end hardware, non updated Android OS’s\n* we have one App, that doesnt come with the stock setup, and is 3 times as large an install base. \n* carriers can block apps, limit download size.. need distributionn model\n\nChannels: moving off XUL Fennec to Native. 6 weeks, we change channels like firefox\n\nTurn to community crowdsourcing for coverage!\n
- \n
- \n
- \n
- \n
- \n
- \n\n
- \n
- \n
- \n
- \n