Enviar pesquisa
Carregar
3 f6 10_testing
âą
0 gostou
âą
895 visualizaçÔes
O
op205
Seguir
Tecnologia
Denunciar
Compartilhar
Denunciar
Compartilhar
1 de 23
Baixar agora
Baixar para ler offline
Recomendados
Lecture 7 Software Engineering and Design User Interface Design
Lecture 7 Software Engineering and Design User Interface Design
op205
Â
3 f6 11_softdevmethodologies
3 f6 11_softdevmethodologies
op205
Â
Lecture 3 Software Engineering and Design Introduction to UML
Lecture 3 Software Engineering and Design Introduction to UML
op205
Â
3 f6 8_databases
3 f6 8_databases
op205
Â
Lecture 1 Software Engineering and Design Introduction
Lecture 1 Software Engineering and Design Introduction
op205
Â
3 f6 9_distributed_systems
3 f6 9_distributed_systems
op205
Â
Half-Push/Half-Polling
Half-Push/Half-Polling
YoungSu Son
Â
Half-Push/Half-Polling pattern
Half-Push/Half-Polling pattern
YoungSu Son
Â
Recomendados
Lecture 7 Software Engineering and Design User Interface Design
Lecture 7 Software Engineering and Design User Interface Design
op205
Â
3 f6 11_softdevmethodologies
3 f6 11_softdevmethodologies
op205
Â
Lecture 3 Software Engineering and Design Introduction to UML
Lecture 3 Software Engineering and Design Introduction to UML
op205
Â
3 f6 8_databases
3 f6 8_databases
op205
Â
Lecture 1 Software Engineering and Design Introduction
Lecture 1 Software Engineering and Design Introduction
op205
Â
3 f6 9_distributed_systems
3 f6 9_distributed_systems
op205
Â
Half-Push/Half-Polling
Half-Push/Half-Polling
YoungSu Son
Â
Half-Push/Half-Polling pattern
Half-Push/Half-Polling pattern
YoungSu Son
Â
Ajs 4 a
Ajs 4 a
Niit Care
Â
Ajs 2 a
Ajs 2 a
Niit Care
Â
Ajs 3 c
Ajs 3 c
Niit Care
Â
Model driven code development using emf and jet
Model driven code development using emf and jet
kowost
Â
Ajs 4 b
Ajs 4 b
Niit Care
Â
Unit 01 - Introduction
Unit 01 - Introduction
DSBW 2011/2002 - Carles Farré - Barcelona Tech
Â
Lecture 4 Software Engineering and Design Brief Introduction to Programming
Lecture 4 Software Engineering and Design Brief Introduction to Programming
op205
Â
3 f6 9a_corba
3 f6 9a_corba
op205
Â
3 f6 security
3 f6 security
op205
Â
E Skills Week Teacher Guide
E Skills Week Teacher Guide
LucianeCurator, EUNEOS Social Media Manager and Trainer
Â
23204961
23204961
radgirl
Â
OltreWeb 2.0 -- Gaudio Fabrizio
OltreWeb 2.0 -- Gaudio Fabrizio
darkgod90
Â
Old and/or young
Old and/or young
Maria Ekström
Â
Open social & cmis oasistc-20100712
Open social & cmis oasistc-20100712
weitzelm
Â
Dibujos de movil
Dibujos de movil
cpremolino
Â
Profile Dynamics Info
Profile Dynamics Info
mwilts
Â
Challenges of AJAX Development for Smartphone Platforms
Challenges of AJAX Development for Smartphone Platforms
MaxMotovilov
Â
Tugas
Tugas
andrewaja
Â
GM Hand Book
GM Hand Book
Sejal Parikh
Â
Sandip Mukherjee CV
Sandip Mukherjee CV
sandipm
Â
BCC 2009 - NSTC
BCC 2009 - NSTC
Duane Blackburn
Â
Baie Des Cochons
Baie Des Cochons
terissarafuse
Â
Mais conteĂșdo relacionado
Mais procurados
Ajs 4 a
Ajs 4 a
Niit Care
Â
Ajs 2 a
Ajs 2 a
Niit Care
Â
Ajs 3 c
Ajs 3 c
Niit Care
Â
Model driven code development using emf and jet
Model driven code development using emf and jet
kowost
Â
Ajs 4 b
Ajs 4 b
Niit Care
Â
Unit 01 - Introduction
Unit 01 - Introduction
DSBW 2011/2002 - Carles Farré - Barcelona Tech
Â
Mais procurados
(6)
Ajs 4 a
Ajs 4 a
Â
Ajs 2 a
Ajs 2 a
Â
Ajs 3 c
Ajs 3 c
Â
Model driven code development using emf and jet
Model driven code development using emf and jet
Â
Ajs 4 b
Ajs 4 b
Â
Unit 01 - Introduction
Unit 01 - Introduction
Â
Destaque
Lecture 4 Software Engineering and Design Brief Introduction to Programming
Lecture 4 Software Engineering and Design Brief Introduction to Programming
op205
Â
3 f6 9a_corba
3 f6 9a_corba
op205
Â
3 f6 security
3 f6 security
op205
Â
E Skills Week Teacher Guide
E Skills Week Teacher Guide
LucianeCurator, EUNEOS Social Media Manager and Trainer
Â
23204961
23204961
radgirl
Â
OltreWeb 2.0 -- Gaudio Fabrizio
OltreWeb 2.0 -- Gaudio Fabrizio
darkgod90
Â
Old and/or young
Old and/or young
Maria Ekström
Â
Open social & cmis oasistc-20100712
Open social & cmis oasistc-20100712
weitzelm
Â
Dibujos de movil
Dibujos de movil
cpremolino
Â
Profile Dynamics Info
Profile Dynamics Info
mwilts
Â
Challenges of AJAX Development for Smartphone Platforms
Challenges of AJAX Development for Smartphone Platforms
MaxMotovilov
Â
Tugas
Tugas
andrewaja
Â
GM Hand Book
GM Hand Book
Sejal Parikh
Â
Sandip Mukherjee CV
Sandip Mukherjee CV
sandipm
Â
BCC 2009 - NSTC
BCC 2009 - NSTC
Duane Blackburn
Â
Baie Des Cochons
Baie Des Cochons
terissarafuse
Â
Collateral Advantage
Collateral Advantage
Melanie Boellinger
Â
Star 2013-pdfa-pdfa
Star 2013-pdfa-pdfa
Dave McAllister
Â
OpenSocial State of the Union 2010
OpenSocial State of the Union 2010
weitzelm
Â
Using The National Science and Technology Council (NSTC)
Using The National Science and Technology Council (NSTC)
Duane Blackburn
Â
Destaque
(20)
Lecture 4 Software Engineering and Design Brief Introduction to Programming
Lecture 4 Software Engineering and Design Brief Introduction to Programming
Â
3 f6 9a_corba
3 f6 9a_corba
Â
3 f6 security
3 f6 security
Â
E Skills Week Teacher Guide
E Skills Week Teacher Guide
Â
23204961
23204961
Â
OltreWeb 2.0 -- Gaudio Fabrizio
OltreWeb 2.0 -- Gaudio Fabrizio
Â
Old and/or young
Old and/or young
Â
Open social & cmis oasistc-20100712
Open social & cmis oasistc-20100712
Â
Dibujos de movil
Dibujos de movil
Â
Profile Dynamics Info
Profile Dynamics Info
Â
Challenges of AJAX Development for Smartphone Platforms
Challenges of AJAX Development for Smartphone Platforms
Â
Tugas
Tugas
Â
GM Hand Book
GM Hand Book
Â
Sandip Mukherjee CV
Sandip Mukherjee CV
Â
BCC 2009 - NSTC
BCC 2009 - NSTC
Â
Baie Des Cochons
Baie Des Cochons
Â
Collateral Advantage
Collateral Advantage
Â
Star 2013-pdfa-pdfa
Star 2013-pdfa-pdfa
Â
OpenSocial State of the Union 2010
OpenSocial State of the Union 2010
Â
Using The National Science and Technology Council (NSTC)
Using The National Science and Technology Council (NSTC)
Â
Semelhante a 3 f6 10_testing
Yamuna chari(experience 3years(automation & manual))
Yamuna chari(experience 3years(automation & manual))
Yamuna Chari
Â
Yamuna Chari(Experience -3years(Automation & Manual))
Yamuna Chari(Experience -3years(Automation & Manual))
Yamuna Chari
Â
WE-06-Testing.ppt
WE-06-Testing.ppt
javed281701
Â
Manual testing testing master.pdf
Manual testing testing master.pdf
synamedia
Â
ManualTestingMaterial.pdf
ManualTestingMaterial.pdf
SCMCpvt
Â
A RELIABLE AND AN EFFICIENT WEB TESTING SYSTEM
A RELIABLE AND AN EFFICIENT WEB TESTING SYSTEM
ijseajournal
Â
A RELIABLE AND AN EFFICIENT WEB TESTING SYSTEM
A RELIABLE AND AN EFFICIENT WEB TESTING SYSTEM
ijseajournal
Â
A_Project_Report_ON_ONLINE_EXAMS_SYSTEM.pdf
A_Project_Report_ON_ONLINE_EXAMS_SYSTEM.pdf
AmishGupta15
Â
A Project Report ON ONLINE EXAMS SYSTEM
A Project Report ON ONLINE EXAMS SYSTEM
Dawn Cook
Â
Bugday bkk-2014 nitisak-auto_perf
Bugday bkk-2014 nitisak-auto_perf
Nitisak Mooltreesri
Â
Online Exam
Online Exam
Sandeep Singh
Â
Functional testing patterns
Functional testing patterns
Premanand Chandrasekaran
Â
Online Examination system mini project -1.ppt
Online Examination system mini project -1.ppt
ParvatiRathod1
Â
User Testing talk by Chris Rourke of User Vision
User Testing talk by Chris Rourke of User Vision
techmeetup
Â
Preliminry report
Preliminry report
Jiten Ahuja
Â
Stop guessing, start testing â mobile testing done right - Timo Euteneuer
Stop guessing, start testing â mobile testing done right - Timo Euteneuer
JAXLondon_Conference
Â
Unit 09: Web Application Testing
Unit 09: Web Application Testing
DSBW 2011/2002 - Carles Farré - Barcelona Tech
Â
Web Usability (Slideshare Version)
Web Usability (Slideshare Version)
Carles Farré
Â
Stc 2015 regional-round-ppt-exlopratory mobile testing with risk analysis
Stc 2015 regional-round-ppt-exlopratory mobile testing with risk analysis
Archana Krushnan
Â
Building and Scaling High Performing Technology Organizations by Jez Humble a...
Building and Scaling High Performing Technology Organizations by Jez Humble a...
Agile India
Â
Semelhante a 3 f6 10_testing
(20)
Yamuna chari(experience 3years(automation & manual))
Yamuna chari(experience 3years(automation & manual))
Â
Yamuna Chari(Experience -3years(Automation & Manual))
Yamuna Chari(Experience -3years(Automation & Manual))
Â
WE-06-Testing.ppt
WE-06-Testing.ppt
Â
Manual testing testing master.pdf
Manual testing testing master.pdf
Â
ManualTestingMaterial.pdf
ManualTestingMaterial.pdf
Â
A RELIABLE AND AN EFFICIENT WEB TESTING SYSTEM
A RELIABLE AND AN EFFICIENT WEB TESTING SYSTEM
Â
A RELIABLE AND AN EFFICIENT WEB TESTING SYSTEM
A RELIABLE AND AN EFFICIENT WEB TESTING SYSTEM
Â
A_Project_Report_ON_ONLINE_EXAMS_SYSTEM.pdf
A_Project_Report_ON_ONLINE_EXAMS_SYSTEM.pdf
Â
A Project Report ON ONLINE EXAMS SYSTEM
A Project Report ON ONLINE EXAMS SYSTEM
Â
Bugday bkk-2014 nitisak-auto_perf
Bugday bkk-2014 nitisak-auto_perf
Â
Online Exam
Online Exam
Â
Functional testing patterns
Functional testing patterns
Â
Online Examination system mini project -1.ppt
Online Examination system mini project -1.ppt
Â
User Testing talk by Chris Rourke of User Vision
User Testing talk by Chris Rourke of User Vision
Â
Preliminry report
Preliminry report
Â
Stop guessing, start testing â mobile testing done right - Timo Euteneuer
Stop guessing, start testing â mobile testing done right - Timo Euteneuer
Â
Unit 09: Web Application Testing
Unit 09: Web Application Testing
Â
Web Usability (Slideshare Version)
Web Usability (Slideshare Version)
Â
Stc 2015 regional-round-ppt-exlopratory mobile testing with risk analysis
Stc 2015 regional-round-ppt-exlopratory mobile testing with risk analysis
Â
Building and Scaling High Performing Technology Organizations by Jez Humble a...
Building and Scaling High Performing Technology Organizations by Jez Humble a...
Â
Mais de op205
3 f6 9a_corba
3 f6 9a_corba
op205
Â
Lecture 6 Software Engineering and Design Good Design
Lecture 6 Software Engineering and Design Good Design
op205
Â
Lecture 5 Software Engineering and Design Design Patterns
Lecture 5 Software Engineering and Design Design Patterns
op205
Â
Lecture 2 Software Engineering and Design Object Oriented Programming, Design...
Lecture 2 Software Engineering and Design Object Oriented Programming, Design...
op205
Â
More on DFT
More on DFT
op205
Â
Digital Signal Processing Summary
Digital Signal Processing Summary
op205
Â
Implementation of Digital Filters
Implementation of Digital Filters
op205
Â
Basics of Analogue Filters
Basics of Analogue Filters
op205
Â
Design of IIR filters
Design of IIR filters
op205
Â
Design of FIR filters
Design of FIR filters
op205
Â
Basics of Digital Filters
Basics of Digital Filters
op205
Â
Introduction to Digital Signal Processing
Introduction to Digital Signal Processing
op205
Â
Fast Fourier Transform
Fast Fourier Transform
op205
Â
Brief Review of Fourier Analysis
Brief Review of Fourier Analysis
op205
Â
3F3 â Digital Signal Processing (DSP) - Part1
3F3 â Digital Signal Processing (DSP) - Part1
op205
Â
Mais de op205
(15)
3 f6 9a_corba
3 f6 9a_corba
Â
Lecture 6 Software Engineering and Design Good Design
Lecture 6 Software Engineering and Design Good Design
Â
Lecture 5 Software Engineering and Design Design Patterns
Lecture 5 Software Engineering and Design Design Patterns
Â
Lecture 2 Software Engineering and Design Object Oriented Programming, Design...
Lecture 2 Software Engineering and Design Object Oriented Programming, Design...
Â
More on DFT
More on DFT
Â
Digital Signal Processing Summary
Digital Signal Processing Summary
Â
Implementation of Digital Filters
Implementation of Digital Filters
Â
Basics of Analogue Filters
Basics of Analogue Filters
Â
Design of IIR filters
Design of IIR filters
Â
Design of FIR filters
Design of FIR filters
Â
Basics of Digital Filters
Basics of Digital Filters
Â
Introduction to Digital Signal Processing
Introduction to Digital Signal Processing
Â
Fast Fourier Transform
Fast Fourier Transform
Â
Brief Review of Fourier Analysis
Brief Review of Fourier Analysis
Â
3F3 â Digital Signal Processing (DSP) - Part1
3F3 â Digital Signal Processing (DSP) - Part1
Â
Ăltimo
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Katpro Technologies
Â
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Michael W. Hawkins
Â
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Delhi Call girls
Â
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Maria Levchenko
Â
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Malak Abu Hammad
Â
đŹ The future of MySQL is Postgres đ
đŹ The future of MySQL is Postgres đ
RTylerCroy
Â
Slack Application Development 101 Slides
Slack Application Development 101 Slides
praypatel2
Â
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
wesley chun
Â
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
The Digital Insurer
Â
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
ThousandEyes
Â
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
Â
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
Enterprise Knowledge
Â
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
Results
Â
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
Â
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
sudhanshuwaghmare1
Â
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Drew Madelung
Â
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc
Â
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Miguel AraĂșjo
Â
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
Delhi Call girls
Â
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
UK Journal
Â
Ăltimo
(20)
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Â
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Â
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
Â
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Â
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Â
đŹ The future of MySQL is Postgres đ
đŹ The future of MySQL is Postgres đ
Â
Slack Application Development 101 Slides
Slack Application Development 101 Slides
Â
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
Â
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
Â
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
Â
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
Â
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
Â
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
Â
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Â
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
Â
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Â
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
Â
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Â
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
Â
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Â
3 f6 10_testing
1.
Testing Elena Punskaya, op205@cam.ac.uk
1
2.
It is All
About Trust Defects destroy the trust required for effective software development. The customers need to be able to trust the software. The managers need to be able to trust reports of progress. The programmers need to be able to trust each other. Defects destroy this trust. Kent Beck and Cynthia Andres, Extreme Programming Explained: Embrace Change Main Questions: ⹠What is Testing? ⹠What to Test? ⹠How to Test? ⹠When to Test? ⹠Who should Test? ⹠What are the Tools for the Job? © 2012 Elena Punskaya 2 Cambridge University Engineering Department 2
3.
What is Testing? âą
Testing is double checking - If you add a column of numbers one way, there are many errors that could cause your sum to be wrong. Add the numbers two different ways, say from top and then from the bottom and the same answer is likely to be the right answer. Finding bugs is somewhat like fishing with a net. We use fine, small nets (unit tests) to catch the minnows, and big, coarse nets (integration tests) to catch the killer sharks. Andrew Hunt and David Thomas, The Pragmatic Programmer âą Many types of tests exist, what follows is just a starting point! âą Many ways to categorise the tests, depending on oneâs perspective - © 2012 Elena Punskaya 3 Cambridge University Engineering Department 3
4.
What to Test?
Let us follow the process: âą Unit test - is code that exercises a small unit (module) of software separate from other units of the application - is the foundation of all other types of testing (if parts donât work by themselves they wonât work together) âą Integration (component) testing - ok, all parts seem to be working but how do they interact with each other, does the entire subsystem work? - with good OO design in place should be straightforward to detect the issues - potentially can be the largest source of bugs in a system âą Validation and verification - ok, I put something together, this seems to be working as my users wanted but is this what they needed? are functional requirements met? © 2012 Elena Punskaya 4 Cambridge University Engineering Department 4
5.
Validation and Verification?
© 2012 Elena Punskaya 5 Cambridge University Engineering Department 5
6.
What to Test? âą
Resource exhaustion, errors, recovery - ok, now the system behaves correctly under ideal condition, what about real-world? - could it run out of ⣠memory? ⣠disk space? ⣠CPU bandwidth? ⣠video resolution? ⣠network bandwidth? ⣠etc. - if it has to fail, will it fail gracefully? âą Performance testing, stress testing or testing under load - ok, it can work under real world conditions but what if the number of users/ transactions/connections increases? is it scalable? - you might need to simulate the load realistically âą Usability testing - ok, seems to work in real world but would it with real users under real conditions? this new tool âfits our handsâ but does it âfit other usersâ handsâ - human factor plays important role, is it intuitively clear how to use? - failure to meet usability requirement is the same as dividing by zero âą Test are also software - donât forget to test them! :-) © 2012 Elena Punskaya 6 Cambridge University Engineering Department 6
7.
Profiling Tools âą
Profiling tools allow to analyse software performance âą In particular, tracking Memory Allocation throughout runtime allows to spot memory leaks and inefficient memory use (requiring too much memory space without a real need) âą Tracking time required execution of program methods helps to identify âbottlenecksâ of performance that could be caused either by ineffective programming techniques or could benefit from optimisation XCode profiling tools, https://developer.apple.com/library/ios/#documentation/Performance/Conceptual/PerformanceOverview/ InitialEvaluation/InitialEvaluation.html#//apple_ref/doc/uid/TP40001410-CH206-SW8 © 2012 Elena Punskaya 7 Cambridge University Engineering Department 7
8.
Usability Testing âą
Usability Testing is should be one of the most critical parts of software development: designing experiences! âą Consider web search â a user wants most relevant results in the fastest way, relevance could be measured by analysing user clicks on the result link, but what about speed? Is 1s fast or 0.3s or 0.03s? Is it about the actual speed or about speedâs perception by the user? âą Usability studies help to answer those questions by using two types of user research: quantitative (data points) and qualitative (perception) âą Quantitative results allow to draw conclusions approximating them on to the whole target user set, i.e. âlaunching a new banking website will cause no more than 0.1% of online banking users to call the support lineâ âą Qualitative results allow to capture how users feel about their experience with the product/ service âą Good usability studies will use both approaches © 2012 Elena Punskaya 8 Cambridge University Engineering Department 8
9.
Usability Study Design âą
Method - define user interactions that are going to be tested, how the user is instructed and how the system is implemented (real product/ prototype / simulator) - define how the user feedback is captured: interviews, scoring cards - define how user interactions are observed and analysed: video and screen capture, eye tracking âą User group (Know Your Customer!) - choose the user group to provide a representative set of product/ service users ⣠an app for remote access to computers (e.g. SSH client) is likely to be used by experienced computer users ⣠a web conferencing app (e.g.WebEx) need to be easier to setup by business (non-technical) users ⣠a call/chat client (e.g. Skype) should be easy to use by any computer user - segment target users by their characteristics: demographics: age, gender; level of experience... âą Results presentation â a report should include - conclusions based on observations of users by the specialists to identifying e.g. a common source of confusion - qualitative user feedback - quantitative results: rankings/scoring by user groups (charts/tables) © 2012 Elena Punskaya 9 Cambridge University Engineering Department 9
10.
Number of Test
Users âą Jacob Nielsen advocated that most effective approach is multiple usability studies with 5 users at a time http://www.useit.com/alertbox/20000319.html âą The chart above applies to the number of users of the same kind, in reality, if the target user set is diverse, so itâd be better to have 5 users for each user segment © 2012 Elena Punskaya 10 Cambridge University Engineering Department 10
11.
Number of Test
Users âą Laura Faulkner in âBeyond the five-user assumption: Benefits of increased sample sizes in usability testingâ showed that 5 users do not necessarily deliver 85% problem discovery but a set of 20 was certainly very reliable âą In practice, the optimum number can depend on the diversity of target usersFAULKNER complexity of the product 382 and Figure 1. The effect of adding users on reducing variance in the percentage of known usability problems found. Each point represents a single set of randomly sampled users. The horizontal lines show the mean for each group of 100. © 2012 Elena Punskaya 11 Cambridge University Engineering Department use those formulas, such as probabilities, make them im- practitioner to approach increasing levels of certainty practical and misleading for ordinary usability practition- that high percentages of existing usability problems have ers. Although practitioners like simple directive answers been found in testing. In a mission-critical system, large 11
12.
Tests from Different
Perspective âą âTesting Quadrantâ categorises the tests according to whether they are Business-Facing or Technology-Facing and whether they support development process or used to critique (review and analyse the project) âą Brian Marick introduced Testing Quadrant, it was further discussed by Lisa Crispin and Janet Gregory in âAgile Testing: A Practical Guide for Testers and Agile Teamsâ and in âContinuous Deliveryâ by Jez Humble and David Farley © 2012 Elena Punskaya 12 Cambridge University Engineering Department 12
13.
Business-Facing tests Supporting
Development âą Acceptance testing - testing conducted by a customer to verify that the system meets the acceptance criteria of the requested application - ideally should be written and automated before the development starts - ideally should be written by the customers or users - for developers they answer the question: How do I know when I http://www.d80.co.uk/post/2011/03/08/Non-Functional- Requirements-the-forgotten-overlooked-and-underestimated.aspx am done? - for users they answer a question: Acceptance tests that concern the Did I get what I wanted? functionality of the system are - typically should run when the known as Functional acceptance system is in a production-like mode tests â the distinction between Functional and Non-Functional is a bit blurry and often misunderstood © 2012 Elena Punskaya 13 Cambridge University Engineering Department 13
14.
Technology-facing tests Supporting
Development ⹠Technology-facing tests are written and maintained exclusively by developers ⹠Usually these are unit tests, component tests (integration tests) and deployment tests ⹠Deployment tests are performed whenever one deploys the application and check that it is installed correctly, configured correctly, able to contact any services required and is responding © 2012 Elena Punskaya 14 Cambridge University Engineering Department 14
15.
Business-Facing tests that
Critique the Project âą The tests are not just about verifying that the application meets the specification but also about checking that the specification is correct âą Applications are never specified perfectly in advance so there is always room for improvement, users try things they are not supposed to try and break them, complain about usability of most commonly performed tasks, identify new features âą Showcases are particularly important - show new functionality to the customers and users as soon as possible to catch any misunderstandings early (can be a blessing and a curse - you might have a lot of suggestions!) âą Exploratory testing - âthe tester actively controls the design of the tests as those tests are performed and uses information gained while testing to design new and better testsâ [James Bach] âą Beta testing - give your application to real user, release new features to selected groups without them even noticing © 2012 Elena Punskaya 15 Cambridge University Engineering Department 15
16.
Technology-facing tests that
Critique the Project âą Acceptance tests may test functionality (functional acceptance tested) but may also test other qualities of the system such as capacity, usability, security, modifiability, availability, etc. âą All these are qualities other than functionality go under umbrella of nonfunctional tests although the distinction is very blurry, however, what is important is to bother testing them! âą It is also not necessarily fair to say that these tests are not business facing â often they are! © 2012 Elena Punskaya 16 Cambridge University Engineering Department 16
17.
How to Test? âą
Regression Tests - compares the output of the previous test with the previous or known values - make sure that bugs fixed today donât break something else - no unpleasant surprises - can run regression tests to make sure the components function correctly, entire subsystem functions, performance, etc. âą Test Data - real-world - âtypical dataâ collected; âtypicalâ might be a surprise and may reveal misunderstandings in requirements - watch out! - synthetic - artificially generated data (not enough real- data, need certain statistical properties, need to stress the boundary condition) - real-world and synthetic - expose different types of bugs âą GUI testing - need special tools to exercise, canât automate everything © 2012 Elena Punskaya 17 Cambridge University Engineering Department 17
18.
Why teams Choose
to Live with Defects? ⹠Is it possible to create BugFree software? Yes in theory but how one would prove this? ⹠How much does it cost to create BugFree software? Are costs astronomical? ⹠Dilemmas: - bugs are expensive (direct costs of fixing and indirect costs of damaged reputation and relationship, lost time) and eliminating bugs is expensive => economically viable option depends on the acceptable defects level in any particular area - there will always be bugs! (Unknown unknows, unexpected circumstances, new situation) © 2012 Elena Punskaya 18 Cambridge University Engineering Department 18
19.
Horses for Courses âą
Not all SOFTWARE is created equal, choose an engineering approach accordingly Our tests suite takes nine minutes to run (distributed across 30-40 machines). Our code pushes take another six minutes. Since these two steps are pipelined that means at peak weâre pushing a new revision of the code to the website every nine minutes. Thatâs 6 deploys an hour. Even at that pace weâre often batching multiple commits into a single test/push cycle. On average we deploy new code fifty times a day. [1] This software never crashes. It never needs to be re- booted. This software is bug-free. It is perfect, as perfect as human beings have achieved. Consider these stats : the last three versions of the program â each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. [2] © 2012 Elena Punskaya 19 Cambridge University Engineering Department 19
20.
When to Test? âą
The sooner one finds the defect the cheaper it is to fix it (catch it the minute it was created and it costs nothing) ⹠Late testing is expensive and leaves many defects We want to start testing as soon as we have code. Those tiny minnows have a nasty habit of becoming giant, man-eating sharks pretty fast, and catching a shark is quite a bit harder. Andrew Hunt and David Thomas, The Pragmatic Programmer Kent Beck and Cynthia Andres, Extreme Programming Explained: Embrace Change ⹠Frequent testing reduces costs and defects (fix sooner and cheaply) Kent Beck and Cynthia Andres, Extreme Programming Explained: Embrace Change © 2012 Elena Punskaya 20 Cambridge University Engineering Department 20
21.
Who Should Test? âą
Frequent testing (difficult to have a dedicated tester) might mean programmers themselves write the tests, i.e. the same people who make mistakes write the tests - they need another prospective! Buddy programmer? Think like a users? âą Consider two prospectives: programmers and users, involve the users at the very early stages âą Most developers hate testing - they tend to be very gentle (subconsciously) with their creations and tend to avoid the weak spots intuitively. âą Who likes testing??? - testing could be very tedious â given a 100 pages long test script most humans turn on the âzombie modeâ ;) âą Solution? Automate whenever possible (so all the tedious bits are done by the machines)! - But someone still needs to define all test cases and implement them in the automated testing framework - Also, it is difficult to substitute manual testing for capturing unexpected user behaviour, the best testers have a six sense for breaking things :) © 2012 Elena Punskaya 21 Cambridge University Engineering Department 21
22.
Automated Testing âą
The goal is to reach âcoverageâ â cover the most practically possible number of test cases âą It would typically include âą All unit tests â developers implement and commit those to the source code repository during the development process âą Regression tests â given the set of know inputs, does the system still produce the same set of outputs? âą UI testing â cover various screen flows, difficult to automate but increasingly more and more coverage is possible âą Probably enough for a âstandardâ app, but what about large distributed services in the âCloudâ? â Test Automation Cloud - Salesforce.com: average 600 change lists per day - âTest automation cloud provides accurate, complete and fast feedbacks to every change to the system, on a per change list basisâ - Hardware: 2000 Linux Virtual Machines and 1000 Selenium Virtual Machines to run all required tests © 2012 Elena Punskaya 22 Cambridge University Engineering Department 22
23.
Tools for the
Job âą The test tools are growing and maturing rapidly, many contributed to Open Source by big commercial vendors and many are created and evangelised by startups, some as examples are: âą xUnit â a unit testing framework, ported to many languages, including JUnit (Java) and NUnit (C#) âą KIF â âKeep it Functionalâ â a new iOS UI testing framework, allows to specify user interactions for test scenarios and check appâs resulting behaviour (which screen appeared etc.) - http://corner.squareup.com/2011/07/ios-integration-testing.html âą Selenium â âAutomates Browsersâ â allows to simulate interactions and check results with websites - http://seleniumhq.org/ âą Robotium â âItâs like Selenium, but for Androidâ - http://code.google.com/p/robotium/ âą Static code analysers â checking for code inconsistency/ inefficiency, security etc., e.g. FxCop (for .NET) © 2012 Elena Punskaya 23 Cambridge University Engineering Department 23
Baixar agora