SlideShare uma empresa Scribd logo
1 de 88
OPTIMIZATION QA:
How to Avoid
Launching Tests that
Break Your Site
When you break shit you
pollute data, lose money
and eventually get fired.
Here’s how to avoid that
pain and look like a hero in
the process.
@joeljharvey #convcon
Joel Harvey, COO
Conversion Sciences
@joeljharvey #convcon
Goal
@joeljharvey #convcon
What’s QA?
@joeljharvey #convcon
Let’s say that …
@joeljharvey #convcon
30% of your
visitors
get a broken
experience on
your site
@joeljharvey #convcon
Y
O
U
R
SITE IS
BROKEN ON ONE OF
T
H
E
S
E
SCREENS
@joeljharvey #convcon
No One On Your
Team Knows
@joeljharvey #convcon
20% RPV
IE BUG
@joeljharvey #convcon
INTERNET
EXPLORER
@joeljharvey #convcon
You’re about to hire this guy as your CPA
@joeljharvey #convcon
QA MATH
@joeljharvey #convcon
Internet Explorer Traffic: Feb 2013
Internet Explorer Traffic: Feb 2014
@joeljharvey #convcon
 120% Increase Internet Explorer (IE)
Revenue
 20% Increase in All Revenue
 20% Increase in IE Average Order Value
 167% Increase in IE Conversion Rate
$1,500,000 Per Year
@joeljharvey #convcon
Who Are
You?
@joeljharvey #convcon
Who Am I?
- COO & Managing Partner,
Conversion Sciences
- Conversion Scientist
- Data Geek
- Entrepreneurial MBA
@joeljharvey #convcon
RE-DEFINING QA
#1
@joeljharvey #convcon
STOP STEALING FROM YOURSELF
@joeljharvey #convcon
30% of your
visitors
get a broken
experience on
your site
QA DRIVES
REVENUE
@joeljharvey #convcon
OLD BORING
DEFINITION
@joeljharvey #convcon
@joeljharvey #convcon
QA is just a Cost
Center
We can’t
support
everything!
QA prevents us
from being agile
People on really old
devices probably
aren’t buyers
QA is an IT /
Development Task
@joeljharvey #convcon
QA IS A
MARKETING
FUNCTION
@joeljharvey #convcon
QA STRATEGY
#2
@joeljharvey #convcon
INFUSE QA INTO YOUR OPTIMIZATION DNA
@joeljharvey #convcon
1. CULTURE
3. TECH PARADIGMS
2. REAL USER USE CASES
QA IS
OPTIMIZATION
@joeljharvey #convcon
1. Understand your visitors
@joeljharvey #convcon
2. Make sure control is not
broken
@joeljharvey #convcon
3. Generate great hypotheses &
test them
@joeljharvey #convcon
@joeljharvey #convcon
4. Don’t
pollute
your test
data
5. Do what the data tells you to
do
@joeljharvey #convcon
BUILDING A
QA CULTURE
@joeljharvey #convcon
@joeljharvey #convcon
#QAHAMMER
@joeljharvey #convcon
Build Destroy
HAMMER UP
@joeljharvey #convcon
BUY IN
@joeljharvey #convcon
@joeljharvey #convcon
@joeljharvey #convcon
20% RPV
IE BUG
@joeljharvey #convcon
REAL USE
CASES
@joeljharvey #convcon
90%
@joeljharvey #convcon
VISITORS
DON’T USE
YOUR SITE THE
WAY YOU
THINK THEY DO
@joeljharvey #convcon
@joeljharvey #convcon
SCREEN RECORDING
@joeljharvey #convcon
USER TESTING
DOCUMENT
AND QA ALL
USE CASES
@joeljharvey #convcon
YOUR JOB IS TO
BREAK YOUR
SITE
@joeljharvey #convcon
@joeljharvey #convcon
WHAT
SHOULD YOU
SUPPORT?
@joeljharvey #convcon
5%
@joeljharvey #convcon
@joeljharvey #convcon
Old New
BROWSER SUPPORT
 Internet Explorer 8
 Chrome 7
 Firefox 4
 Safari 4.1
 iOS 5.1 *
 Internet Explorer 11
 Chrome 33
 Firefox 27
 Safari 7.0
 iOS 7.1 *
@joeljharvey #convcon
OS SUPPORT
Old New
 Windows XP
 OS X 10.4 (Tiger)
 Android 2.3
 iOS 5.1
 Windows Phone 7
 Windows 8
 OS X 10.9 (Maverick)
 Android 4.4
 iOS 7.1
 Windows Phone 8.1
5% MINIMUM
@joeljharvey #convcon
@joeljharvey #convcon
@joeljharvey #convcon
@joeljharvey #convcon
@joeljharvey #convcon
@joeljharvey #convcon
@joeljharvey #convcon
@joeljharvey #convcon
@joeljharvey #convcon
@joeljharvey #convcon
QA TOOLS
#4
@joeljharvey #convcon
BUILD YOUR QA’TION STATION
@joeljharvey #convcon
SEE YOUR
SITES LIKE
YOUR
VISITORS
iOS 7.1?
@joeljharvey #convcon
@joeljharvey #convcon
SOFTWARE
vs.
HARDWARE
@joeljharvey
#convcon
iPad,
iOS 5
iPad,
iOS
7.1
Windows
Vista
Android 4.3
iPhones:
iOS4, 5, 7
Android:
2.3.6, 4.0.4,
Windows
8, no
updates
OS X 10.6 Windows 8,
regular updates
+ VM’s
OS X 10.8
OS X
10.9
OS X 10.7
+ VM’s
Windows XP
Windows
7
@joeljharvey #convcon
@joeljharvey #convcon
?
@joeljharvey
#convcon
1. Browser
2. Operating System
3. “Other”
- Visitor Internet Speed
- Browser Plugins
- Flash
- Resolution
QA’TION STATION CHECKLIST
QA PROCESS
#4
@joeljharvey #convcon
BE THE HAMMER
QA
EXPERIENCES,
NOT PAGES
@joeljharvey #convcon
@joeljharvey #convcon
@joeljharvey #convcon
@joeljharvey #convcon
TEST QA
PROCESS
@joeljharvey #convcon
1.
Build Your Test
@joeljharvey #convcon
2.
Use Cookie, Query
String and/or Location
Targeting so Only You
See Experience
@joeljharvey #convcon
3.
Pause / Disable All
But One Experience
@joeljharvey #convcon
4.
Review Live
Experience In All
Supported
Browser/OS Combos
(like REAL USERS)
@joeljharvey #convcon
5.
Clear Cookies, Pause
Good-To-Go
Experience and
Unpause the Next One
@joeljharvey #convcon
6.
Repeat
@joeljharvey #convcon
A FEW TIPS
RANDOMIZE
YOUR
APPROACH
CAUTION WITH
CLASSES
WATCH SCRIPT
STACKS
FEAR THE
WYSIWYG
OWNERSHIP
@joeljharvey #convcon
Need to Borrow a QA Hammer?
Call 1-888-961-6604
and mention this
session for a chance
to win a Free QA
Audit ($15,000 Value)
Conversion Conference San Francisco, 2014
Joel Harvey, Conversion Sciences
Joel@ConversionSciences.com

Mais conteúdo relacionado

Semelhante a QA: How I Learned to Stop Breaking and Start Loving Tests

Semelhante a QA: How I Learned to Stop Breaking and Start Loving Tests (20)

3rd Party Footprint @ Webrebels
3rd Party Footprint @ Webrebels3rd Party Footprint @ Webrebels
3rd Party Footprint @ Webrebels
 
Why Are We Doing Cro? What Are We Doing Here? - Paul Rouke at Conversion Conf...
Why Are We Doing Cro? What Are We Doing Here? - Paul Rouke at Conversion Conf...Why Are We Doing Cro? What Are We Doing Here? - Paul Rouke at Conversion Conf...
Why Are We Doing Cro? What Are We Doing Here? - Paul Rouke at Conversion Conf...
 
Visual Studio Mobile Center: A story about mobile DevOps
Visual Studio Mobile Center: A story about mobile DevOpsVisual Studio Mobile Center: A story about mobile DevOps
Visual Studio Mobile Center: A story about mobile DevOps
 
Shift left-devoxx-pl
Shift left-devoxx-plShift left-devoxx-pl
Shift left-devoxx-pl
 
BSides LA/PDX
BSides LA/PDXBSides LA/PDX
BSides LA/PDX
 
The New Renaissance of JavaScript - SMX London 2019
The New Renaissance of JavaScript - SMX London 2019The New Renaissance of JavaScript - SMX London 2019
The New Renaissance of JavaScript - SMX London 2019
 
The New Renaissance of JavaScript - SMX London 2019
The New Renaissance of JavaScript - SMX London 2019The New Renaissance of JavaScript - SMX London 2019
The New Renaissance of JavaScript - SMX London 2019
 
Data driven devops as presented at Codemash 2018
Data driven devops as presented at Codemash 2018Data driven devops as presented at Codemash 2018
Data driven devops as presented at Codemash 2018
 
10 Rules for Landing Page Copy That Converts
10 Rules for Landing Page Copy That Converts10 Rules for Landing Page Copy That Converts
10 Rules for Landing Page Copy That Converts
 
SearchLove Boston 2018 - Emily Grossman - The Marketer’s Guide to Performance...
SearchLove Boston 2018 - Emily Grossman - The Marketer’s Guide to Performance...SearchLove Boston 2018 - Emily Grossman - The Marketer’s Guide to Performance...
SearchLove Boston 2018 - Emily Grossman - The Marketer’s Guide to Performance...
 
Contributing to StrongLoop LoopBack (and other open source projects)
Contributing to StrongLoop LoopBack (and other open source projects)Contributing to StrongLoop LoopBack (and other open source projects)
Contributing to StrongLoop LoopBack (and other open source projects)
 
The Technical Seo Renaissance - Mike King
 The Technical Seo Renaissance - Mike King   The Technical Seo Renaissance - Mike King
The Technical Seo Renaissance - Mike King
 
Distance from Perfect: SEO and PPC Common Sense
Distance from Perfect: SEO and PPC Common SenseDistance from Perfect: SEO and PPC Common Sense
Distance from Perfect: SEO and PPC Common Sense
 
Codemotion Progressive Web Applications Pwa Webinar - Jorge Ferreiro - @jgfer...
Codemotion Progressive Web Applications Pwa Webinar - Jorge Ferreiro - @jgfer...Codemotion Progressive Web Applications Pwa Webinar - Jorge Ferreiro - @jgfer...
Codemotion Progressive Web Applications Pwa Webinar - Jorge Ferreiro - @jgfer...
 
Building the future as a full stack dev
Building the future as a full stack devBuilding the future as a full stack dev
Building the future as a full stack dev
 
DevOps Patterns & Antipatterns for Continuous Software Updates @ NADOG April ...
DevOps Patterns & Antipatterns for Continuous Software Updates @ NADOG April ...DevOps Patterns & Antipatterns for Continuous Software Updates @ NADOG April ...
DevOps Patterns & Antipatterns for Continuous Software Updates @ NADOG April ...
 
Panoramic Quality: The Fellowship of Testing in DevOps
Panoramic Quality: The Fellowship of Testing in DevOpsPanoramic Quality: The Fellowship of Testing in DevOps
Panoramic Quality: The Fellowship of Testing in DevOps
 
Panoramic Quality by Brendan Connolly
Panoramic Quality by Brendan ConnollyPanoramic Quality by Brendan Connolly
Panoramic Quality by Brendan Connolly
 
Cloudbrew 2019 observability driven development
Cloudbrew 2019   observability driven developmentCloudbrew 2019   observability driven development
Cloudbrew 2019 observability driven development
 
Devoxx 2014 Monitoring
Devoxx 2014 Monitoring Devoxx 2014 Monitoring
Devoxx 2014 Monitoring
 

Mais de Brian Massey

What We Can Learn from Direct Marketers by Brian Massey and Debra Zahay
What We Can Learn from Direct Marketers by Brian Massey and Debra ZahayWhat We Can Learn from Direct Marketers by Brian Massey and Debra Zahay
What We Can Learn from Direct Marketers by Brian Massey and Debra Zahay
Brian Massey
 

Mais de Brian Massey (20)

Enterpreneurs Don't Guess: Building an MVNP
Enterpreneurs Don't Guess: Building an MVNPEnterpreneurs Don't Guess: Building an MVNP
Enterpreneurs Don't Guess: Building an MVNP
 
Avoiding Disaster: What a Successful Website Redesign Looks Like
Avoiding Disaster: What a Successful Website Redesign Looks LikeAvoiding Disaster: What a Successful Website Redesign Looks Like
Avoiding Disaster: What a Successful Website Redesign Looks Like
 
The Unexpected Website Formulas of The Conversion Scientist™
The Unexpected Website Formulas of The Conversion Scientist™The Unexpected Website Formulas of The Conversion Scientist™
The Unexpected Website Formulas of The Conversion Scientist™
 
Redesign Your Website for Leads and Profit
Redesign Your Website for Leads and ProfitRedesign Your Website for Leads and Profit
Redesign Your Website for Leads and Profit
 
Optimizing the Blog Volcano
Optimizing the Blog VolcanoOptimizing the Blog Volcano
Optimizing the Blog Volcano
 
Luxury Products Landing Page Critique
Luxury Products Landing Page CritiqueLuxury Products Landing Page Critique
Luxury Products Landing Page Critique
 
Attempting to Predict the Unpredictable
Attempting to Predict the Unpredictable Attempting to Predict the Unpredictable
Attempting to Predict the Unpredictable
 
Turn Recorded Webinars into Sharable, Lead Generating Content
Turn Recorded Webinars into Sharable, Lead Generating ContentTurn Recorded Webinars into Sharable, Lead Generating Content
Turn Recorded Webinars into Sharable, Lead Generating Content
 
How Amazon Crushes the Competition-Bryan Eisenberg
How Amazon Crushes the Competition-Bryan EisenbergHow Amazon Crushes the Competition-Bryan Eisenberg
How Amazon Crushes the Competition-Bryan Eisenberg
 
The Chemistry of a Successful Landing Page Infographic
The Chemistry of a Successful Landing Page InfographicThe Chemistry of a Successful Landing Page Infographic
The Chemistry of a Successful Landing Page Infographic
 
Instagraph of Craig Sullivan's Presentation at ConversionSUMMIT 2013
Instagraph of Craig Sullivan's Presentation at ConversionSUMMIT 2013Instagraph of Craig Sullivan's Presentation at ConversionSUMMIT 2013
Instagraph of Craig Sullivan's Presentation at ConversionSUMMIT 2013
 
Killer Conversion Copy: Getting Past the Bouncers in Your Brain
Killer Conversion Copy: Getting Past the Bouncers in Your BrainKiller Conversion Copy: Getting Past the Bouncers in Your Brain
Killer Conversion Copy: Getting Past the Bouncers in Your Brain
 
What We Can Learn from Direct Marketers by Brian Massey and Debra Zahay
What We Can Learn from Direct Marketers by Brian Massey and Debra ZahayWhat We Can Learn from Direct Marketers by Brian Massey and Debra Zahay
What We Can Learn from Direct Marketers by Brian Massey and Debra Zahay
 
Optimizing With Online Buyer Personas
Optimizing With Online Buyer PersonasOptimizing With Online Buyer Personas
Optimizing With Online Buyer Personas
 
Chemistry of Conversion by Brian Massey
Chemistry of Conversion by Brian MasseyChemistry of Conversion by Brian Massey
Chemistry of Conversion by Brian Massey
 
Web Analytics: Tools and Best Practices
Web Analytics: Tools and Best PracticesWeb Analytics: Tools and Best Practices
Web Analytics: Tools and Best Practices
 
Conversion Science Brian Massey and Heather Lloyd Martin
Conversion Science Brian Massey and Heather Lloyd MartinConversion Science Brian Massey and Heather Lloyd Martin
Conversion Science Brian Massey and Heather Lloyd Martin
 
Killing Brad Pitt: Define and Understand Your Visitors
Killing Brad Pitt: Define and Understand Your VisitorsKilling Brad Pitt: Define and Understand Your Visitors
Killing Brad Pitt: Define and Understand Your Visitors
 
Killing Brad Pitt: Why Buyers Fail to Take Action on Your Web Site
Killing Brad Pitt: Why Buyers Fail to Take Action on Your Web SiteKilling Brad Pitt: Why Buyers Fail to Take Action on Your Web Site
Killing Brad Pitt: Why Buyers Fail to Take Action on Your Web Site
 
Search + Conversion = Results
Search + Conversion = ResultsSearch + Conversion = Results
Search + Conversion = Results
 

Último

Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...
Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...
Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...
Sheetaleventcompany
 
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabiunwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
Abortion pills in Kuwait Cytotec pills in Kuwait
 
0183760ssssssssssssssssssssssssssss00101011 (27).pdf
0183760ssssssssssssssssssssssssssss00101011 (27).pdf0183760ssssssssssssssssssssssssssss00101011 (27).pdf
0183760ssssssssssssssssssssssssssss00101011 (27).pdf
Renandantas16
 
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876
dlhescort
 
Call Girls In Noida 959961⊹3876 Independent Escort Service Noida
Call Girls In Noida 959961⊹3876 Independent Escort Service NoidaCall Girls In Noida 959961⊹3876 Independent Escort Service Noida
Call Girls In Noida 959961⊹3876 Independent Escort Service Noida
dlhescort
 
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
amitlee9823
 

Último (20)

Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...
Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...
Chandigarh Escorts Service 📞8868886958📞 Just📲 Call Nihal Chandigarh Call Girl...
 
Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023
 
Phases of Negotiation .pptx
 Phases of Negotiation .pptx Phases of Negotiation .pptx
Phases of Negotiation .pptx
 
MONA 98765-12871 CALL GIRLS IN LUDHIANA LUDHIANA CALL GIRL
MONA 98765-12871 CALL GIRLS IN LUDHIANA LUDHIANA CALL GIRLMONA 98765-12871 CALL GIRLS IN LUDHIANA LUDHIANA CALL GIRL
MONA 98765-12871 CALL GIRLS IN LUDHIANA LUDHIANA CALL GIRL
 
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabiunwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
 
0183760ssssssssssssssssssssssssssss00101011 (27).pdf
0183760ssssssssssssssssssssssssssss00101011 (27).pdf0183760ssssssssssssssssssssssssssss00101011 (27).pdf
0183760ssssssssssssssssssssssssssss00101011 (27).pdf
 
Falcon Invoice Discounting platform in india
Falcon Invoice Discounting platform in indiaFalcon Invoice Discounting platform in india
Falcon Invoice Discounting platform in india
 
B.COM Unit – 4 ( CORPORATE SOCIAL RESPONSIBILITY ( CSR ).pptx
B.COM Unit – 4 ( CORPORATE SOCIAL RESPONSIBILITY ( CSR ).pptxB.COM Unit – 4 ( CORPORATE SOCIAL RESPONSIBILITY ( CSR ).pptx
B.COM Unit – 4 ( CORPORATE SOCIAL RESPONSIBILITY ( CSR ).pptx
 
Dr. Admir Softic_ presentation_Green Club_ENG.pdf
Dr. Admir Softic_ presentation_Green Club_ENG.pdfDr. Admir Softic_ presentation_Green Club_ENG.pdf
Dr. Admir Softic_ presentation_Green Club_ENG.pdf
 
It will be International Nurses' Day on 12 May
It will be International Nurses' Day on 12 MayIt will be International Nurses' Day on 12 May
It will be International Nurses' Day on 12 May
 
RSA Conference Exhibitor List 2024 - Exhibitors Data
RSA Conference Exhibitor List 2024 - Exhibitors DataRSA Conference Exhibitor List 2024 - Exhibitors Data
RSA Conference Exhibitor List 2024 - Exhibitors Data
 
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876
Call Girls in Delhi, Escort Service Available 24x7 in Delhi 959961-/-3876
 
Call Girls In Noida 959961⊹3876 Independent Escort Service Noida
Call Girls In Noida 959961⊹3876 Independent Escort Service NoidaCall Girls In Noida 959961⊹3876 Independent Escort Service Noida
Call Girls In Noida 959961⊹3876 Independent Escort Service Noida
 
Business Model Canvas (BMC)- A new venture concept
Business Model Canvas (BMC)-  A new venture conceptBusiness Model Canvas (BMC)-  A new venture concept
Business Model Canvas (BMC)- A new venture concept
 
Katrina Personal Brand Project and portfolio 1
Katrina Personal Brand Project and portfolio 1Katrina Personal Brand Project and portfolio 1
Katrina Personal Brand Project and portfolio 1
 
BAGALUR CALL GIRL IN 98274*61493 ❤CALL GIRLS IN ESCORT SERVICE❤CALL GIRL
BAGALUR CALL GIRL IN 98274*61493 ❤CALL GIRLS IN ESCORT SERVICE❤CALL GIRLBAGALUR CALL GIRL IN 98274*61493 ❤CALL GIRLS IN ESCORT SERVICE❤CALL GIRL
BAGALUR CALL GIRL IN 98274*61493 ❤CALL GIRLS IN ESCORT SERVICE❤CALL GIRL
 
Call Girls In Panjim North Goa 9971646499 Genuine Service
Call Girls In Panjim North Goa 9971646499 Genuine ServiceCall Girls In Panjim North Goa 9971646499 Genuine Service
Call Girls In Panjim North Goa 9971646499 Genuine Service
 
Value Proposition canvas- Customer needs and pains
Value Proposition canvas- Customer needs and painsValue Proposition canvas- Customer needs and pains
Value Proposition canvas- Customer needs and pains
 
Pharma Works Profile of Karan Communications
Pharma Works Profile of Karan CommunicationsPharma Works Profile of Karan Communications
Pharma Works Profile of Karan Communications
 
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
 

QA: How I Learned to Stop Breaking and Start Loving Tests