Featuring speakers from Rue La La, Princess Cruises
Abby Lokshin, Product Manager, Mobile, Rue La La
Abel Ramos, Product Owner, Shopping and Booking Web Scrum Team, Princess Cruises
Testing is an iterative process, oftentimes with confusing or inconclusive results. However, every experiment has something to teach you, and all too often, critical information is left unnoticed! Learn how to look for insights hidden behind the numbers and maximize the utility of every experiment you run.
2. #opticon2015
Iteration – The Key to
your Testing Success
Abel Ramos
Product Manager, Ecommerce Princess Cruises
@abelramos58
abelramos@princesscruises.com
27. TEST PRIORITIZATION
60 SECOND BUSINESS CASE
Coordination Effort
Creative
Effort
Initial
Development
Effort
Rollout Effort
Strategic
Alignment
Revenue
Impact
Customer
Experience
Value
Lowers
Cost
PM Priority
RANK: HIGH, MEDIUM, LOW
SCORE: 100 PTS. TOTAL
+
EFFORT
BUSINESS
VALUE
30. EXPIRING FREE SHIPPING TEST
TESTING GOALS:
• USER SEES QUICK TIP
• USER ADDS TO CART
• USER CHECKS OUT
• PRIMARY GOAL: REVENUE
RESULTS: 4.2%+ CONVERSION
NEXT STEPS: ROLL OUT!
QUICK TIP ON PRODUCT DETAIL
LEVEL OF EFFORT: LOW
BUSINESS VALUE: HIGH
32. MEN’S SUIT SHOP
MIX & MATCH BOUTIQUE
TESTING GOALS:
• USER BUILDS A SUIT
• USER ADDS TO CART
• USER CHECKS OUT
• PRIMARY GOAL: REVENUE
RESULTS: NO CLEAR WINNER
37. CATEGORY SORT TEST
SHOPPING PREFERENCES
TESTING GOALS:
• USER INTERACTS WITH
SORTING
• USER CHECKS OUT
• PRIMARY GOAL: REVENUE
RESULTS: NO CLEAR WINNER
VAR. A VAR. B
38. CATEGORY SORT TEST
WHAT HAPPENED
NO IMPACT ON THE WHOLE
MANY CATEGORY COMBINATIONS
CAN WE FIND ANY TRENDS?
VAR. A VAR. B
39. CATEGORY SORT TEST
INTERESTING TRENDS
TREND IN VARIATION A:
FEMALES RANK WOMEN’S FIRST
• 4%+ IN $/VISIT
TREND IN VARIATION B:
MALES RANK MEN’S FIRST
• 31%+ IN $/VISIT
• 32%+ IN PURCHASES
VAR. A VAR. B
40. CATEGORY SORT TEST
INTERESTING TRENDS
NEW TESTS:
• DEFAULT MEN TO MEN’S
SECTION
• RE-OPEN TO LAST VIEWED
CATEGORY
VAR. A VAR. B
41. REMEMBER…
TESTING IS NEVER ONE AND DONE
CONSIDER THE DATA
COLLECT FEEDBACK
REMEMBER THE PRIMARY GOAL
KEEP ITERATING