1. How to give your Optimisation Program Wings
Frans Poldervaart Tom Waterfall
Web Analytics Manager Director of Optimisation Solutions

3. - 75 country websites
- 23 different languages
- 1100 pages per country
- Largest travel site in
175.000
Netherlands (Twinkle
T100)
transac-ons
per
24%
online
month
KLM.com:
By the numbers
3

4. 4

5. 1. How to get started
PEOPLE DRIVE
SUCCESS

6. 1. How to get started:
Identify Your Goals Upfront
• Increase share in online sales
• Use the most effective acquisition channel
• Increase flying blue members
• Make it easier for customers to navigate and purchase
• Increase ancillary revenue
• One stop shop
• Ensure customers’ experiences are more positive
6
Name of presentation - Version 1.01 dd-mm-2008

7. 1. How to get started
7
Name of presentation - Version 1.01 dd-mm-2008

8. 1. How to get started

9. 1. EXAMPLE: How we took off
1 Control Functional
2
Inspiration Trust
3 4
9

10. 1. EXAMPLE: How we took off
Inspiration
+14.11% +5,400 +€3.8
Conv. Bookings Million
10

11. 1. How to get started: Following Take Off
• Begin Brainstorming test ideas • Use all the tools at your disposal
11

12. 12
Name of presentation - Version 1.01 dd-mm-2008

13. 2. Getting buy-in: Start an illegal gambling ring
13

14. 1. EXAMPLE: Starting an illegal gambling ring
14

15. 2. Getting buy-in: Infiltrate & Cultivate
15

16. 2. Getting buy-in: Shout about results & learning
16

17. 2. EXAMPLE: Shout about results & learning
17

18. 2. EXAMPLE: Shout about results & learning
+23%
Lift
18

19. © 2011 Webtrends, All Rights Reserved. | 19

20. 3. Planning & Prioritization: Companies struggle
20

21. 3. Planning: Focus on quick wins
This chart is based on tentative information
presented, and elements might move around
based on new data available. Starbursts indicate Booking Details page tests HIGH
revenue lift potential.
HIGH
Booking Details page tests
Simple Complex
Changes to structure, arrangement or
MID functionality
Search results page functionality
Tests with filters, modifying search
MID
Strikethrough pricing
Affecting display, but only doable during sales
Exceptions testing Search results page display
? No results pages etc.
MID Promo codes, supplier logos,
messaging
Across the site
? Inclusions, messaging of any kind
Homepage tests
Any display/messaging
HIGH
Related tests
Revenue Potential
21

22. 3. EXAMPLE: ROI Potential vs. Complexity
22

23. 3. Planning: Prioritise a mix of simple & complex
23

24. 3. EXAMPLE: Mixing Simple and Complex
+2.6%
Lift

25. 3. EXAMPLE: Mixing Simple and Complex
+39%
Lift

27. 3. EXAMPLE: Mix & Match
Control
27

28. 3. EXAMPLE: Mixing Simple and Complex
Level 1 Level 2 Level 3 Level 4
(Control)
Lift / Booking
‘Next step’ Lift /
Level Chance to Beat Control Confirmation
conversion rate Chance to Beat Control
Rate
1 33.23% --- 3.41% ---
All alternate button copies drove significantly more visitors to the next
+5.05%
2 step 34.91%also had a chance to beat impact on Bookings.
and (99.43% positive 3.21% -5.84%
(Not Significant)
control)
+5.48% +8.64%
3 35.05% (99.64% chance to beat 3.71%
(85%% chance to beat control)
control)
+8.69% +6.04%
4 36.12% (99.79% chance to beat 3.62% (Not Significant chance to beat
control) control)
28

29. 3. EXAMPLE: Mixing Simple and Complex
Level 1
(Control)
Level 2
Underlining City and Price and making the whole row clickable
Lift /
is driving significantly more visitors to theConfirmation
Level
‘Next step’ Chance to Beat next step,Chance tois
Booking
and / Beat
Lift
conversion rate Control
showing promise with Bookings Rate Control
1 33.76% --- 3.37% ---
6.47% +6.69%
2 35.95% (99.99% chance to beat 3.60% (87% chance to beat
control) control)
29

30. 3. EXAMPLE: Mixing Simple and Complex
+150
Bookings
+6.69%
Lift
The ‘Clickable Row’ had the highest impact at the
booking confirmation rate, with a +6.69% lift
30

31. 3. Planning: Mix & Match between markets/journeys
US Traffic Rest of
Bookings World
Car Hire English
Spanish
Loyalty
Registration
Email Sign-Up Mobile / Tablet Site
31

32. 3. Planning: Revisit priorities quarterly
32

33. © 2011 Webtrends, All Rights Reserved. | 33

34. 4. EXAMPLE: EU Cookie Law Test  Background
34

35. 4. EXAMPLE: EU Cookie Law Test  Background
Control Level 4
Optimal
Experiment Conversion Confirmation page Conversion
Run Date Click button
Page Lift (confidence interval) Lift
Control - --- 2.98% (+/- 0.21%) ---
12/11/2012 –
26/11/2012
Optimal 2.43% (+/- 0.07%) - 3.09% (+/- 0.21%) +3.64%
35

36. 4. EXAMPLE: Mobile & Tablet
“My unborn nephew owns a
smartphone,” says person on street.
36

37. 4. EXAMPLE: Mobile & Tablet – make a case
+61.32%
“Despite the [rapid Search
growth of mobile] , 69
companies in the FTSE
100 had done nothing
to “optimise” their
websites for mobiles…”
37

38. 4. EXAMPLE: Mobile & Tablet – fix it
38

39. 4. Be Thoughtful: Track more, Learn more
39

40. 4. EXAMPLE: Track more, Learn More
When the “Filter by Location” option was
amended to a dropdown…
…visitors used the ‘Star Rating’, ‘Filter by
Board’, and Map tools significantly more.
Explanation: the dropdown takes up far
less real estate and therefore places
further emphasis on other filters.
Additional Insight: the increased usage of
star rating, by board and the map tool
correlated positively with an increase in
hotels booked.
Recommendation: continue to test filter
ordering, emphasis, and style to identify
the real drivers behind hotel bookings.
40

41. 5. The Path Ahead for
• From 0.5 FTE to 2 FTE dedicated to the conversion
optimization program
• Large and complex testing continuing with full
support from experts
• Small tests conceived and built by KLM
• Increase communication within the organization
• Proactive testing associated with site developments
• Responsive design
• New applications
41

42. Lessons Learned
• Takeoff requires a central owner and initial testing that
focuses on business value
• Attain departmental buy-in through a healthy mix of sharing
and gambling
• Plan ahead, and revisit your plans regularly
• Keep it simple and look at all your journeys, markets and
goals
• Keep your program fresh through innovation and relevance
• Once you’ve taken off, don’t plan on ever landing!

45. © 2011 Webtrends, All Rights Reserved. | 45