Deck from speech at Big Data Innovation Summit 2014. Being a web company whose revenue comes from offline subscription by agents we have long struggled to embed Web Analytics into our business processes. The big change came when we started overlaying the classic Quantitative analytics with Qualitative input from our real users.

1. Representing
Tomas Vaitulevicius
@earnedmarketing
Rightmove
the User

2. Who works for a company
where Analytics say jump and
the Business asks how high?
Well… Neither do I

3. Top 10 UK website
100m Visits
1.5bn Page Views
60%-80% market
share of UK users
73% profit margin
90% market
penetration

4. 10k+ distinct customers
Revenue more
sensitive to sales and
customer service
Varied levels of
value measurement
among customers
Long term fixed
subscription
contracts

5. < 2008 –
Log counting
The history of Analytics at Rightmove – before 2008 all we had was scripts counting access logs

6. > 2008 –
Move to Web Analytics
REPORTING | DASHBOARDS
| INSIGHTS
We then moved into Click-Stream Analytics and tried to deliver actionable insights

7. But for some reason we were creating little change / action

8. It wasn’t because of
people not caring!
PRIDE| EXTRA-MILE |
COMPETITIVE
People were interested in analytics and passionate about improving what they were
doing, but couldn’t relate to the data and weren’t inspired to act

9. Two Routes to Persuasion
Central Peripheral
Extremely
powerful
Requires
• High involvement
• Ability to process
• Clear facts in your favour
Somewhat weaker
to trigger
immediate action
Requires
None of that
Understanding and internalising stats is hard – that’s what analysts are for.
Getting everyone to act on data requires the addition of anecdotal & emotive drive!

10. Finding the
Passion
We found the passion that ignites change through overlaying the hard data
with direct input from real users of our product

11. Getting user input
Usability
Studies
Voice-of-customer
Session
Reviews
Surveys
Web
MVTs analytics

12. Usability Studies – Acumen Fieldwork
For £100 Acumen can source for you a specific type of user for an hour’s usability session.
This is very qualitative, but your user telling you why you suck is as emotive as it gets!
Usability
Studies
Voice-of-customer
Session
Reviews
Surveys MVTs
Web
analytics

13. Usability
Studies
Voice-of-customer
Session
Reviews
Surveys MVTs
Web
analytics
Remote Usability Testing - GoToMeeting
And you don’t even need to get people into your office – online/remote usability is great too

14. Voice-of-customer
Usability
studies
Session
Reviews
Surveys MVTs
Web
analytics
User Feedback - UserVoice
UserVoice is great for getting a sense of relative priority among the desires of your users.
It is also the perfect test to your company’s commitment to customer-centricity.

15. Voice-of-customer
Usability
studies
Session
Reviews
Surveys MVTs
Web
analytics
Social Listening – All Social Networks
Cheap, Quick & Public

16. “Digital experience (session) replay has emerged as
the most effective method in helping to understand
the customer experience, with three in five companies
considering it ‘very’ effective. However, only a quarter
(28%) of companies use this method.
Digital experience replay is also considered the most
effective method for identifying problems or issues with
the digital experience, with the proportion of
organisations saying this is a ‘very’ effective method
increasing by 10% in the last 12 months.“
Econsultancy
“Reducing Customer Struggle 2013”
Session
reviews
Usability
studies
Surveys MVTs
Web
analytics
Voice-of-customer

17. “Digital experience (session) replay has emerged as
the most effective method in helping to understand
the customer experience, with three in five companies
considering it ‘very’ effective. However, only a quarter
(28%) of companies use this method.
Digital experience replay is also considered the most
effective method for identifying problems or issues with
the digital experience, with the proportion of
organisations saying this is a ‘very’ effective method
increasing by 10% in the last 12 months.“
Econsultancy
“Reducing Customer Struggle 2013”
Session
reviews
Usability
studies
Surveys MVTs
Web
analytics
Voice-of-customer

18. “Digital experience (session) replay has emerged as
the most effective method in helping to understand
the customer experience, with three in five companies
considering it ‘very’ effective. However, only a quarter
(28%) of companies use this method.
Digital experience replay is also considered the most
effective method for identifying problems or issues with
the digital experience, with the proportion of
organisations saying this is a ‘very’ effective method
increasing by 10% in the last 12 months.“
Econsultancy
“Reducing Customer Struggle 2013”
Session
reviews
Usability
studies
Surveys MVTs
Web
analytics
Voice-of-customer

19. Session Replay (Server-side Recording) - Tealeaf
Session
reviews
Usability
studies
Surveys MVTs
Web
analytics
Voice-of-customer

20. Session Replay (Server Logs) - Spyglass
Spyglass is our internal “replay” tool which is due to be released to the public
Session
reviews
Usability
studies
Surveys MVTs
Web
analytics
Voice-of-customer

21. Session Replay (Browser-side recording) - GhostRec
Session
reviews
Usability
studies
Surveys MVTs
Web
analytics
Voice-of-customer

22. Session
reviews
Usability
studies
Surveys MVTs
Web
analytics
Voice-of-customer
Heat Mapping

23. NPS - Vovici
Surveys
Usability
studies
MVTs
Web
analytics
Voice-of-customer
Session
reviews

24. Qualaroo
Usability
studies
This is where qualitative really meets quantitative
MVTs
Web
analytics
Voice-of-customer
Session
reviews
Surveys

25. 5 Second test
Describe the website in three words. This immediately started a website redesign project!
Usability
studies
MVTs
Web
analytics
Voice-of-customer
Session
reviews
Surveys

26. 5 Second test
And the follow-up gave us confidence that we were moving in the right direction
Usability
studies
MVTs
Web
analytics
Voice-of-customer
Session
reviews
Surveys

27. Usability
studies
Web
analytics
MVTs (Multivariate Tests)
MVT - the ultimate test of User Preference.
In our first test ever we wanted more people to contact our agents…
Voice-of-customer
Session
reviews Surveys MVTs

28. MVTs
Usability
studies
Web
analytics
…and tested a (semi-random) variety of Call-to-Action buttons
Voice-of-customer
Session
reviews Surveys MVTs

29. MVTs
Usability
studies
Web
analytics
The colour did make a little bit of a difference…
Voice-of-customer
Session
reviews Surveys MVTs

30. MVTs
…but it was the text that created a step change through shifting the meaning of the CTA
Usability
studies
Web
analytics
Voice-of-customer
Session
reviews Surveys MVTs

31. Big impact requires big
changes
Usability
studies
Web
analytics
Only testing small tweaks expect small changes in user behaviour
Voice-of-customer
Session
reviews Surveys MVTs

32. Big impact requires big
changes
Only testing variations that you think will increase the performance guarantees
Usability
studies
Web
analytics
We don’t know better
that you miss out on big impact findings!
Voice-of-customer
Session
reviews Surveys MVTs

33. Usability
studies
Web
analytics
Voice-of-customer
Session
reviews Surveys MVTs
Factor
Motivation
Aspects
Quick and easy
Not missing out
Fear of irrelevant
information/trust
Register with multiple
agents – buy/rent
Can’t find what you’re
looking for
Factor
Visibility
Aspects
Location
Presentation
Factor
Usability
Aspects
Legibility / descriptiveness
Ease of use
We now use a framework of identifying Factors > Aspects > Variations to ensure that we cover
all the factors/aspects that might impact performance instead of just testing 20 different colours

34. This is the easy part…
Usability
studies
Voice-of-customer
Session
reviews Surveys Web
analytics
MVTs

35. Questions?
Usability
studies
Voice-Session
reviews Surveys Web
analytics
Session Surveys MMVVTTss
reviews
Voice-of-customer
Usability
Studies