Chinwag Psych London 2014 @Chinwag psychmatters.co Craig Sullivan, Optimal Visit Ltd. “Neuro Marketing is Bulls***t”

1. Neuro Marketing is Bulls**t
@OptimiseOrDieChinwag Psych 05/14

3. 3Bill & Ted

4. 4
The Invisible Gorilla

5. The Neuro Marketing Bullshit Toolkit
#1 Get out of the office
#2 Immerse yourself
#3 Get Session Replay
#4 Voice of Customer
#5 Act like a Private Eye
#6 Test things!
#7 Split testing tools
#8 Hypothesis design
#9 Invest in Copywriting
#10 Find, Read, Learn stuff
#11 Agile, Lean and CRO
#11 Work the dark side
#13Collaborative toolsets
#14 Photography
SUMMARY
@OptimiseOrDie

6. #1 : GET OUT OF THE OFFICE
@OptimiseOrDie

7. @OptimiseOrDie
1a : Lab Based Testing

8. @OptimiseOrDie
1b : Remote UX Testing
1
2
3
3
3
1 Moderator
Participant2
Viewers3
3

9. Som, feedbackRemote UX tools (P=Panel, S=Site recruited, B=Both)
Usertesting (B) www.usertesting.com
Userlytics (B) www.userlytics.com
Userzoom (S)www.userzoom.com
Intuition HQ (S) www.intuitionhq.com
Mechanical turk (S) www.mechanicalturk.com
Loop11 (S) www.loop11.com
Open Hallway (S)www.openhallway.com
What Users Do (P) www.whatusersdo.com
Feedback army (P) www.feedbackarmy.com
User feel (P) www.userfeel.com
Ethnio (For Recruiting) www.ethnio.com
Feedback on Prototypes / Mockups
Pidoco www.pidoco.com
Verify from Zurb www.verifyapp.com
Five second test www.fivesecondtest.com
Conceptshare www.conceptshare.com
Usabilla www.usabilla.com
9
@OptimiseOrDie
1c : Crowdsourced Testing

10. @OptimiseOrDie
1d : Beer, Caffeine and Work Breaks

11. DESKTOP & LAPTOP
CamStudio (free) www.camstudio.org
Mediacam AV (cheap) www.netu2.co.uk
Silverback (Mac) www.silverbackapp.com
Screenflow (Mac) www.telestream.net
MOBILE
UX Recorder (iOS) www.uxrecorder.com
Skype Hugging bit.ly/tesTfm
Reflectorbit.ly/JnwtMo
@OptimiseOrDie
1e : Guerrilla Testing

12. @OptimiseOrDie
1f : Interview Like a Pro!
• Read these:
“Don‟t Make Me Think” amzn.to/1gIZEJn
“Rocket Surgery Made Easy” amzn.to/1e0hnUL
“Talking to Customers” bit.ly/1e0hT58
“Talking with Participants” bit.ly/1kKL3LE
“Don‟t listen to Users” bit.ly/1cQpiIE
“Interviewing Tips” bit.ly/1fKqu03
“More interviewing Tips”bit.ly/1bmvGT

13. @OptimiseOrDie
1f : Multichannel, Device, Location, Context
http://www.nomensa.com/blog/2014/diary-studies-for-multi-device-user-research/

14. #2 : IMMERSE
YOURSELF
@OptimiseOrDie
• Test ALL key campaigns
• Use Real Devices
• Get your own emails
• Order your products
• Call the phone numbers
• Send an email
• Send 11 shoes back
• Be difficult
• Break things
• Experience the end-end
• Do the same for competitors
• Team are ALL mystery shoppers
• Wear the magical slippers
• Be careful about dogfood
though!

15. • Gives you insight missing from other tools
• Rich source of data on visitor experiences
• Segment by browser, visitor type, behaviour, errors
• Forms Analytics (when instrumented) are awesome
• Can be used to optimise in real time!
Session replay tools
• Clicktale (Client) www.clicktale.com
• SessionCam (Client) www.sessioncam.com
• Inspectlet www.inspectlet.com
• Mouseflow (Client) www.mouseflow.com
• Ghostrec (Client) www.ghostrec.com
• Usabilla (Client) www.usabilla.com
• Tealeaf (Hybrid) www.tealeaf.com
• UserReplay (Server) www.userreplay.com
@OptimiseOrDie
#3 : GET SESSION REPLAY

16. • Sitewide Omnipresent Feedback
• Triggered (Behavioural) Feedback
• Use of Features, Cancellation, Abandonment
• 4Q Task Gap Analysis very good
• Kampyle
www.kampyle.com
• Qualaroo
www.qualaroo.com
• Feedback Daddy
www.feedbackdaddy.com
• 4Q
4q.iperceptions.com
• Usabilla
www.usabilla.com
#4 : GET THEIR VOICE

17. • Surveys are an art form.
• They’re also highlyunderrated.
• The Autoglass (Belron) NPS survey got a 35% email
clickthrough rate and a 94% fillout rate. Why?
Luke Wroblewski www.lukew.com
Caroline Jarrett www.formsthatwork.com
Sticky Content www.stickycontent.com
• Higher NPS than a well known consumer brand!
Why?
• Systematic programme of sampling of millions of
customers a year. Figuring out the drivers of delight
and dissatisfaction. Optimise. Repeat.
#4 : GET THEIR VOICE

18. • Make contact and feedback easy & encouraged
• Add contact & feedback to everything (e.g. all emails)
• Run regular NPS and behaviourally triggered surveys
• Get ratings on Service Metrics
• Find what drives the ‘level’ of delight
• Listen to customers regularly
• Always be asking customers questions(in person or survey)
• Make the team spend ½ a day a month at the Call Centre
• Meet with your Sales and Support teams ALL the time
• Tip : Take them for Beers and encourage bitching
#4 : GET THEIR VOICE

19. • For your brand(s) and competitors
• Check review sites, Discussion boards, News
• Use Google Alerts on various brands & keywords
• See what tools they’re using (www.ghostery.com)
• Sign up for all competitor emails
• Run Cross Competitor surveys
• This was VITAL for LOVEFiLM
• Use Social & Competitor Monitoring tools :
slidesha.re/1k7bflG
blog.kissmetrics.com/james-bond-of-the-web/
#5 : ACT LIKE A PRIVATE EYE

20. #4 –Test or Die!
Email testing www.litmus.com
www.returnpath.com
www.lyris.com
Browser testingwww.crossbrowsertesting.com
www.browserstack.com
www.spoon.net
www.saucelabs.com
www.multibrowserviewer.com
Mobile devices www.appthwack.com
www.deviceanywhere.com
www.mobilexweb.com/emulators
www.opendevicelab.com
@OptimiseOrDie
#6 : 8 Million Pounds a Year – one bug

21. • Google Content Experiments
bit.ly/Ljg7Ds
• Optimizely
www.optimizely.com
• Visual Website Optimizer
www.visualwebsiteoptimizer.com
• Multi Armed Bandit Explanation
bit.ly/Xa80O8
• New Machine Learning Tools
www.conductrics.com
www.rekko.com
@OptimiseOrDie
#7 : MACHINE LEARNING

22. Insight - Inputs
#FAIL
Competitor
copying
Guessing
Dice rolling
An article
the CEO
read
Competitor
change
Panic
Ego
Opinion
Cherished
notions
Marketing
whims Cosmic rays
Not ‘on
brand’
enough
IT
inflexibility
Internal
company
needs
Some
dumbass
consultant
Shiny
feature
blindness
Knee jerk
reactons
#8 : Building a Hypothesis
@OptimiseOrDie

23. Insight - Inputs
Insight
Segmentation
Surveys
Sales and
Call Centre
Session
Replay
Social
analytics
Customer
contact
Eye tracking
Usability
testing
Forms
analytics
Search
analytics Voice of
Customer
Market
research
A/B and
MVT testing
Big &
unstructured
data
Web
analytics
Competitor
evalsCustomer
services
#8 : These are the inputs you need…
@OptimiseOrDie

24. Insight - Inputs
@OptimiseOrDie
#8 : Brainstorming the test
• Check your inputs
• Assemble the widest possible team
• Share your data and research
• Design Emotive Writing guidelines

25. Insight - Inputs
@OptimiseOrDie
#8 : Emotive Writing - example
Customers do not know what to do and need support and advice
• Emphasize the fact that you understand that their situation is stressful
• Emphasize your expertise and leadership in vehicle glazing and willhelp them get the best
solution for their situation
• Explain what they will need to do online and during the call-back so that they know what the
next steps will be
• Explain that they will be able ask any other questions they might have during the call-back
Customers do not feel confident in assessing the damage
• Emphasize the fact that you will help them assess the damage correctly online
Customers need to understand the benefits of booking online
• Emphasize that the online booking system is quick, easy and provides all the information
they need in regards with their appointment and general cost information
Customers mistrust insurers and find dealing with their insurance situation very frustrating
• Where possible communicate the fact that the job is most likely to be free for insured
customers, or good value for money for cash customers
• Show that you understand the hassle of dealing with insurance companies – emphasise that
you will help with their insurance paperwork for them, freeing them of this burden
Some customers cannot be bothered to take action to fix their car glass
• Emphasize the consequences of not doing anything,
e.g. ‘It’s going to cost you more if the chip develops into a crack’

26. Insight - Inputs
@OptimiseOrDie
#8 : THE DARK SIDE
“Keep your family safe and get back on the
road fast with Autoglass.”

27. Insight - Inputs
@OptimiseOrDie
#8 : NOW YOU CAN BEGIN
• You should have inputs, research, data, guidelines
• Sit down with the team and prompt with 12 questions:
– Who is this page (or process) for?
– What problem does this solve for the user?
– How do we know they need it?
– What is the primary action we want people to take?
– What might prompt the user to take this action?
– How will we know if this is doing what we want it to do?
– How do people get to this page?
– How long are people here on this page?
– What can we remove from this page?
– How can we test this solution with people?
– How are we solving the users needs in different and better ways than
other places on our site?
– If this is a homepage, ask these too (bit.ly/1fX2RAa)

28. Insight - Inputs
@OptimiseOrDie
#8 : PROMPT YOURSELF
• Check your Copywriting guidelines.
• Use Get Mental Notes
• What levers can we apply now?
• Create a hypothesis:
“WE BELIEVE THAT DOING [A]
FOR PEOPLE [B] WILL MAKE
OUTCOME [C] HAPPEN.
WE'LL KNOW THIS WHEN WE
SEE DATA [D] AND FEEDBACK
[E]”
www.GetMentalNotes.com

29. Insight - Inputs
@OptimiseOrDie
#8 : THE FUN BIT!
• Collaborative Sketching
• Brainwriting
• Refine and Test!

30. We believe that doing [A] for
People [B] will make
outcome [C] happen.
We’ll know this when we
observe data [D] and obtain
feedback [E]. (reverse)
@OptimiseOrDie

31. Insight - Inputs
@OptimiseOrDie
#8 : HYPOTHESIS DESIGN SUMMARY
• Inputs – get the right stuff
• Research, Guidelines, Data
• Framing the problem(s)
• Questions to get you going
• Use card prompts for Psychology
• Create a hypothesis
• Collaborative Sketching
• Brainwriting
• Refine and Check Hypothesis
• Instrument and Test

32. “On the average, five times as many people
read the headline as read the body copy.
When you have written your headline, you
have spent eighty cents out of your dollar.”
David Ogilvy
“In 9 years and 40M split tests with visitors,
the majority of gains came from playing
with the words.”
@OptimiseOrDie
#9 : HIRE OR TRAIN GREAT
COPYWRITERS

33. Why?
• Because every word, every piece of copy, each error message, all
the emails, the forms, the web pages – are your BRAND and
EXPERIENCE
• The words set the emotional landscape and lens through which
your entire product is viewed
• If you aren‟t writing for simplicity, comprehension and persuasion,
you‟re missing the biggest trick in all this work
• There is no dumbing down of content
• Get some training from www.stickycontent.co.uk
• STOP FILLING IN (Developers & Error Messages)
• We know lots of new things about the brain but the techniques in
persuasive writing have been around for decades
• Eye control and copy decoration : mklnd.com/RENTbr
#9 : HIRE OR TRAIN GREAT
COPYWRITERS

34. Scarcity principle...
#9 : EXAMPLE – SPAM MESSAGING

35. Scarcity principle...
#9: EXAMPLE – SPAM MESSAGING
We promise never to maim
or kill you whilst shopping!

36. #10 : READ STUFF

37. #10 : READ STUFF

38. #10 : FIND STUFF
@OptimiseOrDie
@danbarker Analytics
@fastbloke Analytics
@timlb Analytics
@jamesgurd Analytics
@therustybear Analytics
@carmenmardirosAnalytics
@davechaffey Analytics
@priteshpatel9 Analytics
@cutroni Analytics
@avinash Analytics
@Aschottmuller Analytics, CRO
@cartmetrix Analytics, CRO
@Kissmetrics CRO / UX
@Unbounce CRO / UX
@Morys CRO / Neuro
@UXFeeds UX / Neuro
@Psyblog Neuro
@Gfiorelli1 SEO / Analytics
@PeepLaja CRO
@TheGrok CRO
@UIE UX
@LukeW UX / Forms
@cjforms UX / Forms
@axbom UX
@iatv UX
@Chudders Photo UX
@JeffreyGroks Innovation
@StephanieRieger Innovation
@BrianSolis Innovation
@DrEscotet Neuro
@TheBrainLady Neuro
@RogerDooley Neuro
@Cugelman Neuro
@Smashingmag Dev / UX
@uxmag UX
@Webtrends UX / CRO

39. #10 : LEARN STUFF
@OptimiseOrDie
Baymard.com
Lukew.com
Smashingmagazine.com
ConversionXL.com
Medium.com
Whichtestwon.com
Unbounce.com
Measuringusability.com
RogerDooley.com
Kissmetrics.com
Uxmatters.com
Smartinsights.com
Econsultancy.com
Cutroni.com
www.GetMentalNotes.com

40. #11 : Agile, Lean and Optimisation
@OptimiseOrDie

41. CRO
41

42. Lean CRO
Inspection
Immersion
Identify
Triage &
Triangulate
Outcome
Streams
Measure
Learn
Instrument
42

43. Think like a
store owner!
If you can‟t refurbish
the entire store,
which floors or
departments will you
invest in optimising?
Wherever there is:
• Footfall
• Low return
• Opportunity
@OptimiseOrDie

44. #12 : WORK THE DARK SIDE TOO
• Your customer acquisition costs may be high
but churn, returns or lack of repeat custom
could be totally eating your growth ability
• Churn and Cancellation are vital sources
• Talk to your complaints or escalation teams
• Seek out unhappy people
• Interview these pissed off customers
• Learn from #fail

45. #13 – Use Great Photos
24 Jan 2012
• Persuasion / Influence /
Direction / Explanation
• Helps people process
information and stories
• Vital to sell an „experience‟
• Helps people recognise and
discriminate between things
• Supports Scanning Visitors
• Drives emotional response
short.cx/YrBczl

46. @OptimiseOrDie
#13 : SPLIT TESTING PHOTOGRAPHY

47. Terrible Stock Photos : headsethotties.com&awkwardstockphotos.com
Laughing at Salads : womenlaughingwithsalad.tumblr.com
Other Stock Memes : linkli.st/optimiseordie/7Fdxz
BBC Fake Smile Test : bbc.in/5rtnv @OptimiseOrDie

48. SPAIN
+22% over control
99% confidence
@OptimiseOrDie
#13 : PHOTOGRAPHY UX

49. The Best Companies…
• Have great copywriters at the heart of the process
• Source photos and content that support persuasion and utility
• Have cross channel, cross device design, testing and QA
• Invest continually in analytics instrumentation, tools, people
• Prioritise development based on data and customer insight
• Have an Agile, iterative, cross-silo, one team project culture
• Practice real continuous product improvement
• Continuallyreduce cycle (iteration) time in their process
• Prefer collaborative tools and methods to lots of meetings
• Segment their data for valuable insights, every test or change
• Blend ‘long’ design, continuous improvement AND split tests
• Make optimisation the engine of change, not the slave of ego

50. “Congratulations!
Today you‟re the lucky
winner of our random
awards programme.
You get all these extra
features for free, on us.
Enjoy.”
Business Future Testing

51. Insight - Inputs
@OptimiseOrDie
SUMMARY
• Great new stuff about the brain
• It‟s all about the inputs & execution
• Don‟t randomly test – gamethe system
• Drive split tests using Data, Psychology, Customers
• If you read all this stuff, you‟ll do well
• Last 18m : 25%-800% increase in revenue
• Biggest insight of the last 5 years?
“Blending Lean and Agile UX with conversion
optimisation techniques* and participative,
collaborative cultures – will produce exceptional
products and insights.”
*(persuasive copywriting & psychology, analytics, split testing, Kaizen, Kano)

52. If it isn‟t working, you‟re not doing it right
@OptimiseOrDie

53. 53
WRAPUP

54. Email
Twitter
: sullivac@gmail.com
: @OptimiseOrDie
: linkd.in/pvrg14
Slides will be posted on www.Slideshare.net/sullivac

55. RESOURCE PACK
• Maturity model
• Belron methodology example
• Building a ring model
• Manual Models for Analytics
@OptimiseOrDie

56. Ad Hoc
Local Heroes
Chaotic Good
Level 1
Starter Level
Guessing
A/B testing
Basic tools
Analytics
Surveys
Contact Centre
Low budget
usability
Outline process
Small team
Low hanging fruit
+ Multi variate
Session replay
No segments
+Regular usability
testing/research
Prototyping
Session replay
Onsite feedback
________________________________________________________________________
_____________________ _
Dedicated team
Volume
opportunities
Cross silo team
Systematic tests
Ninja Team
Testing in the
DNA
Well developed Streamlined Company wide
+Funnel
optimisation
Call tracking
Some segments
Micro testing
Bounce rates
Big volume
landing pages
+ Funnel analysis
Low converting
& High loss pages
+ offline
integration
Single channel
picture
+ Funnel fixes
Forms analytics
Channel switches
+Cross channel
testing
Integrated CRO
and analytics
Segmentation
+Spread tool use
Dynamic adaptive
targeting
Machine learning
Realtime
Multichannel
funnels
Cross channel
synergy
________________________________________________________________________
_______________________
________________________________________________________________________
________________________
Testing
focus
Culture
Process
Analytics
focus
Insight
methods
+User Centered
Design
Layered feedback
Mini product tests
Get buyin
_________________________________________________________________________
_______________________Mission Prove ROI Scale the testing Mine value
Continual
improvement
+ Customer sat
scores tied to UX
Rapid iterative
testing and
design
+ All channel view
of customer
Driving offline
using online
All promotion
driven by testing
Level 2
Early maturity
Level 3
Serious testing
Level 4
Core business value
Level 5
You rock, awesomely
________________________________________________________________________
________________________
56

57. 5 - Belron example – Funnel replacement
Final
prototype
Usability
issues left
Final changes Release build
Legal review
kickoff
Cust services
review kickoff
Marketing
review
Test Plan
Signoff
(Legal,
Mktng, CCC)
Instrument
analytics
Instrument
Contact
Centre
Offline
tagging
QA testing
End-End
testing
Launch
90/10%
Monitor
Launch
80/20%
Monitor < 1
week
Launch
50/50%
Go live 100%
Analytics
review
Washup and
actions
New
hypotheses
New test
design
Rinse and
Repeat!

58. #1 Building a Model
#1 Avinash article
#2 The Ring Model
#3 3 examples
#4 Benefits
#5 Summary
58

59. 6.1 – Avinash “See-Think-Do”
• Avinash Kaushik, analytics guru, proposes a very nice model for marketing. A
brilliant article can be read here:
• http://www.kaushik.net/avinash/see-think-do-content-marketing-measurement-
business-framework/
• But this sort of thinking is also relevant to optimisation
• CRO often focuses on purely the ‘Do’ stage – rather than ‘See’ or ‘Think’ stages.
59

60. 6.1 – Example
60

61. 6.2 – The Ring Model
• Simply looking at conversion points is not enough
• We need a way to look at the ‘layers’ or ‘levels’ reached
• So I developed a ring or engagement model
• This works for many (but not all) websites
• Focuses on depth of engagement, not pages viewed
• Helps to see the key loss steps, like a funnel
• It’s not a replacement for funnel diagrams
• It helps to see the ‘big picture’ involved
• So – let’s try some examples
61

62. 6.3 – Examples – Concept
62
Bounce
Engage
Outcome

63. 6.3 – Examples – Shoprush.com
63
Bounce
Search or
Category
Product
Page
Add to
basket
View
basket
Checkout
Complete

64. 6.3 – Examples – 16-25Railcard.co.uk
64
Bounce
Login to
Account
Content
Engage
Start
Application
Type and
Details
Eligibility
Photo
Complete

65. 6.3 – Examples – Guide Dogs
65
Bounce
Content
Engage
Donation
Pathway
Donation
Page
Starts
process
Funnel
steps
Complete

66. 6.3 – Within a layer
66
Page 1
Page 2
Page 3
Page 4 Page 5
Exit
Deeper
Layer
Email
LikeContact
Wishlist
Micro
Conversions

67. 6.4 – Benefits
• Helps you see where flow is ‘stuck’
• Sorts out small opportunities from big wins
• Ignores pages in favour of ‘Macro’ and ‘Micro’ conversions
• Lets you show the client where focus should be
• Helps flush flow or traffic through to lower levels
• Avoids prioritising the wrong part of the model!
• Example – Shoprush problem is basket adds, not checkout
• If you had 300k product page views, 5k adds and 1k
checkouts – where would your problem be?
• If you had 300k product page views, 100k adds and 1k
checkouts – it’s a different place!
• Example – Google adwords site has bad traffic, not
conversion
67

68. 6.5 – Benefits contd.
• A nice simple way to visualise complex websites
• Does not rely on pages – more ‘steps’ or ‘layers’
• Helps you see where traffic is ‘stuck’ or ‘failing to engage
more deeply’
• The combination of traffic potential, UX and persuasion
issues combines to identify opportunity
• Avoids visual bias when doing an expert review
• In the e-commerce example, Rush have optimised product
page first, not homepage.
• Questions?
68