SlideShare uma empresa Scribd logo
1 de 146
Baixar para ler offline
Knowing ranking factors won’t
be enough
How to avoid losing your job to a robot
@willcritchlow
I’m going to tell you about a robot
that understands ranking factors
better than any of you
...but before I get to that, let’s look at a bit of history...
The other day I searched:
Unsurprisingly, I got
an answer
But it got me thinking
about how, in 2009,
the results would
have looked more like
this.
In 2009, it would have
looked more like this.
With every title
containing the
keyphrase.
In 2009, it would have
looked more like this.
With every title
containing the
keyphrase.
Most at the beginning.
OK. Maybe wikipedia
would have been #1.
My mental model for ~2009 ranking factors had
three different modes:
One in the
hyper-competitive
head
My mental model for ~2009 ranking factors had
three different modes:
One in the
competitive
mid-tail
...and
one
in
the
long-tail
One in the
hyper-competitive
head
Tons of perfectly on-topic
pages to choose from
One in the
hyper-competitive
head
So pick only perfectly-on-topic pages
One in the
hyper-competitive
head
(*) Page authority, but the
domain inevitably factors into
that calculation. This is why
so many homepages ranked
One in the
hyper-competitive
head
...and rank by authority (*)
This resulted in a mix
of homepages of
mid-size sites, and
inner pages on huge
sites
One in the
hyper-competitive
head
But the general way
to move up was
through increased
authority
One in the
hyper-competitive
head
Kind of search
result
Pages ranking To move up...
Head Homepages of mid-size
sites and inner pages of
massive sites. All
perfectly-targeted.
Improve authority.
Mid-tail
Long-tail
One in the
hyper-competitive
head
One in the
competitive
mid-tail
Wealth of ROUGHLY
on-topic pages to
choose from
One in the
competitive
mid-tail
PERFECTLY on-topic
could do well even on
a relatively weak site
One in the
competitive
mid-tail
Rank the roughly
on-topic pages by
authority x “on-topicness”
One in the
competitive
mid-tail
Move up with better
targeting or more
authority
One in the
competitive
mid-tail
Kind of search
result
Pages ranking To move up...
Head Homepages of mid-size
sites and inner pages of
massive sites. All
perfectly-targeted.
Improve authority.
Mid-tail Perfectly on-topic pages
on relatively weak sites
plus roughly on-topic on
bigger sites.
Improve targeting or
authority.
Long-tail
One in the
hyper-competitive
head
One in the
competitive
mid-tail
...and
one
in
the
long-tail
In the long-tail, a site
of arbitrary weakness
could rank if it was the
most relevant
...and
one
in
the
long-tail
Otherwise, massive
sites rank with
off-topic pages that
mention something
similar
...and
one
in
the
long-tail
Generally, move up
with better targeting
...and
one
in
the
long-tail
Kind of search
result
Pages ranking To move up...
Head Homepages of mid-size
sites and inner pages of
massive sites. All
perfectly-targeted.
Improve authority.
Mid-tail Perfectly on-topic pages
on relatively weak sites
plus roughly on-topic on
bigger sites.
Improve targeting or
authority.
Long-tail Arbitrarily-weak on-topic
pages and
roughly-targeted deep
pages on massive sites.
Improve targeting.
Kind of search
result
Pages ranking To move up...
Head Homepages of mid-size
sites and inner pages of
massive sites. All
perfectly-targeted.
Improve authority.
Mid-tail Perfectly on-topic pages
on relatively weak sites
plus roughly on-topic on
bigger sites.
Improve targeting or
authority.
Long-tail Arbitrarily-weak on-topic
pages and
roughly-targeted deep
pages on massive sites.
Improve targeting.
So that was
~2009
It’s not so simple any more.
Google is harder to understand these days.
PageRank
(the first algorithm to
use the link structure of
the web)
We know how we got to ~2009...
Information
retrieval
PageRank
Information
retrieval
PageRank Original research
Information
retrieval
PageRank Original research TWEAKS
...with growing complexity in subsequent years
Particularly this comment from a user called Kevin Lacker (@lacker):
I was thinking about it like it was a
math puzzle and if I just thought
really hard it would all make sense.
-- Kevin Lacker (@lacker)
Hey why don't you take the square
root?
-- Amit Singhal according to Kevin Lacker (@lacker)
oh... am I allowed to write code that
doesn't make any sense?
-- Kevin Lacker (@lacker)
-- Amit Singhal according to Kevin Lacker (@lacker)
Multiply by 2 if it helps, add 5,
whatever, just make things work
and we can make it make sense
later.
3 big reasons:
High-
dimension
Non-linear
Discontinuous
High-
dimension
Non-linear
Discontinuous
High-
dimension
Non-linear
Discontinuous
High-
dimension
Non-linear
Discontinuous
You might know what any one of
the levers does, but they can
interact with each other in complex
ways
This is what a high-dimensional function looks like
High-
dimension
Non-linear
Discontinuous
We sell custom cigar humidors. Our
custom cigar humidors are handmade. If
you’re thinking of buying a custom cigar
humidor, please contact our custom
cigar humidor specialists at
custom.cigar.humidors@example.com
What this needs is another mention of [cigar humidors]
With no mentions of [cigar] or [humidor] this
page would be unlikely to rank
And yet you can clearly go too far, and have the effect turn negative.
This is called nonlinearity.
The cigar example is taken directly from Google’s quality guidelines.
High-
dimension
Non-linear
Discontinuous
Discontinuities are steps in the
function
Think about so-called “over-optimization” tipping points
Think about category pages:
Do you recommend removing “SEO text”?
We’ve tested it, so we know the answer.
If you said “yes”, congratulations
(+3.1% organic sessions in a split-test)
Unless you’re responsible for this site
No effect / possible negative effect
No, but I’m still pretty good at this
You’re thinking this to yourself right now.
I promised to tell you about a robot
that is better than even
experienced SEOs...
Well. It turns out all we needed was a coin to flip. You’re all fired.
It’s only going to get worse under Sundar Pichai
Who knows who this is?
(This is the only CC-licensed photo of him on the internet)
ENHANCE
What about now?
John Giannandrea - Google’s head of search
Sundar’s choice to lead search after Amit. Previously running machine learning.
...and of course Jeff Dean is doing Jeff Dean things
(c.f. Chuck Norris)
Jeff Dean puts his pants on one leg
at a time, but if he had more legs,
you would see that his approach is
O(log n).
Source: Jeff Dean facts
Once, in early 2002, when the
search back-ends went down, Jeff
Dean answered user queries
manually for two hours.
Result quality improved markedly during this time
When Jeff Dean goes on vacation,
production services across Google
mysteriously stop working within a
few days.
This was reportedly actually true
The original Google Translate was
the result of the work of hundreds of
engineers over 10 years.
Director of Translate, Macduff
Hughes said that it sounded to him
as if maybe they could pull off a
neural-network-based replacement
in three years.
Jeff Dean said “we can do it by the
end of the year, if we put our minds
to it”.
Hughes: “I’m not going to be the one
to say Jeff Dean can’t deliver speed.”
A month later, the work of a team of
3 engineers was tested against the
existing system. The improvement
was roughly equivalent to the
improvement of the old system over
the previous 10 years.
Hughes sent his team an email. All
projects on the old system were to be
suspended immediately.
[Read the whole story ]
Background reading:(backchannel, bloomberg)
How to avoid losing your job to a
robot
This is what you promised, Will.
Let’s start by
understanding
some robot
weaknesses
What’s this?
Ooh. Ooh.
I know this one.
-- robot
“It’s a leopard. I’m like 99% sure.”
Computers are better than humans at
classification, but struggle with adversaries
Read more about this here -- Cheetah, Leopard, Jaguar
We don’t fully understand all ML mistakes
See: adversarial AI
And when you’re trying to fool the machine...
See: adversarial AI
And when you’re trying to fool the machine...
See: adversarial AI
You get some really wild examples
See: adversarial AI
Lesson:
We expect adversarial abilities to
take a step backwards
They will remain good at classifying bad links but will be likely to fall
prey to weird outcomes in adversarial situations
We’re going to see new kinds of
bugs
Rules of ML [PDF] outlines engineering lessons
from getting ML into production at Google
That document also has a section on trying to
understand what the machines are doing
But human explainability may not
even be possible
Not every concept a neural network uses fits neatly into a concept for
which we have a word. It’s not clear this is a weakness per se, but...
...this means that engineers won’t
always know more than we do
about why a page does or doesn’t
rank
The big knowledge gap of the future is data - clickthrough rates,
bounce rates etc.
Check out Tom Capper’s presentation on how
engineers’ statements can be misleading
...and remember the confounding split-tests
It’s already not always as simple as “feature X is good”
Which all means we may need to be more independent-minded and do
more of our own research
So how do we fight
back?
Michael Lewis’ latest book is
about Kahneman and Tversky
spelling.
It recounts a story about a piece
of medical software that existed
in the 1960s.
It was designed to encapsulate
how a range of doctors
diagnosed stomach cancer from
x-rays.
It proceeded to outperform those
same doctors despite only
containing their expertise.
Real people have biases, and fool
themselves.
Encapsulate your own expert
knowledge.
At Distilled, we use a
methodology we call the
balanced digital scorecard.
This encapsulates our beliefs
about how to build a
high-performing business.
Applying it helps avoid our own
biases.
Also, while we are talking about
books, The Checklist Manifesto is
an important part of avoiding the
same cognitive biases.
Focus on consulting skills
I’ve written a few things about
this (DistilledU module, writing
better business documents, using
split-tests to consult better).
Use case studies and creativity.
Computers are better at
diagnosis than cure.
This means: getting things done,
convincing organizations,
applying general knowledge,
learning new things.
We are going to need to be
better than ever at debugging
things.
I wrote about debugging skills for
non-developers here.
A lot of the story of enterprise
consulting is going to be about
figuring out why things have
gone wrong in the face of sparse
or incorrect information from
Google.
Disregard expert surveys
Firstly, there are all the problems
outlined in the search result pairs
study - both in the ability of
experts to understand factors,
and in your ability to use the
information even if they do.
Secondly, they are broken with
another bias called the “law of
small numbers” from Lewis’ book.
PS - I say this as a participant in
many of them
Me
Equally, building your digital
strategy on what Google tells you
to do will become an even worse
idea than it already is.
This is why we have been investing so much in split-testing
Check out odn.distilled.net if you haven’t already. The team will be happy to
demo for you.
We served ~5 billion requests last quarter and recently published
everything from response times to our +£100k / month split test.
Let’s recap
1. Even in a world of 200+ “classical” ranking factors, humans were bad at
understanding the algorithm
Let’s recap
1. Even in a world of 200+ “classical” ranking factors, humans were bad at
understanding the algorithm
2. Machine learning will make this worse, and is accelerating under Sundar
Let’s recap
1. Even in a world of 200+ “classical” ranking factors, humans were bad at
understanding the algorithm
2. Machine learning will make this worse, and is accelerating under Sundar
3. There are things computers remain bad at, and rankings will become more
opaque even to Google engineers
Let’s recap
1. Even in a world of 200+ “classical” ranking factors, humans were bad at
understanding the algorithm
2. Machine learning will make this worse, and is accelerating under Sundar
3. There are things computers remain bad at, and rankings will become more
opaque even to Google engineers
4. We remain relevant by:
a. Using methodologies and checklists to capture human capabilities and
avoid our biases
Let’s recap
1. Even in a world of 200+ “classical” ranking factors, humans were bad at
understanding the algorithm
2. Machine learning will make this worse, and is accelerating under Sundar
3. There are things computers remain bad at, and rankings will become more
opaque even to Google engineers
4. We remain relevant by:
a. Using methodologies and checklists to capture human capabilities and
avoid our biases
b. Becoming great consultants and change agents
Let’s recap
1. Even in a world of 200+ “classical” ranking factors, humans were bad at
understanding the algorithm
2. Machine learning will make this worse, and is accelerating under Sundar
3. There are things computers remain bad at, and rankings will become more
opaque even to Google engineers
4. We remain relevant by:
a. Using methodologies and checklists to capture human capabilities and
avoid our biases
b. Becoming great consultants and change agents
c. Debugging the heck out of everything
Let’s recap
1. Even in a world of 200+ “classical” ranking factors, humans were bad at
understanding the algorithm
2. Machine learning will make this worse, and is accelerating under Sundar
3. There are things computers remain bad at, and rankings will become more
opaque even to Google engineers
4. We remain relevant by:
a. Using methodologies and checklists to capture human capabilities and
avoid our biases
b. Becoming great consultants and change agents
c. Debugging the heck out of everything
d. Avoiding being misled by experts or Google
Let’s recap
1. Even in a world of 200+ “classical” ranking factors, humans were bad at
understanding the algorithm
2. Machine learning will make this worse, and is accelerating under Sundar
3. There are things computers remain bad at, and rankings will become more
opaque even to Google engineers
4. We remain relevant by:
a. Using methodologies and checklists to capture human capabilities and
avoid our biases
b. Becoming great consultants and change agents
c. Debugging the heck out of everything
d. Avoiding being misled by experts or Google
Testing!
What about that robot I promised
you?
The coin flip wasn’t really it
keras.io
The specifics of DeepRank
We started with a broad range
of unbranded keywords from
our STAT rank tracking.
For each of the URLs ranking in
the top 10, we gathered key
metrics about the domain and
page - both from direct crawling
and various APIs.
We turned this into a set of pairs
of URLs {A,B} with their
associated keyword, metrics,
and their rank ordering.
Gather and
process
training data
The specifics of DeepRank
We started with a broad range
of unbranded keywords from
our STAT rank tracking.
For each of the URLs ranking in
the top 10, we gathered key
metrics about the domain and
page - both from direct crawling
and various APIs.
We turned this into a set of pairs
of URLs {A,B} with their
associated keyword, metrics,
and their rank ordering.
We have so far trained on just 10
metrics for a relatively small
sample (hundreds) of keywords.
Our current version is only a few
layers deep with only 10 hidden
dimensions.
The current training samples 30
pairs at a time and trains against
them for 500 epochs.
The specifics of DeepRank
Train the
model
Gather and
process
training data
Train the
model
Gather and
process
training data
The specifics of DeepRank
Model
The next task is to get way more
metrics for thousands of
keywords.
This will enable us to train a
much deeper model for much
longer without overfitting.
We also have some more
hyperparameter tuning to do.
To run the model, we input a
pair of pages with their
associated metrics.
New
input
Model
New
input
We get back a probability of
page A outranking page B.
Model
Probability-
weighted
predictions
New
input
The goal is a winning combination
of human and machine
Human + computer beats computer (for now)
Questions: @willcritchlow
● Mobius strip
● Confusion
● Signal box
● Cigar
● Discontinuity
● Confidence
● Burt Totaro
● Sundar Pichai
● John Giannandrea
● Chuck Norris
● Jeff Dean
● Fencing
● Keyboard
Image credits
● Go
● Robot
● Leopard print sofa
● Leopard
● Bug
● Lego robots
● Iron Man
● Boston

Mais conteúdo relacionado

Mais procurados

Jono Alderson - Structured Data for Content Marketing
Jono Alderson - Structured Data for Content MarketingJono Alderson - Structured Data for Content Marketing
Jono Alderson - Structured Data for Content MarketingSemrush
 
CRO and SEO together: what happens when what's good for users isn't good for ...
CRO and SEO together: what happens when what's good for users isn't good for ...CRO and SEO together: what happens when what's good for users isn't good for ...
CRO and SEO together: what happens when what's good for users isn't good for ...Will Critchlow
 
Delivering Better Onsite Search Results - Brighton SEO Sep 2018
Delivering Better Onsite Search Results - Brighton SEO Sep 2018Delivering Better Onsite Search Results - Brighton SEO Sep 2018
Delivering Better Onsite Search Results - Brighton SEO Sep 2018JP Sherman
 
SearchLove London 2016 | Dom Woodman | How to Get Insight From Your Logs
SearchLove London 2016 | Dom Woodman | How to Get Insight From Your LogsSearchLove London 2016 | Dom Woodman | How to Get Insight From Your Logs
SearchLove London 2016 | Dom Woodman | How to Get Insight From Your LogsDistilled
 
SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach Us
SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach UsSEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach Us
SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach UsTom Anthony
 
SearchLove Boston 2017 | Paul Madden | You, Google and Links: It's Complicated
SearchLove Boston 2017 | Paul Madden | You, Google and Links: It's ComplicatedSearchLove Boston 2017 | Paul Madden | You, Google and Links: It's Complicated
SearchLove Boston 2017 | Paul Madden | You, Google and Links: It's ComplicatedDistilled
 
SearchLove Boston 2016 | Larry Kim | Hacking RankBrain: Four Strategies You’l...
SearchLove Boston 2016 | Larry Kim | Hacking RankBrain: Four Strategies You’l...SearchLove Boston 2016 | Larry Kim | Hacking RankBrain: Four Strategies You’l...
SearchLove Boston 2016 | Larry Kim | Hacking RankBrain: Four Strategies You’l...Distilled
 
SearchLove London 2018 - Els Aerts - User Research Done Right
SearchLove London 2018 - Els Aerts - User Research Done RightSearchLove London 2018 - Els Aerts - User Research Done Right
SearchLove London 2018 - Els Aerts - User Research Done RightDistilled
 
SearchLove London 2016 | Amy Harrison | Stand out to YOUR Crowd: A Simple Fra...
SearchLove London 2016 | Amy Harrison | Stand out to YOUR Crowd: A Simple Fra...SearchLove London 2016 | Amy Harrison | Stand out to YOUR Crowd: A Simple Fra...
SearchLove London 2016 | Amy Harrison | Stand out to YOUR Crowd: A Simple Fra...Distilled
 
SEO Success Factors - SMX Advanced 2014
SEO Success Factors - SMX Advanced 2014SEO Success Factors - SMX Advanced 2014
SEO Success Factors - SMX Advanced 2014Matthew Brown
 
SearchLove London 2018 - John Campbell - Voice Search – Calculating and Seizi...
SearchLove London 2018 - John Campbell - Voice Search – Calculating and Seizi...SearchLove London 2018 - John Campbell - Voice Search – Calculating and Seizi...
SearchLove London 2018 - John Campbell - Voice Search – Calculating and Seizi...Distilled
 
SearchLove San Diego 2017 | Rob Bucci | Snooping Into Google's Insights on Se...
SearchLove San Diego 2017 | Rob Bucci | Snooping Into Google's Insights on Se...SearchLove San Diego 2017 | Rob Bucci | Snooping Into Google's Insights on Se...
SearchLove San Diego 2017 | Rob Bucci | Snooping Into Google's Insights on Se...Distilled
 
3 New Techniques for the Modern Age of SEO
3 New Techniques for the Modern Age of SEO3 New Techniques for the Modern Age of SEO
3 New Techniques for the Modern Age of SEOTom Anthony
 
SearchLove San Diego 2018 | Will Critchlow | From the Horse’s Mouth: What We ...
SearchLove San Diego 2018 | Will Critchlow | From the Horse’s Mouth: What We ...SearchLove San Diego 2018 | Will Critchlow | From the Horse’s Mouth: What We ...
SearchLove San Diego 2018 | Will Critchlow | From the Horse’s Mouth: What We ...Distilled
 
SearchLove London 2016 | Bridget Randolph | The Changing Landscape of Mobile ...
SearchLove London 2016 | Bridget Randolph | The Changing Landscape of Mobile ...SearchLove London 2016 | Bridget Randolph | The Changing Landscape of Mobile ...
SearchLove London 2016 | Bridget Randolph | The Changing Landscape of Mobile ...Distilled
 
Next Era of SEO: A Guide to SEO Split-Testing
Next Era of SEO: A Guide to SEO Split-TestingNext Era of SEO: A Guide to SEO Split-Testing
Next Era of SEO: A Guide to SEO Split-TestingTom Anthony
 
SearchLove London 2016 | Tom Anthony | SEO Split-Testing - How You can Run Te...
SearchLove London 2016 | Tom Anthony | SEO Split-Testing - How You can Run Te...SearchLove London 2016 | Tom Anthony | SEO Split-Testing - How You can Run Te...
SearchLove London 2016 | Tom Anthony | SEO Split-Testing - How You can Run Te...Distilled
 
Google's Hummingbird and the Entity Search Revolution
Google's Hummingbird and the Entity Search RevolutionGoogle's Hummingbird and the Entity Search Revolution
Google's Hummingbird and the Entity Search RevolutionCyrus Shepard
 
SearchLove London 2016 | Larry Kim | Hacking RankBrain and Other Machine Lear...
SearchLove London 2016 | Larry Kim | Hacking RankBrain and Other Machine Lear...SearchLove London 2016 | Larry Kim | Hacking RankBrain and Other Machine Lear...
SearchLove London 2016 | Larry Kim | Hacking RankBrain and Other Machine Lear...Distilled
 
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments' SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments' Distilled
 

Mais procurados (20)

Jono Alderson - Structured Data for Content Marketing
Jono Alderson - Structured Data for Content MarketingJono Alderson - Structured Data for Content Marketing
Jono Alderson - Structured Data for Content Marketing
 
CRO and SEO together: what happens when what's good for users isn't good for ...
CRO and SEO together: what happens when what's good for users isn't good for ...CRO and SEO together: what happens when what's good for users isn't good for ...
CRO and SEO together: what happens when what's good for users isn't good for ...
 
Delivering Better Onsite Search Results - Brighton SEO Sep 2018
Delivering Better Onsite Search Results - Brighton SEO Sep 2018Delivering Better Onsite Search Results - Brighton SEO Sep 2018
Delivering Better Onsite Search Results - Brighton SEO Sep 2018
 
SearchLove London 2016 | Dom Woodman | How to Get Insight From Your Logs
SearchLove London 2016 | Dom Woodman | How to Get Insight From Your LogsSearchLove London 2016 | Dom Woodman | How to Get Insight From Your Logs
SearchLove London 2016 | Dom Woodman | How to Get Insight From Your Logs
 
SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach Us
SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach UsSEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach Us
SEO Tests on Big Sites & Small - What Etsy, Pinterest and Others Can Teach Us
 
SearchLove Boston 2017 | Paul Madden | You, Google and Links: It's Complicated
SearchLove Boston 2017 | Paul Madden | You, Google and Links: It's ComplicatedSearchLove Boston 2017 | Paul Madden | You, Google and Links: It's Complicated
SearchLove Boston 2017 | Paul Madden | You, Google and Links: It's Complicated
 
SearchLove Boston 2016 | Larry Kim | Hacking RankBrain: Four Strategies You’l...
SearchLove Boston 2016 | Larry Kim | Hacking RankBrain: Four Strategies You’l...SearchLove Boston 2016 | Larry Kim | Hacking RankBrain: Four Strategies You’l...
SearchLove Boston 2016 | Larry Kim | Hacking RankBrain: Four Strategies You’l...
 
SearchLove London 2018 - Els Aerts - User Research Done Right
SearchLove London 2018 - Els Aerts - User Research Done RightSearchLove London 2018 - Els Aerts - User Research Done Right
SearchLove London 2018 - Els Aerts - User Research Done Right
 
SearchLove London 2016 | Amy Harrison | Stand out to YOUR Crowd: A Simple Fra...
SearchLove London 2016 | Amy Harrison | Stand out to YOUR Crowd: A Simple Fra...SearchLove London 2016 | Amy Harrison | Stand out to YOUR Crowd: A Simple Fra...
SearchLove London 2016 | Amy Harrison | Stand out to YOUR Crowd: A Simple Fra...
 
SEO Success Factors - SMX Advanced 2014
SEO Success Factors - SMX Advanced 2014SEO Success Factors - SMX Advanced 2014
SEO Success Factors - SMX Advanced 2014
 
SearchLove London 2018 - John Campbell - Voice Search – Calculating and Seizi...
SearchLove London 2018 - John Campbell - Voice Search – Calculating and Seizi...SearchLove London 2018 - John Campbell - Voice Search – Calculating and Seizi...
SearchLove London 2018 - John Campbell - Voice Search – Calculating and Seizi...
 
SearchLove San Diego 2017 | Rob Bucci | Snooping Into Google's Insights on Se...
SearchLove San Diego 2017 | Rob Bucci | Snooping Into Google's Insights on Se...SearchLove San Diego 2017 | Rob Bucci | Snooping Into Google's Insights on Se...
SearchLove San Diego 2017 | Rob Bucci | Snooping Into Google's Insights on Se...
 
3 New Techniques for the Modern Age of SEO
3 New Techniques for the Modern Age of SEO3 New Techniques for the Modern Age of SEO
3 New Techniques for the Modern Age of SEO
 
SearchLove San Diego 2018 | Will Critchlow | From the Horse’s Mouth: What We ...
SearchLove San Diego 2018 | Will Critchlow | From the Horse’s Mouth: What We ...SearchLove San Diego 2018 | Will Critchlow | From the Horse’s Mouth: What We ...
SearchLove San Diego 2018 | Will Critchlow | From the Horse’s Mouth: What We ...
 
SearchLove London 2016 | Bridget Randolph | The Changing Landscape of Mobile ...
SearchLove London 2016 | Bridget Randolph | The Changing Landscape of Mobile ...SearchLove London 2016 | Bridget Randolph | The Changing Landscape of Mobile ...
SearchLove London 2016 | Bridget Randolph | The Changing Landscape of Mobile ...
 
Next Era of SEO: A Guide to SEO Split-Testing
Next Era of SEO: A Guide to SEO Split-TestingNext Era of SEO: A Guide to SEO Split-Testing
Next Era of SEO: A Guide to SEO Split-Testing
 
SearchLove London 2016 | Tom Anthony | SEO Split-Testing - How You can Run Te...
SearchLove London 2016 | Tom Anthony | SEO Split-Testing - How You can Run Te...SearchLove London 2016 | Tom Anthony | SEO Split-Testing - How You can Run Te...
SearchLove London 2016 | Tom Anthony | SEO Split-Testing - How You can Run Te...
 
Google's Hummingbird and the Entity Search Revolution
Google's Hummingbird and the Entity Search RevolutionGoogle's Hummingbird and the Entity Search Revolution
Google's Hummingbird and the Entity Search Revolution
 
SearchLove London 2016 | Larry Kim | Hacking RankBrain and Other Machine Lear...
SearchLove London 2016 | Larry Kim | Hacking RankBrain and Other Machine Lear...SearchLove London 2016 | Larry Kim | Hacking RankBrain and Other Machine Lear...
SearchLove London 2016 | Larry Kim | Hacking RankBrain and Other Machine Lear...
 
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments' SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
SearchLove London | Dave Sottimano, 'Using Data to Win Arguments'
 

Semelhante a SearchLove Boston 2017 | Will Critchlow | Building Robot Allegiances

Knowing Ranking Factors won't be enough!
Knowing Ranking Factors won't be enough!Knowing Ranking Factors won't be enough!
Knowing Ranking Factors won't be enough!Mark Orr
 
Lessons from SEO split-testing
Lessons from SEO split-testingLessons from SEO split-testing
Lessons from SEO split-testingWill Critchlow
 
BrightonSEO: How to generate 8 million SEO test ideas - Will Critchlow
BrightonSEO: How to generate 8 million SEO test ideas - Will CritchlowBrightonSEO: How to generate 8 million SEO test ideas - Will Critchlow
BrightonSEO: How to generate 8 million SEO test ideas - Will CritchlowWill Critchlow
 
How a year of SEO split testing changed how I thought SEO worked
How a year of SEO split testing changed how I thought SEO workedHow a year of SEO split testing changed how I thought SEO worked
How a year of SEO split testing changed how I thought SEO workedDominic Woodman
 
Naming Things (with notes)
Naming Things (with notes)Naming Things (with notes)
Naming Things (with notes)Pete Nicholls
 
SearchLove London 2018 - Dom Woodman - A year of SEO split testing changed ho...
SearchLove London 2018 - Dom Woodman - A year of SEO split testing changed ho...SearchLove London 2018 - Dom Woodman - A year of SEO split testing changed ho...
SearchLove London 2018 - Dom Woodman - A year of SEO split testing changed ho...Distilled
 
SearchLove San Diego - Dom Woodman - A Year of SEO Split Testing Changed How ...
SearchLove San Diego - Dom Woodman - A Year of SEO Split Testing Changed How ...SearchLove San Diego - Dom Woodman - A Year of SEO Split Testing Changed How ...
SearchLove San Diego - Dom Woodman - A Year of SEO Split Testing Changed How ...Distilled
 
Analytics for SEO
Analytics for SEOAnalytics for SEO
Analytics for SEOIan Lurie
 
Amazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEOAmazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEOWill Critchlow
 
CFP workshop
CFP workshopCFP workshop
CFP workshopAmit Zur
 
SEO meetup Utrecht
SEO meetup UtrechtSEO meetup Utrecht
SEO meetup UtrechtRoy Huiskes
 
Using SEO to Build Your Business
Using SEO to Build Your BusinessUsing SEO to Build Your Business
Using SEO to Build Your BusinessKatie Spence
 
Five Ways to Get Better Data From Our Users
Five Ways to Get Better Data From Our UsersFive Ways to Get Better Data From Our Users
Five Ways to Get Better Data From Our UsersSajid Reshamwala
 
Get More Sales Through Better Search Engine Optimization (SEO) and Social Eng...
Get More Sales Through Better Search Engine Optimization (SEO) and Social Eng...Get More Sales Through Better Search Engine Optimization (SEO) and Social Eng...
Get More Sales Through Better Search Engine Optimization (SEO) and Social Eng...Duncan Connor
 
Using SEO to Build Your Business
Using SEO to Build Your BusinessUsing SEO to Build Your Business
Using SEO to Build Your BusinessSpryIdeas
 
Master the essentials of conversion optimization
Master the essentials of conversion optimizationMaster the essentials of conversion optimization
Master the essentials of conversion optimizationArnas Rackauskas
 

Semelhante a SearchLove Boston 2017 | Will Critchlow | Building Robot Allegiances (20)

Knowing Ranking Factors won't be enough!
Knowing Ranking Factors won't be enough!Knowing Ranking Factors won't be enough!
Knowing Ranking Factors won't be enough!
 
Lessons from SEO split-testing
Lessons from SEO split-testingLessons from SEO split-testing
Lessons from SEO split-testing
 
BrightonSEO: How to generate 8 million SEO test ideas - Will Critchlow
BrightonSEO: How to generate 8 million SEO test ideas - Will CritchlowBrightonSEO: How to generate 8 million SEO test ideas - Will Critchlow
BrightonSEO: How to generate 8 million SEO test ideas - Will Critchlow
 
How a year of SEO split testing changed how I thought SEO worked
How a year of SEO split testing changed how I thought SEO workedHow a year of SEO split testing changed how I thought SEO worked
How a year of SEO split testing changed how I thought SEO worked
 
SEO: Debunking The Myths
SEO: Debunking The MythsSEO: Debunking The Myths
SEO: Debunking The Myths
 
Naming Things (with notes)
Naming Things (with notes)Naming Things (with notes)
Naming Things (with notes)
 
SearchLove London 2018 - Dom Woodman - A year of SEO split testing changed ho...
SearchLove London 2018 - Dom Woodman - A year of SEO split testing changed ho...SearchLove London 2018 - Dom Woodman - A year of SEO split testing changed ho...
SearchLove London 2018 - Dom Woodman - A year of SEO split testing changed ho...
 
SearchLove San Diego - Dom Woodman - A Year of SEO Split Testing Changed How ...
SearchLove San Diego - Dom Woodman - A Year of SEO Split Testing Changed How ...SearchLove San Diego - Dom Woodman - A Year of SEO Split Testing Changed How ...
SearchLove San Diego - Dom Woodman - A Year of SEO Split Testing Changed How ...
 
Analytics for SEO
Analytics for SEOAnalytics for SEO
Analytics for SEO
 
Amazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEOAmazon Search Summit - the need for split testing in SEO
Amazon Search Summit - the need for split testing in SEO
 
CFP workshop
CFP workshopCFP workshop
CFP workshop
 
SEO meetup Utrecht
SEO meetup UtrechtSEO meetup Utrecht
SEO meetup Utrecht
 
Using SEO to Build Your Business
Using SEO to Build Your BusinessUsing SEO to Build Your Business
Using SEO to Build Your Business
 
Ahrefs Tutorial
Ahrefs TutorialAhrefs Tutorial
Ahrefs Tutorial
 
Naming Things
Naming ThingsNaming Things
Naming Things
 
Five Ways to Get Better Data From Our Users
Five Ways to Get Better Data From Our UsersFive Ways to Get Better Data From Our Users
Five Ways to Get Better Data From Our Users
 
Get More Sales Through Better Search Engine Optimization (SEO) and Social Eng...
Get More Sales Through Better Search Engine Optimization (SEO) and Social Eng...Get More Sales Through Better Search Engine Optimization (SEO) and Social Eng...
Get More Sales Through Better Search Engine Optimization (SEO) and Social Eng...
 
Using SEO to Build Your Business
Using SEO to Build Your BusinessUsing SEO to Build Your Business
Using SEO to Build Your Business
 
Master the essentials of conversion optimization
Master the essentials of conversion optimizationMaster the essentials of conversion optimization
Master the essentials of conversion optimization
 
Scalding at Etsy
Scalding at EtsyScalding at Etsy
Scalding at Etsy
 

Mais de Distilled

SearchLove London 2019 - Will Critchlow - Misunderstood Concepts at the Heart...
SearchLove London 2019 - Will Critchlow - Misunderstood Concepts at the Heart...SearchLove London 2019 - Will Critchlow - Misunderstood Concepts at the Heart...
SearchLove London 2019 - Will Critchlow - Misunderstood Concepts at the Heart...Distilled
 
SearchLove London 2019 - Stacey MacNaught - Actioning Search Intent: What to ...
SearchLove London 2019 - Stacey MacNaught - Actioning Search Intent: What to ...SearchLove London 2019 - Stacey MacNaught - Actioning Search Intent: What to ...
SearchLove London 2019 - Stacey MacNaught - Actioning Search Intent: What to ...Distilled
 
SearchLove London 2019 - Lindsay Wassell - Managing Multinational & Multiling...
SearchLove London 2019 - Lindsay Wassell - Managing Multinational & Multiling...SearchLove London 2019 - Lindsay Wassell - Managing Multinational & Multiling...
SearchLove London 2019 - Lindsay Wassell - Managing Multinational & Multiling...Distilled
 
SearchLove London 2019 - Dr. Pete Meyers - Scaling Keyword Research: More Isn...
SearchLove London 2019 - Dr. Pete Meyers - Scaling Keyword Research: More Isn...SearchLove London 2019 - Dr. Pete Meyers - Scaling Keyword Research: More Isn...
SearchLove London 2019 - Dr. Pete Meyers - Scaling Keyword Research: More Isn...Distilled
 
SearchLoveLondon 2019 - Faisal Anderson - Spying on Google: Using Log File An...
SearchLoveLondon 2019 - Faisal Anderson - Spying on Google: Using Log File An...SearchLoveLondon 2019 - Faisal Anderson - Spying on Google: Using Log File An...
SearchLoveLondon 2019 - Faisal Anderson - Spying on Google: Using Log File An...Distilled
 
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your Audience
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your AudienceSearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your Audience
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your AudienceDistilled
 
SearchLove London 2019 - Rand Fishkin - The Search Landscape in 2019
SearchLove London 2019 - Rand Fishkin - The Search Landscape in 2019SearchLove London 2019 - Rand Fishkin - The Search Landscape in 2019
SearchLove London 2019 - Rand Fishkin - The Search Landscape in 2019Distilled
 
SearchLove London 2019 - Jes Scholtz - Giving Robots an All Access Pass
SearchLove London 2019 - Jes Scholtz - Giving Robots an All Access PassSearchLove London 2019 - Jes Scholtz - Giving Robots an All Access Pass
SearchLove London 2019 - Jes Scholtz - Giving Robots an All Access PassDistilled
 
SearchLove London 2019 - Heather Physioc - Building a Discoverability Powerhouse
SearchLove London 2019 - Heather Physioc - Building a Discoverability PowerhouseSearchLove London 2019 - Heather Physioc - Building a Discoverability Powerhouse
SearchLove London 2019 - Heather Physioc - Building a Discoverability PowerhouseDistilled
 
SearchLove London 2019 - Andi Jarvis - The Science of Persuasion
SearchLove London 2019 - Andi Jarvis - The Science of PersuasionSearchLove London 2019 - Andi Jarvis - The Science of Persuasion
SearchLove London 2019 - Andi Jarvis - The Science of PersuasionDistilled
 
SearchLove London 2019 - Luke Carthy - Finding Powerful CRO and UX Opportunit...
SearchLove London 2019 - Luke Carthy - Finding Powerful CRO and UX Opportunit...SearchLove London 2019 - Luke Carthy - Finding Powerful CRO and UX Opportunit...
SearchLove London 2019 - Luke Carthy - Finding Powerful CRO and UX Opportunit...Distilled
 
SearchLove London 2019 - Greg Gifford - Doc Brown's Plutonium-powered Local S...
SearchLove London 2019 - Greg Gifford - Doc Brown's Plutonium-powered Local S...SearchLove London 2019 - Greg Gifford - Doc Brown's Plutonium-powered Local S...
SearchLove London 2019 - Greg Gifford - Doc Brown's Plutonium-powered Local S...Distilled
 
SearchLove London 2019 - Sarah Gurbach - Using Qualitative Data to Make Human...
SearchLove London 2019 - Sarah Gurbach - Using Qualitative Data to Make Human...SearchLove London 2019 - Sarah Gurbach - Using Qualitative Data to Make Human...
SearchLove London 2019 - Sarah Gurbach - Using Qualitative Data to Make Human...Distilled
 
SearchLove London 2019 - Marie Haynes - Practical Tips for Improving E-A-T
SearchLove London 2019 - Marie Haynes - Practical Tips for Improving E-A-TSearchLove London 2019 - Marie Haynes - Practical Tips for Improving E-A-T
SearchLove London 2019 - Marie Haynes - Practical Tips for Improving E-A-TDistilled
 
SearchLove Boston 2019 - Rand Fishkin - Building Influence in 2019
SearchLove Boston 2019 - Rand Fishkin - Building Influence in 2019SearchLove Boston 2019 - Rand Fishkin - Building Influence in 2019
SearchLove Boston 2019 - Rand Fishkin - Building Influence in 2019Distilled
 
SearchLove Boston 2019 - Courtney Cox Wakefield - Voice Search and Instant An...
SearchLove Boston 2019 - Courtney Cox Wakefield - Voice Search and Instant An...SearchLove Boston 2019 - Courtney Cox Wakefield - Voice Search and Instant An...
SearchLove Boston 2019 - Courtney Cox Wakefield - Voice Search and Instant An...Distilled
 
SearchLove Boston 2019 - Tom Anthony - Search in 2020: Technologies That Will...
SearchLove Boston 2019 - Tom Anthony - Search in 2020: Technologies That Will...SearchLove Boston 2019 - Tom Anthony - Search in 2020: Technologies That Will...
SearchLove Boston 2019 - Tom Anthony - Search in 2020: Technologies That Will...Distilled
 
SearchLove Boston 2019 - Derek Gleason - Benchmarking Success for Client Site...
SearchLove Boston 2019 - Derek Gleason - Benchmarking Success for Client Site...SearchLove Boston 2019 - Derek Gleason - Benchmarking Success for Client Site...
SearchLove Boston 2019 - Derek Gleason - Benchmarking Success for Client Site...Distilled
 
SearchLove Boston 2019 - Kameron Jenkins - The Modern Search Writer’s Toolkit
SearchLove Boston 2019 - Kameron Jenkins - The Modern Search Writer’s ToolkitSearchLove Boston 2019 - Kameron Jenkins - The Modern Search Writer’s Toolkit
SearchLove Boston 2019 - Kameron Jenkins - The Modern Search Writer’s ToolkitDistilled
 
SearchLove Boston 2019 - Joy Hawkins - 10 Ways to Get Results with Local SEO
SearchLove Boston 2019 - Joy Hawkins - 10 Ways to Get Results with Local SEOSearchLove Boston 2019 - Joy Hawkins - 10 Ways to Get Results with Local SEO
SearchLove Boston 2019 - Joy Hawkins - 10 Ways to Get Results with Local SEODistilled
 

Mais de Distilled (20)

SearchLove London 2019 - Will Critchlow - Misunderstood Concepts at the Heart...
SearchLove London 2019 - Will Critchlow - Misunderstood Concepts at the Heart...SearchLove London 2019 - Will Critchlow - Misunderstood Concepts at the Heart...
SearchLove London 2019 - Will Critchlow - Misunderstood Concepts at the Heart...
 
SearchLove London 2019 - Stacey MacNaught - Actioning Search Intent: What to ...
SearchLove London 2019 - Stacey MacNaught - Actioning Search Intent: What to ...SearchLove London 2019 - Stacey MacNaught - Actioning Search Intent: What to ...
SearchLove London 2019 - Stacey MacNaught - Actioning Search Intent: What to ...
 
SearchLove London 2019 - Lindsay Wassell - Managing Multinational & Multiling...
SearchLove London 2019 - Lindsay Wassell - Managing Multinational & Multiling...SearchLove London 2019 - Lindsay Wassell - Managing Multinational & Multiling...
SearchLove London 2019 - Lindsay Wassell - Managing Multinational & Multiling...
 
SearchLove London 2019 - Dr. Pete Meyers - Scaling Keyword Research: More Isn...
SearchLove London 2019 - Dr. Pete Meyers - Scaling Keyword Research: More Isn...SearchLove London 2019 - Dr. Pete Meyers - Scaling Keyword Research: More Isn...
SearchLove London 2019 - Dr. Pete Meyers - Scaling Keyword Research: More Isn...
 
SearchLoveLondon 2019 - Faisal Anderson - Spying on Google: Using Log File An...
SearchLoveLondon 2019 - Faisal Anderson - Spying on Google: Using Log File An...SearchLoveLondon 2019 - Faisal Anderson - Spying on Google: Using Log File An...
SearchLoveLondon 2019 - Faisal Anderson - Spying on Google: Using Log File An...
 
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your Audience
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your AudienceSearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your Audience
SearchLove London 2019 - Rory Truesdale - Using the SERPs to Know Your Audience
 
SearchLove London 2019 - Rand Fishkin - The Search Landscape in 2019
SearchLove London 2019 - Rand Fishkin - The Search Landscape in 2019SearchLove London 2019 - Rand Fishkin - The Search Landscape in 2019
SearchLove London 2019 - Rand Fishkin - The Search Landscape in 2019
 
SearchLove London 2019 - Jes Scholtz - Giving Robots an All Access Pass
SearchLove London 2019 - Jes Scholtz - Giving Robots an All Access PassSearchLove London 2019 - Jes Scholtz - Giving Robots an All Access Pass
SearchLove London 2019 - Jes Scholtz - Giving Robots an All Access Pass
 
SearchLove London 2019 - Heather Physioc - Building a Discoverability Powerhouse
SearchLove London 2019 - Heather Physioc - Building a Discoverability PowerhouseSearchLove London 2019 - Heather Physioc - Building a Discoverability Powerhouse
SearchLove London 2019 - Heather Physioc - Building a Discoverability Powerhouse
 
SearchLove London 2019 - Andi Jarvis - The Science of Persuasion
SearchLove London 2019 - Andi Jarvis - The Science of PersuasionSearchLove London 2019 - Andi Jarvis - The Science of Persuasion
SearchLove London 2019 - Andi Jarvis - The Science of Persuasion
 
SearchLove London 2019 - Luke Carthy - Finding Powerful CRO and UX Opportunit...
SearchLove London 2019 - Luke Carthy - Finding Powerful CRO and UX Opportunit...SearchLove London 2019 - Luke Carthy - Finding Powerful CRO and UX Opportunit...
SearchLove London 2019 - Luke Carthy - Finding Powerful CRO and UX Opportunit...
 
SearchLove London 2019 - Greg Gifford - Doc Brown's Plutonium-powered Local S...
SearchLove London 2019 - Greg Gifford - Doc Brown's Plutonium-powered Local S...SearchLove London 2019 - Greg Gifford - Doc Brown's Plutonium-powered Local S...
SearchLove London 2019 - Greg Gifford - Doc Brown's Plutonium-powered Local S...
 
SearchLove London 2019 - Sarah Gurbach - Using Qualitative Data to Make Human...
SearchLove London 2019 - Sarah Gurbach - Using Qualitative Data to Make Human...SearchLove London 2019 - Sarah Gurbach - Using Qualitative Data to Make Human...
SearchLove London 2019 - Sarah Gurbach - Using Qualitative Data to Make Human...
 
SearchLove London 2019 - Marie Haynes - Practical Tips for Improving E-A-T
SearchLove London 2019 - Marie Haynes - Practical Tips for Improving E-A-TSearchLove London 2019 - Marie Haynes - Practical Tips for Improving E-A-T
SearchLove London 2019 - Marie Haynes - Practical Tips for Improving E-A-T
 
SearchLove Boston 2019 - Rand Fishkin - Building Influence in 2019
SearchLove Boston 2019 - Rand Fishkin - Building Influence in 2019SearchLove Boston 2019 - Rand Fishkin - Building Influence in 2019
SearchLove Boston 2019 - Rand Fishkin - Building Influence in 2019
 
SearchLove Boston 2019 - Courtney Cox Wakefield - Voice Search and Instant An...
SearchLove Boston 2019 - Courtney Cox Wakefield - Voice Search and Instant An...SearchLove Boston 2019 - Courtney Cox Wakefield - Voice Search and Instant An...
SearchLove Boston 2019 - Courtney Cox Wakefield - Voice Search and Instant An...
 
SearchLove Boston 2019 - Tom Anthony - Search in 2020: Technologies That Will...
SearchLove Boston 2019 - Tom Anthony - Search in 2020: Technologies That Will...SearchLove Boston 2019 - Tom Anthony - Search in 2020: Technologies That Will...
SearchLove Boston 2019 - Tom Anthony - Search in 2020: Technologies That Will...
 
SearchLove Boston 2019 - Derek Gleason - Benchmarking Success for Client Site...
SearchLove Boston 2019 - Derek Gleason - Benchmarking Success for Client Site...SearchLove Boston 2019 - Derek Gleason - Benchmarking Success for Client Site...
SearchLove Boston 2019 - Derek Gleason - Benchmarking Success for Client Site...
 
SearchLove Boston 2019 - Kameron Jenkins - The Modern Search Writer’s Toolkit
SearchLove Boston 2019 - Kameron Jenkins - The Modern Search Writer’s ToolkitSearchLove Boston 2019 - Kameron Jenkins - The Modern Search Writer’s Toolkit
SearchLove Boston 2019 - Kameron Jenkins - The Modern Search Writer’s Toolkit
 
SearchLove Boston 2019 - Joy Hawkins - 10 Ways to Get Results with Local SEO
SearchLove Boston 2019 - Joy Hawkins - 10 Ways to Get Results with Local SEOSearchLove Boston 2019 - Joy Hawkins - 10 Ways to Get Results with Local SEO
SearchLove Boston 2019 - Joy Hawkins - 10 Ways to Get Results with Local SEO
 

Último

How to Leverage Behavioral Science Insights for Direct Mail Success
How to Leverage Behavioral Science Insights for Direct Mail SuccessHow to Leverage Behavioral Science Insights for Direct Mail Success
How to Leverage Behavioral Science Insights for Direct Mail SuccessAggregage
 
Brighton SEO April 2024 - The Good, the Bad & the Ugly of SEO Success
Brighton SEO April 2024 - The Good, the Bad & the Ugly of SEO SuccessBrighton SEO April 2024 - The Good, the Bad & the Ugly of SEO Success
Brighton SEO April 2024 - The Good, the Bad & the Ugly of SEO SuccessVarn
 
Social Media Marketing PPT-Includes Paid media
Social Media Marketing PPT-Includes Paid mediaSocial Media Marketing PPT-Includes Paid media
Social Media Marketing PPT-Includes Paid mediaadityabelde2
 
BDSM⚡Call Girls in Sector 150 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 150 Noida Escorts >༒8448380779 Escort ServiceBDSM⚡Call Girls in Sector 150 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 150 Noida Escorts >༒8448380779 Escort ServiceDelhi Call girls
 
The+State+of+Careers+In+Retention+Marketing-2.pdf
The+State+of+Careers+In+Retention+Marketing-2.pdfThe+State+of+Careers+In+Retention+Marketing-2.pdf
The+State+of+Careers+In+Retention+Marketing-2.pdfSocial Samosa
 
Defining Marketing for the 21st Century,kotler
Defining Marketing for the 21st Century,kotlerDefining Marketing for the 21st Century,kotler
Defining Marketing for the 21st Century,kotlerAmirNasiruog
 
personal branding kit for music business
personal branding kit for music businesspersonal branding kit for music business
personal branding kit for music businessbrjohnson6
 
Brand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdfBrand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdftbatkhuu1
 
Call Us ➥9654467111▻Call Girls In Delhi NCR
Call Us ➥9654467111▻Call Girls In Delhi NCRCall Us ➥9654467111▻Call Girls In Delhi NCR
Call Us ➥9654467111▻Call Girls In Delhi NCRSapana Sha
 
Unraveling the Mystery of the Hinterkaifeck Murders.pptx
Unraveling the Mystery of the Hinterkaifeck Murders.pptxUnraveling the Mystery of the Hinterkaifeck Murders.pptx
Unraveling the Mystery of the Hinterkaifeck Murders.pptxelizabethella096
 
Factors-Influencing-Branding-Strategies.pptx
Factors-Influencing-Branding-Strategies.pptxFactors-Influencing-Branding-Strategies.pptx
Factors-Influencing-Branding-Strategies.pptxVikasTiwari846641
 
BDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort ServiceBDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort ServiceDelhi Call girls
 
Brand experience Dream Center Peoria Presentation.pdf
Brand experience Dream Center Peoria Presentation.pdfBrand experience Dream Center Peoria Presentation.pdf
Brand experience Dream Center Peoria Presentation.pdftbatkhuu1
 
Google 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
Google 3rd-Party Cookie Deprecation [Update] + 5 Best StrategiesGoogle 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
Google 3rd-Party Cookie Deprecation [Update] + 5 Best StrategiesSearch Engine Journal
 
The Science of Landing Page Messaging.pdf
The Science of Landing Page Messaging.pdfThe Science of Landing Page Messaging.pdf
The Science of Landing Page Messaging.pdfVWO
 

Último (20)

How to Leverage Behavioral Science Insights for Direct Mail Success
How to Leverage Behavioral Science Insights for Direct Mail SuccessHow to Leverage Behavioral Science Insights for Direct Mail Success
How to Leverage Behavioral Science Insights for Direct Mail Success
 
Brighton SEO April 2024 - The Good, the Bad & the Ugly of SEO Success
Brighton SEO April 2024 - The Good, the Bad & the Ugly of SEO SuccessBrighton SEO April 2024 - The Good, the Bad & the Ugly of SEO Success
Brighton SEO April 2024 - The Good, the Bad & the Ugly of SEO Success
 
Social Media Marketing PPT-Includes Paid media
Social Media Marketing PPT-Includes Paid mediaSocial Media Marketing PPT-Includes Paid media
Social Media Marketing PPT-Includes Paid media
 
Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...
Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...
Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...
 
Generative AI Master Class - Generative AI, Unleash Creative Opportunity - Pe...
Generative AI Master Class - Generative AI, Unleash Creative Opportunity - Pe...Generative AI Master Class - Generative AI, Unleash Creative Opportunity - Pe...
Generative AI Master Class - Generative AI, Unleash Creative Opportunity - Pe...
 
BDSM⚡Call Girls in Sector 150 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 150 Noida Escorts >༒8448380779 Escort ServiceBDSM⚡Call Girls in Sector 150 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 150 Noida Escorts >༒8448380779 Escort Service
 
The+State+of+Careers+In+Retention+Marketing-2.pdf
The+State+of+Careers+In+Retention+Marketing-2.pdfThe+State+of+Careers+In+Retention+Marketing-2.pdf
The+State+of+Careers+In+Retention+Marketing-2.pdf
 
Defining Marketing for the 21st Century,kotler
Defining Marketing for the 21st Century,kotlerDefining Marketing for the 21st Century,kotler
Defining Marketing for the 21st Century,kotler
 
personal branding kit for music business
personal branding kit for music businesspersonal branding kit for music business
personal branding kit for music business
 
Brand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdfBrand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdf
 
Call Us ➥9654467111▻Call Girls In Delhi NCR
Call Us ➥9654467111▻Call Girls In Delhi NCRCall Us ➥9654467111▻Call Girls In Delhi NCR
Call Us ➥9654467111▻Call Girls In Delhi NCR
 
Creator Influencer Strategy Master Class - Corinne Rose Guirgis
Creator Influencer Strategy Master Class - Corinne Rose GuirgisCreator Influencer Strategy Master Class - Corinne Rose Guirgis
Creator Influencer Strategy Master Class - Corinne Rose Guirgis
 
Unraveling the Mystery of the Hinterkaifeck Murders.pptx
Unraveling the Mystery of the Hinterkaifeck Murders.pptxUnraveling the Mystery of the Hinterkaifeck Murders.pptx
Unraveling the Mystery of the Hinterkaifeck Murders.pptx
 
No Cookies No Problem - Steve Krull, Be Found Online
No Cookies No Problem - Steve Krull, Be Found OnlineNo Cookies No Problem - Steve Krull, Be Found Online
No Cookies No Problem - Steve Krull, Be Found Online
 
Factors-Influencing-Branding-Strategies.pptx
Factors-Influencing-Branding-Strategies.pptxFactors-Influencing-Branding-Strategies.pptx
Factors-Influencing-Branding-Strategies.pptx
 
BDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort ServiceBDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort Service
BDSM⚡Call Girls in Sector 144 Noida Escorts >༒8448380779 Escort Service
 
Brand experience Dream Center Peoria Presentation.pdf
Brand experience Dream Center Peoria Presentation.pdfBrand experience Dream Center Peoria Presentation.pdf
Brand experience Dream Center Peoria Presentation.pdf
 
Foundation First - Why Your Website and Content Matters - David Pisarek
Foundation First - Why Your Website and Content Matters - David PisarekFoundation First - Why Your Website and Content Matters - David Pisarek
Foundation First - Why Your Website and Content Matters - David Pisarek
 
Google 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
Google 3rd-Party Cookie Deprecation [Update] + 5 Best StrategiesGoogle 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
Google 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
 
The Science of Landing Page Messaging.pdf
The Science of Landing Page Messaging.pdfThe Science of Landing Page Messaging.pdf
The Science of Landing Page Messaging.pdf
 

SearchLove Boston 2017 | Will Critchlow | Building Robot Allegiances

  • 1. Knowing ranking factors won’t be enough How to avoid losing your job to a robot @willcritchlow
  • 2. I’m going to tell you about a robot that understands ranking factors better than any of you ...but before I get to that, let’s look at a bit of history...
  • 3. The other day I searched:
  • 5. But it got me thinking about how, in 2009, the results would have looked more like this.
  • 6. In 2009, it would have looked more like this. With every title containing the keyphrase.
  • 7. In 2009, it would have looked more like this. With every title containing the keyphrase. Most at the beginning.
  • 9.
  • 10. My mental model for ~2009 ranking factors had three different modes:
  • 11. One in the hyper-competitive head My mental model for ~2009 ranking factors had three different modes: One in the competitive mid-tail ...and one in the long-tail
  • 13. Tons of perfectly on-topic pages to choose from One in the hyper-competitive head
  • 14. So pick only perfectly-on-topic pages One in the hyper-competitive head
  • 15. (*) Page authority, but the domain inevitably factors into that calculation. This is why so many homepages ranked One in the hyper-competitive head ...and rank by authority (*)
  • 16. This resulted in a mix of homepages of mid-size sites, and inner pages on huge sites One in the hyper-competitive head
  • 17. But the general way to move up was through increased authority One in the hyper-competitive head
  • 18. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Long-tail
  • 19. One in the hyper-competitive head One in the competitive mid-tail
  • 20. Wealth of ROUGHLY on-topic pages to choose from One in the competitive mid-tail
  • 21. PERFECTLY on-topic could do well even on a relatively weak site One in the competitive mid-tail
  • 22. Rank the roughly on-topic pages by authority x “on-topicness” One in the competitive mid-tail
  • 23. Move up with better targeting or more authority One in the competitive mid-tail
  • 24. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail
  • 25. One in the hyper-competitive head One in the competitive mid-tail ...and one in the long-tail
  • 26. In the long-tail, a site of arbitrary weakness could rank if it was the most relevant ...and one in the long-tail
  • 27. Otherwise, massive sites rank with off-topic pages that mention something similar ...and one in the long-tail
  • 28. Generally, move up with better targeting ...and one in the long-tail
  • 29. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail Arbitrarily-weak on-topic pages and roughly-targeted deep pages on massive sites. Improve targeting.
  • 30. Kind of search result Pages ranking To move up... Head Homepages of mid-size sites and inner pages of massive sites. All perfectly-targeted. Improve authority. Mid-tail Perfectly on-topic pages on relatively weak sites plus roughly on-topic on bigger sites. Improve targeting or authority. Long-tail Arbitrarily-weak on-topic pages and roughly-targeted deep pages on massive sites. Improve targeting. So that was ~2009
  • 31. It’s not so simple any more. Google is harder to understand these days.
  • 32. PageRank (the first algorithm to use the link structure of the web) We know how we got to ~2009...
  • 35. Information retrieval PageRank Original research TWEAKS ...with growing complexity in subsequent years
  • 36. Particularly this comment from a user called Kevin Lacker (@lacker):
  • 37. I was thinking about it like it was a math puzzle and if I just thought really hard it would all make sense. -- Kevin Lacker (@lacker)
  • 38. Hey why don't you take the square root? -- Amit Singhal according to Kevin Lacker (@lacker)
  • 39. oh... am I allowed to write code that doesn't make any sense? -- Kevin Lacker (@lacker)
  • 40. -- Amit Singhal according to Kevin Lacker (@lacker) Multiply by 2 if it helps, add 5, whatever, just make things work and we can make it make sense later.
  • 41.
  • 46. You might know what any one of the levers does, but they can interact with each other in complex ways This is what a high-dimensional function looks like
  • 48. We sell custom cigar humidors. Our custom cigar humidors are handmade. If you’re thinking of buying a custom cigar humidor, please contact our custom cigar humidor specialists at custom.cigar.humidors@example.com What this needs is another mention of [cigar humidors]
  • 49. With no mentions of [cigar] or [humidor] this page would be unlikely to rank And yet you can clearly go too far, and have the effect turn negative. This is called nonlinearity. The cigar example is taken directly from Google’s quality guidelines.
  • 51. Discontinuities are steps in the function Think about so-called “over-optimization” tipping points
  • 52.
  • 53. Think about category pages: Do you recommend removing “SEO text”? We’ve tested it, so we know the answer.
  • 54. If you said “yes”, congratulations (+3.1% organic sessions in a split-test)
  • 55. Unless you’re responsible for this site No effect / possible negative effect
  • 56. No, but I’m still pretty good at this You’re thinking this to yourself right now.
  • 57.
  • 58.
  • 59.
  • 60.
  • 61.
  • 62.
  • 63.
  • 64.
  • 65.
  • 66.
  • 67.
  • 68.
  • 69.
  • 70.
  • 71.
  • 72.
  • 73. I promised to tell you about a robot that is better than even experienced SEOs... Well. It turns out all we needed was a coin to flip. You’re all fired.
  • 74.
  • 75.
  • 76.
  • 77. It’s only going to get worse under Sundar Pichai
  • 78. Who knows who this is? (This is the only CC-licensed photo of him on the internet)
  • 80. John Giannandrea - Google’s head of search Sundar’s choice to lead search after Amit. Previously running machine learning.
  • 81. ...and of course Jeff Dean is doing Jeff Dean things (c.f. Chuck Norris)
  • 82. Jeff Dean puts his pants on one leg at a time, but if he had more legs, you would see that his approach is O(log n). Source: Jeff Dean facts
  • 83. Once, in early 2002, when the search back-ends went down, Jeff Dean answered user queries manually for two hours. Result quality improved markedly during this time
  • 84. When Jeff Dean goes on vacation, production services across Google mysteriously stop working within a few days. This was reportedly actually true
  • 85. The original Google Translate was the result of the work of hundreds of engineers over 10 years.
  • 86. Director of Translate, Macduff Hughes said that it sounded to him as if maybe they could pull off a neural-network-based replacement in three years.
  • 87. Jeff Dean said “we can do it by the end of the year, if we put our minds to it”.
  • 88. Hughes: “I’m not going to be the one to say Jeff Dean can’t deliver speed.”
  • 89. A month later, the work of a team of 3 engineers was tested against the existing system. The improvement was roughly equivalent to the improvement of the old system over the previous 10 years.
  • 90. Hughes sent his team an email. All projects on the old system were to be suspended immediately. [Read the whole story ]
  • 92. How to avoid losing your job to a robot This is what you promised, Will.
  • 95. Ooh. Ooh. I know this one. -- robot
  • 96. “It’s a leopard. I’m like 99% sure.”
  • 97. Computers are better than humans at classification, but struggle with adversaries Read more about this here -- Cheetah, Leopard, Jaguar
  • 98. We don’t fully understand all ML mistakes See: adversarial AI
  • 99. And when you’re trying to fool the machine... See: adversarial AI
  • 100. And when you’re trying to fool the machine... See: adversarial AI
  • 101. You get some really wild examples See: adversarial AI
  • 102. Lesson: We expect adversarial abilities to take a step backwards They will remain good at classifying bad links but will be likely to fall prey to weird outcomes in adversarial situations
  • 103. We’re going to see new kinds of bugs
  • 104. Rules of ML [PDF] outlines engineering lessons from getting ML into production at Google
  • 105. That document also has a section on trying to understand what the machines are doing
  • 106. But human explainability may not even be possible Not every concept a neural network uses fits neatly into a concept for which we have a word. It’s not clear this is a weakness per se, but...
  • 107. ...this means that engineers won’t always know more than we do about why a page does or doesn’t rank The big knowledge gap of the future is data - clickthrough rates, bounce rates etc.
  • 108. Check out Tom Capper’s presentation on how engineers’ statements can be misleading
  • 109. ...and remember the confounding split-tests It’s already not always as simple as “feature X is good” Which all means we may need to be more independent-minded and do more of our own research
  • 110. So how do we fight back?
  • 111. Michael Lewis’ latest book is about Kahneman and Tversky spelling. It recounts a story about a piece of medical software that existed in the 1960s.
  • 112. It was designed to encapsulate how a range of doctors diagnosed stomach cancer from x-rays.
  • 113. It proceeded to outperform those same doctors despite only containing their expertise. Real people have biases, and fool themselves. Encapsulate your own expert knowledge.
  • 114. At Distilled, we use a methodology we call the balanced digital scorecard. This encapsulates our beliefs about how to build a high-performing business. Applying it helps avoid our own biases.
  • 115. Also, while we are talking about books, The Checklist Manifesto is an important part of avoiding the same cognitive biases.
  • 116. Focus on consulting skills I’ve written a few things about this (DistilledU module, writing better business documents, using split-tests to consult better). Use case studies and creativity. Computers are better at diagnosis than cure. This means: getting things done, convincing organizations, applying general knowledge, learning new things.
  • 117. We are going to need to be better than ever at debugging things. I wrote about debugging skills for non-developers here. A lot of the story of enterprise consulting is going to be about figuring out why things have gone wrong in the face of sparse or incorrect information from Google.
  • 118.
  • 119. Disregard expert surveys Firstly, there are all the problems outlined in the search result pairs study - both in the ability of experts to understand factors, and in your ability to use the information even if they do. Secondly, they are broken with another bias called the “law of small numbers” from Lewis’ book. PS - I say this as a participant in many of them Me
  • 120. Equally, building your digital strategy on what Google tells you to do will become an even worse idea than it already is.
  • 121. This is why we have been investing so much in split-testing Check out odn.distilled.net if you haven’t already. The team will be happy to demo for you. We served ~5 billion requests last quarter and recently published everything from response times to our +£100k / month split test.
  • 122. Let’s recap 1. Even in a world of 200+ “classical” ranking factors, humans were bad at understanding the algorithm
  • 123. Let’s recap 1. Even in a world of 200+ “classical” ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar
  • 124. Let’s recap 1. Even in a world of 200+ “classical” ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers
  • 125. Let’s recap 1. Even in a world of 200+ “classical” ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers 4. We remain relevant by: a. Using methodologies and checklists to capture human capabilities and avoid our biases
  • 126. Let’s recap 1. Even in a world of 200+ “classical” ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers 4. We remain relevant by: a. Using methodologies and checklists to capture human capabilities and avoid our biases b. Becoming great consultants and change agents
  • 127. Let’s recap 1. Even in a world of 200+ “classical” ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers 4. We remain relevant by: a. Using methodologies and checklists to capture human capabilities and avoid our biases b. Becoming great consultants and change agents c. Debugging the heck out of everything
  • 128. Let’s recap 1. Even in a world of 200+ “classical” ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers 4. We remain relevant by: a. Using methodologies and checklists to capture human capabilities and avoid our biases b. Becoming great consultants and change agents c. Debugging the heck out of everything d. Avoiding being misled by experts or Google
  • 129. Let’s recap 1. Even in a world of 200+ “classical” ranking factors, humans were bad at understanding the algorithm 2. Machine learning will make this worse, and is accelerating under Sundar 3. There are things computers remain bad at, and rankings will become more opaque even to Google engineers 4. We remain relevant by: a. Using methodologies and checklists to capture human capabilities and avoid our biases b. Becoming great consultants and change agents c. Debugging the heck out of everything d. Avoiding being misled by experts or Google Testing!
  • 130.
  • 131. What about that robot I promised you? The coin flip wasn’t really it
  • 132.
  • 133.
  • 135. The specifics of DeepRank We started with a broad range of unbranded keywords from our STAT rank tracking. For each of the URLs ranking in the top 10, we gathered key metrics about the domain and page - both from direct crawling and various APIs. We turned this into a set of pairs of URLs {A,B} with their associated keyword, metrics, and their rank ordering. Gather and process training data
  • 136. The specifics of DeepRank We started with a broad range of unbranded keywords from our STAT rank tracking. For each of the URLs ranking in the top 10, we gathered key metrics about the domain and page - both from direct crawling and various APIs. We turned this into a set of pairs of URLs {A,B} with their associated keyword, metrics, and their rank ordering.
  • 137. We have so far trained on just 10 metrics for a relatively small sample (hundreds) of keywords. Our current version is only a few layers deep with only 10 hidden dimensions. The current training samples 30 pairs at a time and trains against them for 500 epochs. The specifics of DeepRank Train the model Gather and process training data
  • 138. Train the model Gather and process training data The specifics of DeepRank Model The next task is to get way more metrics for thousands of keywords. This will enable us to train a much deeper model for much longer without overfitting. We also have some more hyperparameter tuning to do.
  • 139. To run the model, we input a pair of pages with their associated metrics. New input
  • 141. We get back a probability of page A outranking page B. Model Probability- weighted predictions New input
  • 142.
  • 143.
  • 144. The goal is a winning combination of human and machine Human + computer beats computer (for now)
  • 146. ● Mobius strip ● Confusion ● Signal box ● Cigar ● Discontinuity ● Confidence ● Burt Totaro ● Sundar Pichai ● John Giannandrea ● Chuck Norris ● Jeff Dean ● Fencing ● Keyboard Image credits ● Go ● Robot ● Leopard print sofa ● Leopard ● Bug ● Lego robots ● Iron Man ● Boston