A large portion of the web has already migrated to the Mobile First Index. However, there is still a huge portion still living in the world of desktop-first. To help those who have yet to make the transition, we recently dug into the data of nearly 500 million URLs over 1.5 billion Googlebot visits to learn from the sites that have fully made the move. In this talk from Botify’s Frank Vitovitch, find out the specific characteristics of sites who saw major improvements and learn the mistakes to avoid from those who lost traffic during their transition.

1. Living in a Mobile First
(Indexing) World
Frank Vitovitch // Botify
slideshare.net/BotifySEO
@FrankieVSEO

2. @botify
{
"@context": "http://schema.org",
"@type":"Person",
"Name": "Frank Vitovitch",
"City": "New York",
"worksFor": {
"@type": "Organization",
"name": "Botify",
"url": "https://www.botify.com",
"jobTitle": "Vice President, Solutions"},
"Twitter": "@FrankieVSEO",
"Blog": "https://www.botify.com/blog/",
"Skillset": [
"Technical SEO",
"Professional Services",
"Tormented Notre Dame fan",
"Proud fan of 2018 WORLD CHAMPION Philadelphia
Eagles" ]
}

3. Agenda
Background on the Mobile First Index
What is the current state of the Mobile First Index?
What appears to be influencing what sites Google moves to the MFI?
Real customer MFI transition stories - the good and the bad

4. Google is rolling out
the mobile-first index
progressively across
all websites.

5. Why are we even here, Google announced this ages ago
Google initially announced it was
introducing a mobile first index in
November 2016, but the roll out has
been slow and steady.
Our data shows that there is still a
large portion of the web still living in
a desktop first world.

6. You know all of this already, but…
No changes in regards to the first 2 fundamental
steps (crawling & indexing)
With the mobile-first index, the relevance of a page
will only depend on the mobile version
Without a mobile site, a desktop site will still be
crawled and used by Google to determine rankings

7. What is the current
state of the Mobile
First Index?

8. Why did we conduct this new study on the mobile-first index?
What is the
current state of
the mobile-first
index?
What are the
criteria that
determine whether
or not a website is
moved to the
mobile-first index?
How does the
mobile-first index
impact Google’s
crawl on
transitioned
websites?
Questions we wanted to answer:

9. Big questions, bigger stats: meet our unprecedented dataset
498
34.5B
66.8B
Unique domains belonging to one of the following industries:
Retail Publisher Classifieds
URLs analyzed, with data taken from Botify Analytics
and server log files.
Unique Googlebot crawls analyzed, taken from 30
days of web server log files for each website.

10. Percentage of compliant pages (indexable pages) crawled by Google
in 30 days.
Crawl Ratio
Desktop Index vs. Mobile-First Index
We consider a website “in” the mobile-first index when the crawl
volume for the Google Smartphone user agent is higher than the
crawl volume for the Google Search user agent for over 30 days.
A brief note on terminology

11. We studied websites of all shapes, sizes, and industries
Dataset by IndustriesDataset by Website Sizes (in pages)

12. Sites with structural, performative, and geographic diversity

13. One quick note about AMP
AMP pages are not an alternative to a mobile
website.
If you have AMP and you don’t have a mobile
website, Google will take into account the Desktop
version to rank the AMP pages.
If you have AMP and a mobile website, Google will
take into account the mobile version to rank the
AMP pages.
In any case, you should make sure that an AMP
page is not much different from the main page.

14. Google continues to roll out the MFI progressively, but more
than 1/3 of websites surveyed have not transitioned
JUNE - JULY ‘18 SEPT - OCT ‘18
9.6% 34%
JUNE-JULY ‘19
63%

15. Publishers have rocketed into MFI, while marketplace/listings
websites have lagged
52%
9%
Potentially due to fresh content, publishers are crawled by Google more often than retailers and classifieds.
This leads to higher crawl ratios. Does that explain the high number of publishers in MFI?
MARKETPLACES RETAIL PUBLISHERS
62%
7%
73%
11%
Increases since October 2018

16. We observed huge spikes in crawling in Summer 2018 as
Google began moving sites to the MFI

17. While Google’s crawl has evolved to more mobile crawls,
mobile/desktop traffic share hasn’t shifted significantly
Google’s Crawl Evolution Traffic Share by Device

18. The largest study of the
mobile-first index based
on real customer data.

19. Website’s Type
Industry
Website’s Size
Mobile Website Type
Language
Log File KPIs
Crawl Volume
(Search & Smartphone)
Crawl Ratio
Desktop Visits
Mobile Visits
Avg. Visits
Structural KPIs
Load Times
PageRank Dilution
Outlinks
Content Size
Content vs. Template %
How does Google prioritize websites for MFI? We
tested for:

20. The Mobile-First Index loves responsive
websites
Responsive websites are much more likely
to enter the mobile-first index first.
Google appears to feel that it’s less risky to
transition responsive websites, since the
same page is served regardless of
device/user agent.
By contrast, Mobile URL and Dynamic
Serving websites serve different pages
depending on the user agent.
This can hurt content parallelism, if the
mobile page doesn’t match the desktop
page.
Mobile-First Index* Desktop Index
PCT OF WEBSITES IN MFI BY MOBILE SETUP

21. Smaller websites are more likely to enter the
MFI first
● Smaller websites have a greater chance of
entering the mobile-first index first.
● Google is giving the largest sites longer to
prepare, or is moving them over at a slower
pace to ensure successful transitions, or
both.
● From a risk assessment point of view, this
makes sense, because smaller websites
should be easier to move for both Google
and the the site stakeholders.
NUMBER OF URLs
Mobile-First Index* Desktop Index
PCT OF WEBSITES IN MFI BY WEBSITE SIZE

22. Websites with more content per page are more in the MFI
PCT OF WEBSITES IN MFI BY AVG CONTENT
SIZE
PCT OF WEBSITES PER INDUSTRY

23. Websites with cleaner signals appear to move
into MFI sooner
Websites that are diluting less PageRank on
non-compliant pages are more into the
Mobile-First Index
Websites with a lower PageRank Dilution
may be considered as higher quality
websites by Google.
These websites are also have fewer
conflicting signals to Google making them
easier to crawl.
Mobile-First Index* Desktop Index
PCT OF WEBSITES IN MFI PAGERANK
DISTRIBUTION

24. Very slow loading websites may be getting
transitioned to MFI slower
Websites that loaded in < 1500 ms , 1000
ms, and 500 ms have transitioned at similar
rates, while websites that took longer than
2000 ms on average have transitioned at a
slower rate.
Mobile-First Index* Desktop Index
PCT OF WEBSITES IN MFI BY AVERAGE LOAD
TIMES

25. Surprisingly, many SEO indicators DO NOT
impact transition to MFI
We’ve tested the following criteria with no significant results:
Content /
Template Ratio
Avg. Visits
Mobile Traffic Share
Mobile vs. Desktop
Content Visibility
Outlinks
Visits
Language

26. Actual data, real
results: customer
use cases
A Publisher that transitioned
When things don’t just go right, but actually get
better, during transition to mobile-first index.1
A Tale of Two Transitions: Uh-Oh
A Mobile URL website sees its crawl plummet
after the transition.2
Sites Who Haven’t Transitioned
Some real-world examples of websites that haven’t
transitioned and the challenges they have3

27. 1 A tale of two transitions: smooth
When things don’t just go right, but actually get better, during a
responsive site’s transition to the mobile-first index.

28. Temporary crawl
volume increases
when the website
enters the mobile-
first index
MFI transition generates huge surge
in crawl activity

29. Backbone of site’s steady performance:
content parallelism between desktop/mobile
3.0%
% URLs Crawled by Google
METRICS TO TRACK
SUCCESS
Visits Volume on URLs
Crawled by Botify
Active URLs from Google
6.2%
18.1%

30. 2 A tale of two transitions: uh-oh
A Mobile URL website sees its crawl plummet after the
transition.

31. Same movie: MFI transition generates
huge surge in crawls… EXCEPT...
Temporary crawl
volume increases
when the website
enters the mobile-
first index
URLS CRAWLED BY GOOGLEBOT BY USER-AGENT BY DAY

32. *From September 9th to October 9th
Long term crawl volume deteriorates.
Key KPIs tank across the board.
28.2%
% URLs Crawled by Google
METRICS TO TRACK
SUCCESS
Visits Volume on URLs
Crawled by Botify
Active URLs from Google
32
19.5%
21.1%
URLS CRAWLED BY GOOGLEBOT BY USER-AGENT BY DAY

33. 87% of the URLs have content similarity below 60% between mobile and desktop versions.
What sealed the doom? Low content similarity
rears its ugly head.

34. 3 Sites Who Haven’t Made the Transition
Some real-world examples of websites that haven’t transitioned
and the challenges they have

35. Despite this website being < 30,000 URLs, it still hasn’t made the switch
A small marketplace mobile website that still
hasn’t transitioned
URL DISTRIBUTION BY COMPLIANCE CRAWL DISTRIBUTION BY USER-AGENT

36. On average, this website’s mobile pages have < 50% average words per page.
Most of its mobile pages are not similar to their
desktop equivalents
URL DISTRIBUTION BY COMPLIANCE CRAWL DISTRIBUTION BY USER-AGENT

37. And many of its bidirectional signals are not
properly configured
BIDIRECTIONAL SIGNAL ISSUES
● The mobile alternate page is not
linked in the Mobile website's
structure
● Mobile page's canonical tag is not
matching the Desktop page
● The alternate tag is missing on
some Desktop pages

38. And finally, a large retail responsive website that
isn’t living in a mobile first (indexing) world
URL DISTRIBUTION BY COMPLIANCE CRAWL DISTRIBUTION BY USER-AGENT

39. More than 40% of this website’s internal page rank is sent to non-compliant pages
Two issues appear to be holding this website
from making the transition
PAGE RANK DILUTION LOW CONTENT SIZE

40. So, are you ready for the
Mobile-First Index?
7 tips for a successful
transition right now.

41. The big 7: key MFI tips that any site can implement
Ensure site performance by enforcing
content parallelism.
Foresee and confirm MFI transition
through log file analysis.
Optimize user experience by
improving mobile load times.
Verify mobile version is accessible to
Google Smartphone.
Embrace coherence by aligning
schema for desktop/mobile
versions.
Maintain seamless navigation by
properly updating your hreflang tags.
Keep all common mobile best
practices (rel=alternate /
rel=canonical)

42. Stop by our booth if you’d like
to learn more or if you’d just
like to have a chat about SEO.
THANK YOU!