This document discusses the impact of Google's upcoming mobile-first indexing approach. It begins by reviewing the history of mobile web development and configurations. It then discusses some of the potential issues sites may face with mobile-first indexing depending on their current mobile setup. This includes possible issues with site architecture, content differences, and tag configurations. The document also outlines the expected priority of indexing and display in search results based on mobile configuration. It recommends using DeepCrawl to audit mobile configurations and identify any issues in preparation for mobile-first indexing.
3. @jondmyers @DeepCrawl
What are we going to cover?
• The mobile world now (pre Mobile-first)
• The mobile world in the future (after Mobile-first)
• Prepare for Mobile-first with DeepCrawl
• The future of the future
6. @jondmyers @DeepCrawl
History of Mobile
WAP
HTML
1999 2007 2009 2014 2015 2017
Separate Mobile
Pages (Mobile Site
or Dynamic Delivery)
Responsive
Design
Deep App
Linking
AMP &
Progressive
Web Apps
Mobile-
first
Indexing
7. @jondmyers @DeepCrawl
History of Mobile SEO Configuration
Desktop
Mobile
Responsive
AMP pages
Mobile first
One version for desktop devices
One to rule them all - one version designed to work equally
good on desktop and mobile
Dedicated light weight version designed for a fast loading
Dedicated mobile pages served on a separate URL e.g.
m.domain or dynamically served on the same URL
Mobile becomes the PRIMARY version.
8. @jondmyers @DeepCrawl
Types of Mobile Configuration
1. No Mobile Configuration (desktop only)
2. Responsive
3. Dynamic
4. Desktop + Dedicated Mobile
5. Desktop + AMP
6. Responsive/Dynamic + AMP
7. Desktop + Dedicated Mobile + AMP
8. Responsive + Dedicated Mobile + AMP
9. Responsive + Dynamic + Dedicated Mobile + AMP
• Mobile/AMP only
• Mobile + AMP
12. @jondmyers @DeepCrawl
Impact of Mobile-first
-CRITICAL- - Desktop only (Heavily Affected)
-HIGH PRIORITY- - Desktop + Dedicated Mobile (Affected)
-HIGH PRIORITY- - Dynamic (Affected e.g. Content Issues)
-MEDIUM PRIORITY- - Responsive (Not affected)
Impact of Mobile-first on different Mobile Configurations:
13. @jondmyers @DeepCrawl
Potential Issues
• Site Architecture: Indexation, Crawlability,
Canonicalisation, PageRank, Linking Structure
• Site Content: Content Differences - Titles / H1s,
Description, Body Content. Missing Content - Responsive:
hidden content (no longer an issue after mobile-first),
Dynamic: missing content, Dedicated: missing content
• Tags and Markups: Schema, Hreflang config
• Order of Mobile-first Indexing
14. @jondmyers @DeepCrawl
Architecture/Tags: Canonicalisation
Canonicalised Desktop - www.domain.com/shirts/colour=red
Primary Desktop - www.domain.com/shirts/
Primary Mobile - mobi.domain.com/shirts/
Canonicalised Mobile - mobi.domain.com/shirts/colour=red
Desktop
Page
AMP
Page
Mobile
Page
Rel=canonical
Rel=alternate amp
NON-INDEXABLE
www.domain.com
/shirts/amp
www.domain.com
/shirts/
m.domain.com
/shirts/
NON-INDEXABLE
Rel=canonical
Rel=alternate
mobile
Desktop
Page
Mobile
Page
INDEXABLE INDEXABLENON-INDEXABLE
15. @jondmyers @DeepCrawl
Architecture/Tags: Canonicalisation
“When Google moves
to mobile first, the rel
alternate and canonical
tags won't need to be
changed.
rel=alternate mobile
will take precedence
over rel=canonical”
Reference:https://www.youtube.com/watch?v=mfJEpru0s
zw#t=49m34s
16. @jondmyers @DeepCrawl
Architecture/Tags: Canonicalisation
“When Google moves
to mobile first, the rel
alternate and canonical
tags won't need to be
changed.
rel=alternate mobile
will take precedence
over rel=canonical”
Reference:https://www.youtube.com/watch?v=mfJEpru0s
zw#t=49m34s
17. @jondmyers @DeepCrawl
1,410
Backlinks
303
Backlinks
303
102
Backlinks
102
1,815
Architecture: Authority distribution
Desktop
Page
AMP
Page
Mobile
Page
Rel=canonical
Rel=alternate amp
NON-INDEXABLE NON-INDEXABLE
Rel=canonical
Rel=alternate
mobile
Desktop
Page
Mobile
Page
INDEXABLE INDEXABLENON-INDEXABLE
* There is no need to change the canonical setup. The page
Authority should be passed via “mobile rel alternate”
20. @jondmyers @DeepCrawl
Architecture: Internal Site Architecture (Link Graph)
“Links on Mobile Pages Will
Be Used For the Link Graph
Mobile-first indexing will
use the links on your mobile
pages for calculating the link
graph”
Reference:https://www.youtube.com/watch?v=7Aq9bFdf
MuE#t=46m05s
21. @jondmyers @DeepCrawl
Tags: Hreflang configuration
Desktop
Page
EN Primary
Desktop
EN alternate hreflang
Rel=canonical
DE Primary
DesktopEN alternate hreflang
Rel=canonical Rel=alternate
mobile
Rel=alternate
mobile
Desktop
Page
EN Mobile
Page
EN alternate hreflang
DE Mobile
PageEN alternate hreflang
EN Primary
Desktop
DE Primary
Desktop
DE Mobile
Page
EN Mobile
Page
?
?
22. @jondmyers @DeepCrawl
Tags: Hreflang configuration
“Yes, you would have to tag
mobile page.”
Wait a second. No, you
actually don’t.
This is still to be confirmed
closer to the launch of
mobile-first as Google is still
experimenting with this.”
Reference:https://www.youtube.com/watch?v=mfJEpru0sz
w#t=49m50s
23. @jondmyers @DeepCrawl
Dynamic
Mobile Page
Order of Mobile-first Indexing
Dynamically served
Desktop
Page
AMP
Page
Mobile
Page
Rel=canonical
Rel=alternate amp
Rel=canonical
Rel=alternate
mobile
Desktop
Page
Mobile
Page
* Which of the different types would
be used for mobile-index?
24. @jondmyers @DeepCrawl
Order of Mobile-first Indexing
“If you have a vary header
which returns a different
page for a mobile user
agent, Google will use that
as the mobile page instead
of the responsive page.”
Reference:https://www.youtube.com/watch?v=8NyZypI
fOzI&t=36m31s
26. @jondmyers @DeepCrawl
Order of Mobile-first Indexing
Desktop
Page
Mobile
Page
Rel=canonical
Rel=alternate
mobile
Responsive
Page
Mobile
Page
INDEXABLENON-INDEXABLE
GOOGLE SERP
28. @jondmyers @DeepCrawl
Dynamic
Mobile Page
Order of Mobile-first Indexing
Dynamically served
Desktop
Page
AMP
Page
Rel=canonical
Rel=alternate amp
Responsive
Page
INDEXABLE
NON-INDEXABLE NON-INDEXABLE
GOOGLE SERP
29. @jondmyers @DeepCrawl
Order of Mobile-first Indexing
Desktop
Page
AMP
Page
Mobile
Page
Rel=canonical
Rel=alternate amp
NON-INDEXABLE NON-INDEXABLE
Rel=canonical
Rel=alternate
mobile
Responsive
Page
Mobile
Page
INDEXABLE INDEXABLENON-INDEXABLE
GOOGLE SERP
30. @jondmyers @DeepCrawl
Order of Mobile-first Indexing
Desktop
Page
AMP
Page
Rel=canonical
Rel=alternate amp
Responsive
Page
NON-INDEXABLE INDEXABLE
GOOGLE SERP
31. @jondmyers @DeepCrawl
Order of Mobile-first Indexing
Desktop
Page
AMP
Page
Rel=canonical
Rel=alternate amp
NON-INDEXABLE INDEXABLE
GOOGLE SERP
Non-mobile
Friendly
32. @jondmyers @DeepCrawl
Order of Mobile-first Indexing
Desktop
Page
AMP
Page
Rel=canonical
Rel=alternate
mobile
Non-mobile
Friendly
NON-INDEXABLEINDEXABLE
GOOGLE SERP
33. @jondmyers @DeepCrawl
Priority of indexing/displaying in mobile-first
Priority of indexing after mobile-
first release:
1. Dedicated Mobile /
Dynamic
2. Responsive
3. Desktop
4. AMP
Priority of displaying in Google
SERPs:
1. AMP (priority)
2. Dedicated Mobile /
Dynamic
3. Responsive
4. Desktop
34. @jondmyers @DeepCrawl
So what’s the optimal setup?
“So what’s the optimal setup?
- “Responsive Design is
Optimal for Mobile-first
John suggests that a
responsive design is the
optimal mobile setup for
mobile-first indexing, as it
avoids the issues of different
content and links on your
mobile pages.”
Reference:ttps://www.youtube.com/watch?v=7Aq9bFdfMu
E#t=54m53s
36. @jondmyers @DeepCrawl
Responsive & Desktop only
RESPONSIVE DESKTOP ONLY
You are likely to be f..ine You are likely to be f..ed
Source: The Big Lebowski
37. @jondmyers @DeepCrawl
If your website uses Dynamic conf
If your website is dynamic I would recommend two crawls:
• Web Crawl
This is to compare your Desktop’s to Mobile’s site architecture, linking structure and
crawlability.
• Analytics / List Crawl (crawling priority high driving traffic pages)
This is to test if you have the mobile equivalents for all your important pages so the
ones that drive significant amount of traffic OR/AND have authority backlinks.
• Both craws will highlight:
• If mobile configuration is setup correctly;
• You don’t have any significant Content differences;
• All the pages are accessible on Mobile as well etc.
38. @jondmyers @DeepCrawl
Dynamic - web crawl setup
#1 Web Crawl
I would recommend running two crawls:
• First with user-agent: Googlebot (desktop)
• Repeat the crawl with user-agent: Googlebot smartphone
#2 Analytics Crawl (crawling pages driving traffic)
Use the Analytics source to upload the priority pages so the ones
that drive most visits to your website e.g. from Google Analytics
I would recommend running two crawls for this source as well:
• First with user-agent: Googlebot (desktop)
• Repeat the crawl with user-agent: Googlebot smartphone
39. @jondmyers @DeepCrawl
Example Reports/Insights:
Configuration & setup:
• Are the tags setup correctly (e.g. Vary: User-agent)
Differences in Content (between desktop and mobile)
• Pages with different Titles (BETA)
• Pages with different Descriptions (BETA)
• Pages with different Content (BETA)
Differences in Architecture (between desktop and mobile)
• Missing Pages / Additional Page
(All Page report for this)
• Pages with different Internal Links out (BETA)
Dynamic - Reports
Differences between Desktop and Mobile
Mobile configuration reports
40. @jondmyers @DeepCrawl
If you have Separate mobile version
If you website have a Separate mobile version I would recommend two crawls:
• 2 types of Web Crawl
This is to compare your Desktop version to Mobile one to see if your
configuration is setup correctly, you don’t have any significant Content
differences, all the pages are accessible on Mobile as well etc.
• Analytics / List Crawl (crawling priority high driving traffic pages)
This is to test if you have the mobile equivalents for all your important
pages so the ones that drive significant traffic OR/AND have authority
backlinks
41. @jondmyers @DeepCrawl
Separate - web crawl setup
#1 Web Crawl
I would recommend running one crawl with:
• User-agent set to Googlebot (desktop)
• Check mobile/AMPHTML alternate tags Link Restrictions
#2 Web Crawl
I would recommend running two crawls:
• First with user-agent: Googlebot (desktop)
• Repeat the crawl with user-agent: Googlebot smartphone and the Test site domain set to: m.domain.com
42. @jondmyers @DeepCrawl
Dynamic - web crawl setup
#3 Analytics Crawl (crawling pages driving traffic)
Use the Analytics source to upload the priority pages so the ones
that drive most visits to your website e.g. from Google Analytics
We would recommend running two crawls then:
• First with user-agent: Googlebot (desktop)
• Repeat the crawl with user-agent: Googlebot smartphone and the
Test site domain set to: m.domain.com
43. @jondmyers @DeepCrawl
Example Reports/Insights:
Configuration & setup:
• Rel alternate tags
• Reciprocation
Differences in Content (between desktop and mobile)
• Pages with different Titles / Descriptions (BETA)
• Pages with different Content - Title+h1 (BETA)
• Word count
• Links in / Links out
Differences in Architecture (between desktop and mobile)
• Broken Mobile Pages
• Missing / Additional Pages
Separate - Reports
Mobile configuration Issues
45. @jondmyers @DeepCrawl
DeepCrawl Initial Research
Initial mobile research, based on the top 100k domains in the majestic million.
Highlights:
• 40% of sites are responsive
• 8.5% are dynamically served
• 1.5% of sites have a separate mobile site and link to it within a rel=alternate tag,
however 5% redirect to another page/site when recrawled with a mobile user agent
• 91% of websites which give a redirect based on the user agent do not have a “Vary:
User-agent header”
47. @jondmyers @DeepCrawl
AMP Pages
Google is backing the AMP page quite
hard
Google preloads AMP pages, images,
and scripts to force them to be fast
Not just for news: AMP added support
for accepting payments via Android pay,
opening the way for ecommerce
websites which are 100% AMP
https://github.com/ampproject/amphtml/issues/7623
48. @jondmyers @DeepCrawl
Go beyond Responsive with PWS
Progressive Web Apps is:
• Progressive
• Responsive
• App-like
• And more...
Progressive Web Apps can fall back to
AMP. Hint… first load as AMP.
You should think about it now.
Progressively Web App is THE FUTURE.
49. @jondmyers @DeepCrawl
New Features (coming soon)
Setup & additional sources:
• Allow to specify mobile user-agent
• Search Console integration
New Reports:
• Mobile traffic vs Desktop traffic
• And many more….
DeepCrawl New Features (coming soon)
50. @jondmyers @DeepCrawl
Other things to mention
It's not likely to happen until next year.
http://www.thesempost.com/google-mobile-first-index-unlikely-launch-2018/
It’s probable they’ll move over batches of sites that work fine on mobile-first indexing first.
https://www.youtube.com/watch?v=5TsGXF4wNqM#t=24m31s
Google will send alerts, via Search Console, for mobile-first issues such as mobile pages having less content and markup than
desktop pages.
https://www.youtube.com/watch?v=nfPHUzGCzmA#t=51m52s
Google is trying to minimise any changes required for mobile-first, and make it easy for people to understand the changes they
need to make. It's possible they might overcome differences in schema and hreflang.
https://www.youtube.com/watch?v=U0Lr_d2aFZs#t=36m24s
Mobile page speed will be incorporated into mobile-first rankings 'at some point'.
https://www.youtube.com/watch?v=U0Lr_d2aFZs#t=7m15s
You should add AMP links to your mobile pages.
https://www.youtube.com/watch?v=t-Ffq6xl6p8&t=45m03s
With mobile-first index the hidden content will probably stop being demoted.
https://www.youtube.com/watch?v=mfJEpru0szw#t=16m48s