Interesting insights from log files are often kept within silos, not shared with content teams. This prevent content teams from reaching their full potential. Learn how they can improve crawling and indexing though leveraging insights from log file analysis, all in real-time.

1. Improve Crawling and
Indexing with
User-Friendly Log File
Insights
Steven van Vessum | Conductor
SLIDESHARE.NET/Stevenvvessum
@STEVENVVESSUM

2. Steven van Vessum
Director of Organic Marketing at Conductor
Previously:
Co-founder & VP of Community at ContentKing
Title Slide Option

3. About

4. 4
Real-Time SEO Auditing & Monitoring
About
#brightonSEO

5. 5
Real-Time SEO Auditing & Monitoring
About
#brightonSEO

6. 6
Technical SEO is the
Foundation of Successful
Organic Marketing
Technical SEO
Technical
Ensure discoverability
Content
Address customer needs
Insights
Optimize performance
Action
Move faster
About
#brightonSEO

7. Log Files

8. What are your backgrounds?
Log Files
8
Technical SEO Content
Developer Other
#brightonSEO

9. What are log files?
Log Files
9
Text files containing records of
● All the requests a server has
received from both humans and
crawlers
● Its responses to these requests.

10. Why Are Logs Important to SEO?

11. 11
Logs show crawlers’
true behavior

12. The Issues We See at Companies

13. They just ship X content pieces a month.
The issue we see at companies
The Issues We See at Companies
13
#brightonSEO

14. The issue we see at companies
The Issues We See at Companies
14
#brightonSEO
They don’t reflect on
the work they’ve done.

15. The issue we see at companies
The Issues We See at Companies
15
#brightonSEO
They don’t understand and can’t explain
obvious search engine behavior.

16. How Companies Can Get Better

17. Easy access to log file insights
How Companies Can Get Better
17
#brightonSEO
Companies need easy access
to log file insights.

18. Thinking critically and getting answers
How Companies Can Get Better
18
Has Google tried crawling pages
while they had issues?
How often do my pages get
recrawled on average?
Has Google crawled these new
pages yet?
Has Google refreshed the pages
we updated?
#brightonSEO

19. Then they can improve
19
Average time to crawl Average time to index
#brightonSEO
How Companies Can Get Better
Average time to rank

20. Then they can improve
20
SEO forecast
#brightonSEO
How Companies Can Get Better
Average time to traffic

21. Easy access to log file insights
How Companies Can Get Better
21
#brightonSEO
By providing easy access
to log file insights….

22. 22
You’ll make SEO
accessible.

23. 23
You’ll make SEO fun.

24. 24
You’ll get buy-in
from management.

25. Log File Analysis As We Know It

26. Log file analysis as we know it….
Log File Analysis As We Know It
26
#brightonSEO

27. Log file analysis as we know it….
Log File Analysis As We Know It
27
Inefficient and time
consuming
Often only performed
once a year
Insights are kept
within silos
#brightonSEO

28. 28
We NEED
to do better
(and we can)

29. Our Dream

30. It would be amazing if….
Our Dream
30
Log data is available
in real-time
Useful insights are
automatically distilled
Made easily accessible for
web and content teams
#brightonSEO

31. Getting Your Hands on Logs

32. Is… tricky
Getting Your Hands on Logs
32
Access logs aren’t
there (anymore)
Office politics
PII concerns Slow processes
#brightonSEO

33. Enter CDN Logs
Getting Your Hands on Logs
33
Real-time Plug’n Play
#brightonSEO

34. Connecting to Content Inventory

35. Connect log insights to content inventory
Connect to Content Inventory
35
#brightonSEO
Coupled to content Enlightenment

36. Old vs New
Connect to Content Inventory
36
#brightonSEO

37. Old vs New
Connect to Content Inventory
37
#brightonSEO

38. Improving Crawling & Indexing

39. New Pages
Improving Crawling & Indexing
39
#brightonSEO
How long does it take for search engines
to crawl new pages?

40. New Pages
Improving Crawling & Indexing
40
#brightonSEO
Can you speed up time to crawl
with better internal linking?

41. New Pages
Improving Crawling & Indexing
41
#brightonSEO
Can you speed up time to crawl
with better content promotion?

42. New Pages
Improving Crawling & Indexing
42
#brightonSEO
Is there a correlation between
crawl frequency and
internal links?

43. New Pages
Improving Crawling & Indexing
43
Move away from
manually parsing raw
logs to automatically
extracted insights.

44. Updated Pages
Improving Crawling & Indexing
44
#brightonSEO
Has Google already picked up my
improvements?

45. Updated Pages
Improving Crawling & Indexing
45
#brightonSEO
How soon does Google refresh pages
after making updates?

46. Updated Pages
Improving Crawling & Indexing
46
#brightonSEO
Is there increased crawl activity after
making changes?

47. Updated Pages
Improving Crawling & Indexing
47
A timeline with changes
and whether they were
seen by search engines.

48. Robots.txt
Improving Crawling & Indexing
48
#brightonSEO
Has Google recrawled the
new robots.txt directives?

49. Robots.txt
Improving Crawling & Indexing
49
#brightonSEO
Has Google tried requesting the
robots.txt while it had a Disallow: /

50. Robots.txt
Improving crawling & indexing
50
Learn what search
engines know.

51. XML Sitemaps
Improving Crawling & Indexing
51
#brightonSEO
Has Google recrawled my XML sitemap
after it was updated?

52. XML Sitemaps
Improving Crawling & Indexing
52
#brightonSEO
How frequently does Google crawl my
oldest XML sitemaps?

53. XML Sitemaps
Improving crawling & indexing
53
Learn what content
search engines know
about.

54. Recap

55. Logs are important
to SEO

56. Lots of companies
underperform without insights
from logs

57. Distilling actionable insights
from logs is time-consuming
and hard

58. Making insights easily
accessible across teams
is hard

59. Getting your hands
on logs is hard

60. Connecting
insights to content
inventory is hard

61. Enter CDN logs.
(With ContentKing)

62. Learn how
your work moves
the needle

63. And keep sharpening
your blade.

64. Shout!
Questions, comments, ideas…

65. FREE 45-day Trial
Experience Real-Time Log File Analysis yourself.
Improve your SEO game in under 20 seconds.
contentkingapp.com/brightonseo