1. Improve Crawling and
Indexing with
User-Friendly
Log File Insights
Slideshare.Net/Stevenvvessum
@STEVENVVESSUM
Steven van Vessum
Conductor

2. Steven van Vessum
Director of Organic Marketing at Conductor
Previously:
Co-founder & VP of Community at ContentKing
Title Slide Option

3. 3
Real-Time SEO Auditing & Monitoring
About
#brightonSEO

4. 4
Real-Time SEO Auditing & Monitoring
About
#brightonSEO

5. 5
Technical SEO is the
Foundation of Successful
Organic Marketing
Technical SEO
Technical
Ensure discoverability
Content
Address customer needs
Insights
Optimize performance
Action
Move faster
About
#brightonSEO

6. Let’s be honest…

7. We’re all having issues
getting content indexed.

8. Possible
reasons…
● The web is growing
faster than ever.
● The rise of AI
generated content.

9. Photo by the
awesome Lily
Ray from
Amsive Digital

10. Possible
reasons…
Crawling, rendering,
indexing, storing and
serving content is
expensive.

11. Possible
reasons…
● Google’s has been
raising the “content
quality bar” over the
last few years.
● Helpful Content
Update in August
2022.

12. Tough shit? 💩

13. The Issues We See
at Companies

14. The issue we
see at
companies
They just ship an X
amount of content
pieces a month.

15. The issue we
see at
companies
They don’t reflect on
the work they’ve done.

16. The issue we
see at
companies
They don’t understand
and can’t explain
obvious search engine
behavior.

17. How Companies
Get Better

18. Enablement Companies need
easy access to
log file insights.

19. Thinking
critically and
getting
answers
● Has Google crawled
these pages yet?
● Has Google tried
crawling pages
while they had
issues?

20. ● How often do my
pages get recrawled
on average?
● Has Google
reindexed the pages
we updated?
Thinking
critically and
getting
answers

21. With these
answers, they
can….
● Improve time to
crawl
● Improve time to
index
● Improve time to
rank

22. ● Improve time to
traffic
● SEO forecast
With these
answers, they
can….

23. By providing easy access
to log file insights….

24. You’ll make SEO
accessible.

25. You’ll make SEO fun.

26. You’ll get buy-in from
management.

27. Log files

28. What are
log files?
● Text files containing
records of all the
requests a server
has received from
both humans and
crawlers.
● Its responses to
these requests.

29. Why Are Logs
Important to SEO?

30. Logs show crawlers’
true behavior

31. Log File Analysis As
We Know It

33. Log file analysis
as we know it Inefficient and time
consuming.

34. Often only performed
once a year.
Log file analysis
as we know it

35. Insights are kept
within silos.
Log file analysis
as we know it

36. I get it though.

37. But we NEED
to do better
(and we can)

38. Our Dream

39. ● Log data is
available in
real-time.
● Insights are
automatically
distilled.
It would be
amazing if….

40. It would be
amazing if….
● Made easily
accessible for
web and
content teams.

41. Getting Your Hands
on Logs

42. Is….tricky
● Access logs aren’t
there (anymore).
● PII concerns.

43. Is….tricky
● Office politics.
● Slow processes.

44. Enter CDN logs
● Real-Time
● Plug ‘n Play

45. Connecting to
Content Inventory

46. Connect log
insights to
content
inventory
● Coupled to
content
● Results in
enlightenment

49. Improving Crawling
& Indexing

50. New Pages How long does it take
for search engines to
crawl new pages?

51. New Pages Can you speed up
time to crawl
with better content
promotion?

52. New Pages Can you speed up time
to crawl with better
internal linking?

53. New Pages Is there a correlation
between
crawl frequency and
internal links?

54. New Pages Is there a correlation
between
crawl frequency and
internal links?

55. Move away from
manually parsing raw
logs to automatically
extracted insights.

56. Updated Pages Has Google already
picked up my
improvements?

57. Updated Pages How soon does Google
refresh pages after
making updates?

58. Updated Pages Is there increased crawl
activity after making
changes?

59. A timeline with
changes and whether
they were seen by
search engines.

60. Robots.txt Has Google recrawled
the new robots.txt
directives?

61. Robots.txt Has Google tried
requesting the
robots.txt while it had a
Disallow: /

62. Learn what search
engines know.

63. XML Sitemap Has Google recrawled
my XML sitemap after it
was updated?

64. XML Sitemap How frequently does
Google crawl my oldest
XML sitemaps?

65. Learn what content
search engines know
about.

66. Recap

67. Logs are important
to SEO

68. Lots of companies
underperform without insights
from logs

69. Distilling actionable insights
from logs is time-consuming
and hard

70. Making insights easily
accessible across teams
is hard

71. Getting your hands
on logs is hard

72. Connecting
insights to content
inventory is hard

73. Enter CDN logs.
(With ContentKing)

74. Learn how
your work moves
the needle

75. And keep sharpening
your blade.

76. Shout!
Questions, comments, ideas…

77. FREE 45-day Trial
Experience Real-Time Log File Analysis yourself.
Improve your SEO game in under 20 seconds.
contentkingapp.com/brightonseo