BrightonSEO: SearchBots: Lost Children or Hungry Psychopaths? What Do Searchbots Actually Do?

•

4 gostaram•2,786 visualizações

We think we know how searchbots interact with our websites, but do we really? What data is there that tell us where they visited, what they requested, how long they spent? This talk examines various data sources, visualises searchbots visits (animating them over time), explores how we can help searchbots - and even attempts to turn their visits into sound.

Tecnologia Design

Searchbots:
Lost Children ... or ... Hungry Psychopaths?
What Do Searchbots Actually Do? (and why it matters)

© 2007-2012 Roland Dunn

“Googlebot ... may be unable to completely index
all the content on your site”

© 2007-2012 Roland Dunn

Doesn’t tell us what searchbots index

© 2007-2012 Roland Dunn

Webserver Logfiles ....

© 2007-2012 Roland Dunn

Over 2 months, 40% of all Googlebot’s requests
were to …. just 2 URL’s (on-site search URLs)

© 2007-2012 Roland Dunn

Over 2 months, 40% of all Googlebot’s requests
were to …. just 2 URL’s (on-site search URLs)

On a website with approximately 40,000 URLs in
Google’s index (allegedly, according to site:…)

© 2007-2012 Roland Dunn

Over 2 months, 40% of all Googlebot’s requests
were to …. 2 URL’s.

On a website with approximately 40,000 URLs in
Google’s index.

On a website serving approx. 150-200K unique
natural search visits/month

© 2007-2012 Roland Dunn

Categorise URLs by “top level section” – perhaps more
useful than just URLs ...

© 2007-2012 Roland Dunn

Table of “top level sections” requested by Googlebot

© 2007-2012 Roland Dunn

Chart of “top level sections” requested by Gbot – 6 months

© 2007-2012 Roland Dunn

Does Googlebot request all URLs served by a website?

© 2007-2012 Roland Dunn

Does Googlebot request all URLs served by a website?

No!

© 2007-2012 Roland Dunn

How Does Googlebot Spend Its Time? What Does it Request?

© 2007-2012 Roland Dunn

Does Googlebot spend its time cost-effectively?

© 2007-2012 Roland Dunn

Does Googlebot spend its time cost-effectively?

Not always.

© 2007-2012 Roland Dunn

Googlebot does not always request all content

© 2007-2012 Roland Dunn

Googlebot can become distracted, obsessed, or even lost:

© 2007-2012 Roland Dunn

Googlebot may not spend its time efficiently
It needs help focusing on what we value

© 2007-2012 Roland Dunn

SMX West, March 2011, Matt Cutts:
(http://goo.gl/ZZz7E):
“... if Google determines a site isn’t as useful to
users, they may not crawl it as frequently"

© 2007-2012 Roland Dunn

Conclusions:
• Searchbots do not always request all content
• Searchbots can become distracted
• Searchbots may use their time inefficiently
• Their visits are precious – we need to treasure them
• We may need to help them focus

© 2007-2012 Roland Dunn

Suggestions:
• Embrace logfiles – full of useful information

© 2007-2012 Roland Dunn

Suggestions:
• Embrace logfiles – full of useful information
• Check searchbot behaviour

© 2007-2012 Roland Dunn

Suggestions:
• Embrace logfiles – full of useful information
• Check searchbot behaviour
• If distracted, lost, inefficient, poor experience:
• Alter internal navigation & linking (e.g. flatten hierarchy)
• Robots.txt out (blunt approach)
• Alter URL construction (expensive!)

© 2007-2012 Roland Dunn

Lost Children ... or ... Hungry Psychopaths?

© 2007-2012 Roland Dunn

Or .... Distracted Teenager?

© 2007-2012 Roland Dunn

Sonification: What do Search Engines Requests Sound Like?

© 2007-2012 Roland Dunn

Online Branding: Natural Search:
http://www.refinedpractice.com/ http://www.cloudshapes.co.uk/
T: @RefinedPractice T: @roland_dunn

Slides Available At:
http://www.cloudshapes.co.uk/talks/

© 2007-2012 Roland Dunn

Mais conteúdo relacionado

Semelhante a BrightonSEO: SearchBots: Lost Children or Hungry Psychopaths? What Do Searchbots Actually Do?

LDD Summer Seminar 2013 - Pandas, Penguins and Search Engines - the update

Adido

Pandas penguins ldd-v1

Andy Headington

[Webinar] Building Better B2B Landing Pages + Live site reviews

SiteTuners Conversion Rate Optimization

Improving Search Engine Optimization (SEO) with Google Analytics

Jackson Lo

@DavidMihm @SLCSEM Local Search Ranking Factors 2012

Utah Digital Marketing Collective

The world of search engine optimization is changing faster than ever, thanks to artificial intelligence, and marketers are clamoring for new updates to websites to meet these changes. What changes are most important to the website and how do you manage these? What should you be proactive about, and how do you stay on track with an ever-evolving discipline? Learning Objectives & Outcomes: In this session, attendees will learn the key points behind the biggest changes in search, and the common requests they'll hear from their marketing teams. Attendees will leave equipped to speak to these requests and prioritize according to their capacity and impact on site performance.

TCDrupal 2018: SEO! Snippets! Schema!

Diane Kulseth

1/3 SEO: Keyword Research - Inede (Máster en Marketing Digital)

rodcul

Google panda

Shiva Kulshreshtha

SEO Training Workshop Presentation

Tim Aldiss

Local Search Marketing by WSI and Grow Plumbing Gregg Towsley

Towsley Associates

Local search

WSI Dave Scott

Rexton - Digital Marketing Presentation - 1.27.2014

Stephen Lella

DMI Webinar Series - SEO Audits (Part 1 of 3)

Digital Marketing Institute

500co seo talk

500 Startups

Be found EI Presentation March 2014

John Caldwell

International Search Placement | John Caldwell - CreatorSEO

Enterprise Ireland

Social Media and SEO Tips to make your Business succeed

Navneet Kaushal

Keynote - Conversion Rate Optimization - The Four Pillars of Building Instant...

E-Commerce Brasil

John Straw on SEO principles

John Straw

PubCon Las Vegas 2013

SiteTuners Conversion Rate Optimization

Semelhante a BrightonSEO: SearchBots: Lost Children or Hungry Psychopaths? What Do Searchbots Actually Do? (20)

LDD Summer Seminar 2013 - Pandas, Penguins and Search Engines - the update

Pandas penguins ldd-v1

[Webinar] Building Better B2B Landing Pages + Live site reviews

Improving Search Engine Optimization (SEO) with Google Analytics

@DavidMihm @SLCSEM Local Search Ranking Factors 2012

TCDrupal 2018: SEO! Snippets! Schema!

1/3 SEO: Keyword Research - Inede (Máster en Marketing Digital)

Google panda

SEO Training Workshop Presentation

Local Search Marketing by WSI and Grow Plumbing Gregg Towsley

Local search

Rexton - Digital Marketing Presentation - 1.27.2014

DMI Webinar Series - SEO Audits (Part 1 of 3)

500co seo talk

Be found EI Presentation March 2014

International Search Placement | John Caldwell - CreatorSEO

Social Media and SEO Tips to make your Business succeed

Keynote - Conversion Rate Optimization - The Four Pillars of Building Instant...

John Straw on SEO principles

PubCon Las Vegas 2013

Último

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Enterprise Knowledge’s Urmi Majumder, Principal Data Architecture Consultant, and Fernando Aguilar Islas, Senior Data Science Consultant, presented "Driving Behavioral Change for Information Management through Data-Driven Green Strategy" on March 27, 2024 at Enterprise Data World (EDW) in Orlando, Florida. In this presentation, Urmi and Fernando discussed a case study describing how the information management division in a large supply chain organization drove user behavior change through awareness of the carbon footprint of their duplicated and near-duplicated content, identified via advanced data analytics. Check out their presentation to gain valuable perspectives on utilizing data-driven strategies to influence positive behavioral shifts and support sustainability initiatives within your organization. In this session, participants gained answers to the following questions: - What is a Green Information Management (IM) Strategy, and why should you have one? - How can Artificial Intelligence (AI) and Machine Learning (ML) support your Green IM Strategy through content deduplication? - How can an organization use insights into their data to influence employee behavior for IM? - How can you reap additional benefits from content reduction that go beyond Green IM?

Driving Behavioral Change for Information Management through Data-Driven Gree...

Enterprise Knowledge

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Miguel Araújo

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Finology Group – Insurtech Innovation Award 2024

The Digital Insurer

Real Time Object Detection Using Open CV

Khem

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

GenAI Risks & Security Meetup 01052024.pdf

lior mazor

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

Tech Trends Report 2024 Future Today Institute.pdf

hans926745

The presentation explores the development and application of artificial intelligence (AI) from its inception to its current status in the modern world. The term "artificial intelligence" was first coined by John McCarthy in 1956 to describe efforts to develop computer programs capable of performing tasks that typically require human intelligence. This concept was first introduced at a conference held at Dartmouth College, where programs demonstrated capabilities such as playing chess, proving theorems, and interpreting texts. In the early stages, Alan Turing contributed to the field by defining intelligence as the ability of a being to respond to certain questions intelligently, proposing what is now known as the Turing Test to evaluate the presence of intelligent behavior in machines. As the decades progressed, AI evolved significantly. The 1980s focused on machine learning, teaching computers to learn from data, leading to the development of models that could improve their performance based on their experiences. The 1990s and 2000s saw further advances in algorithms and computational power, which allowed for more sophisticated data analysis techniques, including data mining. By the 2010s, the proliferation of big data and the refinement of deep learning techniques enabled AI to become mainstream. Notable milestones included the success of Google's AlphaGo and advancements in autonomous vehicles by companies like Tesla and Waymo. A major theme of the presentation is the application of generative AI, which has been used for tasks such as natural language text generation, translation, and question answering. Generative AI uses large datasets to train models that can then produce new, coherent pieces of text or other media. The presentation also discusses the ethical implications and the need for regulation in AI, highlighting issues such as privacy, bias, and the potential for misuse. These concerns have prompted calls for comprehensive regulations to ensure the safe and equitable use of AI technologies. Artificial intelligence has also played a significant role in healthcare, particularly highlighted during the COVID-19 pandemic, where it was used in drug discovery, vaccine development, and analyzing the spread of the virus. The capabilities of AI in healthcare are vast, ranging from medical diagnostics to personalized medicine, demonstrating the technology's potential to revolutionize fields beyond just technical or consumer applications. In conclusion, AI continues to be a rapidly evolving field with significant implications for various aspects of society. The development from theoretical concepts to real-world applications illustrates both the potential benefits and the challenges that come with integrating advanced technologies into everyday life. The ongoing discussion about AI ethics and regulation underscores the importance of managing these technologies responsibly to maximize their their benefits while minimizing potential harms.

Artificial Intelligence: Facts and Myths

Joaquim Jorge

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Handwritten Text Recognition for manuscripts and early printed texts

Maria Levchenko

Imagine a world where information flows as swiftly as thought itself, making decision-making as fluid as the data driving it. Every moment is critical, and the right tools can significantly boost your organization’s performance. The power of real-time data automation through FME can turn this vision into reality. Aimed at professionals eager to leverage real-time data for enhanced decision-making and efficiency, this webinar will cover the essentials of real-time data and its significance. We’ll explore: FME’s role in real-time event processing, from data intake and analysis to transformation and reporting An overview of leveraging streams vs. automations FME’s impact across various industries highlighted by real-life case studies Live demonstrations on setting up FME workflows for real-time data Practical advice on getting started, best practices, and tips for effective implementation Join us to enhance your skills in real-time data automation with FME, and take your operational capabilities to the next level.

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Safe Software

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Neo4j

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

BrightonSEO: SearchBots: Lost Children or Hungry Psychopaths? What Do Searchbots Actually Do?

18. Over 2 months, 40% of all Googlebot’s requests were to …. just 2 URL’s (on-site search URLs) On a website with approximately 40,000 URLs in Google’s index (allegedly, according to site:…) © 2007-2012 Roland Dunn

19. Over 2 months, 40% of all Googlebot’s requests were to …. just 2 URL’s (on-site search URLs) On a website with approximately 40,000 URLs in Google’s index (allegedly, according to site:…) On a website serving approx. 150-200K unique natural search visits/month © 2007-2012 Roland Dunn

20. Over 2 months, 40% of all Googlebot’s requests were to …. 2 URL’s. On a website with approximately 40,000 URLs in Google’s index. On a website serving approx. 150-200K unique natural search visits/month © 2007-2012 Roland Dunn

33. Googlebot can become distracted, obsessed, or even lost: • On-site search • Additive filters/faceted URLs e.g. “shoes?size=3&colour=green&price=10&brand=smith” • Sections with thin and/or very similar content © 2007-2012 Roland Dunn

38. Conclusions: • Searchbots do not always request all content • Searchbots can become distracted • Searchbots may use their time inefficiently • Their visits are precious – we need to treasure them • We may need to help them focus © 2007-2012 Roland Dunn

41. Suggestions: • Embrace logfiles – full of useful information • Check searchbot behaviour • If distracted, lost, inefficient, poor experience: • Alter internal navigation & linking (e.g. flatten hierarchy) • Robots.txt out (blunt approach) • Alter URL construction (expensive!) © 2007-2012 Roland Dunn

49. Online Branding: Natural Search: http://www.refinedpractice.com/ http://www.cloudshapes.co.uk/ T: @RefinedPractice T: @roland_dunn Slides Available At: http://www.cloudshapes.co.uk/talks/ © 2007-2012 Roland Dunn

BrightonSEO: SearchBots: Lost Children or Hungry Psychopaths? What Do Searchbots Actually Do?

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a BrightonSEO: SearchBots: Lost Children or Hungry Psychopaths? What Do Searchbots Actually Do?

Semelhante a BrightonSEO: SearchBots: Lost Children or Hungry Psychopaths? What Do Searchbots Actually Do? (20)

Último

Último (20)

BrightonSEO: SearchBots: Lost Children or Hungry Psychopaths? What Do Searchbots Actually Do?