O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Evaluating a search
engine
Phil Bradley, Websearch Academy, Internet Librarian International
2017
Who, what, where, when?
• Whois? https://whois.icann.org/en
Wayback machine
“About us”
What sort of search engine is it? Free text?
Directory?
Multi/meta search?
Social media?
Help screens or FAQs?
Advanced search functionality?
Sometimes you need to dig deeper!
And then deeper still!
Use a search engine to search
• “Crib sheet”
• “Cheat sheet”
• “Advanced search”
• “Search functionality”
• “Advanced feat...
Search functionality
• Term A Term B
• “Term A Term B”
• Term A Term B OR Term C
• (Term A Term B) OR Term C
• Term A AND ...
… continued
• Site, country or region based search
• Re-arrange results by date or relevance
• Can you search for symbols ...
Search result settings
• Number of results displayed on the screen
• Adult filter
• Country or region
• Auto-load results ...
Privacy
• Are searches private to you, or are they shared with the search
engine, stored and used in the future?
• What’s ...
Coverage
Set a benchmark
• Choose your preferred search engine and run a search in an area
that you’re familiar with, and note the ...
Search results
• Where does it pull its results from?
• How does it get its material?
Number of results – does it matter?
Fox red labrador
Fox red labrador
• 18 different sites listed
• 2 of them found by all three engines
• 1,2,2 and 2,3,1
• 5 found by two of ...
How are results relevance ranked? (1)
• On page and off page
• The position of words
• Next/near to each other
• On the we...
How are results relevance ranked? (2)
• Off page
• How many links point back to the page in question?
• What is the author...
How and when are algorithmic changes
evaluated?
• Google employs hundreds of quality evaluation raters
• Individual sites ...
Difficulties in evaluating search results
• Trying to work out what the users ‘intent’ was for the query
• Eg. ‘Olympics’
Authority and validity of the results
• “Martin Luther King” ‘A true historical examination’
• 9th in Google
• 70th in Bin...
Instant answers
Remember…
• One search engine is never enough
• To be continued…
Evaluating search engines
Evaluating search engines
Próximos SlideShares
Carregando em…5
×

Evaluating search engines

Criteria to take into account when evaluating a new search engines

  • Entre para ver os comentários

  • Seja a primeira pessoa a gostar disto

Evaluating search engines

  1. 1. Evaluating a search engine Phil Bradley, Websearch Academy, Internet Librarian International 2017
  2. 2. Who, what, where, when? • Whois? https://whois.icann.org/en
  3. 3. Wayback machine
  4. 4. “About us”
  5. 5. What sort of search engine is it? Free text?
  6. 6. Directory?
  7. 7. Multi/meta search?
  8. 8. Social media?
  9. 9. Help screens or FAQs?
  10. 10. Advanced search functionality?
  11. 11. Sometimes you need to dig deeper!
  12. 12. And then deeper still!
  13. 13. Use a search engine to search • “Crib sheet” • “Cheat sheet” • “Advanced search” • “Search functionality” • “Advanced features” • “Search operators” • “Search criteria” • Don’t forget images and .pdf files either!
  14. 14. Search functionality • Term A Term B • “Term A Term B” • Term A Term B OR Term C • (Term A Term B) OR Term C • Term A AND (Term B or Term C) • What other Boolean operators work? • Of special interest is NEAR • File type searches?
  15. 15. … continued • Site, country or region based search • Re-arrange results by date or relevance • Can you search for symbols such as ‘@’ or ‘#’? • Wildcards • Related options • What specialist search options do they provide you with?
  16. 16. Search result settings • Number of results displayed on the screen • Adult filter • Country or region • Auto-load results when scrolling, or auto suggest terms • Keyboard shortcuts
  17. 17. Privacy • Are searches private to you, or are they shared with the search engine, stored and used in the future? • What’s the advantage of less privacy, and why you might want more privacy
  18. 18. Coverage
  19. 19. Set a benchmark • Choose your preferred search engine and run a search in an area that you’re familiar with, and note the top results. • Specifically note the position of what you consider to be the ‘best’ results, however you define them. • Run the same search on the new search engine and compare
  20. 20. Search results • Where does it pull its results from? • How does it get its material?
  21. 21. Number of results – does it matter?
  22. 22. Fox red labrador
  23. 23. Fox red labrador • 18 different sites listed • 2 of them found by all three engines • 1,2,2 and 2,3,1 • 5 found by two of the three engines • 11 found by one of the three engines
  24. 24. How are results relevance ranked? (1) • On page and off page • The position of words • Next/near to each other • On the web page • Repetition • Exact terms and/or related terms (fuzzy logic) • Rare and unusual terms V more common terms • How quickly the page loads • URL structures, site maps, unbroken links
  25. 25. How are results relevance ranked? (2) • Off page • How many links point back to the page in question? • What is the authority of the linking page? • The use of anchor terms to link to the page listed in the results • Age of the website URL • Viewed on multiple devices
  26. 26. How and when are algorithmic changes evaluated? • Google employs hundreds of quality evaluation raters • Individual sites and searches can be checked • Are the results that move up or down in accordance with the views of the raters? • Live experiments to see if new changes mean more clicks • A short Matt Cutts video explains in more detail at: • https://youtu.be/muZuX9OaMLo
  27. 27. Difficulties in evaluating search results • Trying to work out what the users ‘intent’ was for the query • Eg. ‘Olympics’
  28. 28. Authority and validity of the results • “Martin Luther King” ‘A true historical examination’ • 9th in Google • 70th in Bing • 6th in Yandex • Cure for cancer 1st results • Cancertutor.com in Google • Express Newspaper in Bing • The truth about cancer in Yandex (book website)
  29. 29. Instant answers
  30. 30. Remember… • One search engine is never enough • To be continued…

×