This is a seminar/course I gave at the Metropolitan Library Council aimed at introducing layperson librarians to information retrieval.

1. Inside Search
Metro, NYC
November 15, 2017

2. Hello

3. Hello

4. Hello

5. Hello

6. Hello

7. EXERCISE

8. Exercise
Tell me about you.
1. What’s your position?
2. What kind of organization are you
from?
3. Why are you interested in search?

9. Agenda

11. don’t worry

12. 1. IR Basics
2. Search in the Modern Era
Agenda

13. Information
Retrieval 101

14. InformationUser
IR 101

15. InformationUser
IR 101
“pluto” Documents

16. IR 101
pluto?

17. Inverted Index

18. Inverted Index
1
2
3
“pluto and goofy...”
“pluto the dwarf planet...”
“stranger things...”

19. “pluto and goofy”
pluto
and
goofy
Inverted Index

20. Inverted Index
1pluto 2
planet 2
stranger 3

21. Inverted Index
1pluto 2
planet 2
stranger 3

22. Inverted Index
1 2

23. Ranking
Term
Frequency
Inverse
Document
Frequency
Term Frequency
Document Frequency

24. 1
2
“pluto and goofy...”
“pluto the dwarf planet...pluto”
Ranking

25. 1
2
Term Frequency = 1
Term Frequency = 2
Document Frequency = 2
Ranking

26. 1
2
TFIDF = 1/2 = 0.5
TFIDF = 2/2 = 1
Ranking

27. 1
2
“pluto and goofy...”, TFIDF = 0.5
“pluto the dwarf planet...pluto”, TFIDF = 1
Ranking

28. IR 101
Index Time Query Time
Forward Index
Tokenize
Inverted Index
Matching
Ranking

29. Multiple Terms
IR 101

30. Multiple Terms
“pluto planet”

31. “pluto planet”
pluto
planet
Multiple Terms

32. Multiple Terms
1pluto 2
planet 2
stranger 3

33. Multiple Terms
OR AND

34. Multiple Terms
1pluto 2
planet 2
1 2

35. Multiple Terms
Term Frequency
Document Frequency∑

36. Multiple Terms
score = tf-idfpluto + tf-
idfplanet

37. 1
2
“pluto and goofy...”
“pluto the dwarf planet...”
Multiple Terms

38. 1
2
tf-idfpluto = ½ tf-idfplanet = 0/2
tf-idfpluto = ½ tf-idfplanet = 1/1
Multiple Terms
Document Frequencypluto = 2
Document Frequencyplanet = 1

39. 1
2
½ + 0 = 0.5
½ + 1 = 1.5
Multiple Terms

40. 1
2
“pluto and goofy...”, TFIDF = 0.5
“pluto the dwarf planet...”, TFIDF = 1.5
Multiple Terms

41. Measurement
IR 101

42. Measurement
Precision Recall
How good are the results? How many of the good
documents are in the results?

43. Measurement
Precision Recall
# Good Results
# Results
# Good Results
# Good Documents

44. Measurement
What’s good?

45. 2
1
“pluto and goofy…”
“pluto the dwarf planet...”
Measurement
“pluto planet”

46. 5
1
“Pluto lost its status…”
“pluto the dwarf planet...”
Measurement
“pluto”

47. Measurement
Precision Recall

48. EXERCISE

49. Exercise
Find a precision and a recall problem.

50. Analysis
IR 101

51. 5
1
“Pluto lost its status…”
“pluto the dwarf planet...”
Analysis
“pluto”

52. “Pluto lost its status...”
Pluto
lost
its
status
Analysis

53. Pluto
lost
its
status
Analysis
pluto
lost
its
status

54. Analysis
Analysis
1. Tokenize
2. Transform each Token

55. Analysis
What happens if I search for Pluto?
1pluto 2

56. Analysis
Index Time Query Time
Forward Index
TokenizeAnalysis
Inverted Index
TokenizeAnalysis
Matching
Ranking

57. Analysis
What happens if I search for planets?
1planet 2

58. Analysis
planetsplanets

59. Analysis
precisionrecall
stemming
lemmatization
aggressive
stemming
planetsjournalism

60. Analysis
Whitespace
Tokenizer
StemLowercase
Analysis Chain

61. EXERCISE

62. Exercise
Think about
1. Tokenization
2. Normalization
3. Stemming
Find 3 analysis-related problems.

63. Beyond the 70s

64. Machine-Learned
Relevance
Beyond the 70s

65. Machine-Learned Relevance
1. Supervised (Learning to Rank)
2. Unsupervised

66. Learning to Rank
TF-IDF ≠ relevance

67. Learning to Rank
“pluto”
0.8
Document
Model

68. Learning to Rank
“pluto”
Yes/No
Document

69. Learning to Rank

70. Learning to Rank
Problems
1. Require lots of data

71. Learning to Rank
Problems
1. Require lots of data
2. Difficult to train
3. Difficult to administer/serve

72. Learning to Rank
Problems
1. Require lots of data
2. Difficult to train
3. Difficult to administer/serve
4. Bias

73. SKIN WINS

74. Unsupervised

75. Unsupervised
How do you know it’s any good?

76. Search is UX
Beyond the 70s

77. Search is UX
Search ≠ Ranking

78. Search is UX

80. Search is UX
Tools
1. Facets

81. Search is UX
Tools
1. Facets
2. Results

82. Search is UX
Tools
1. Facets
2. Results
3. Autosuggest

83. Search is UX
Tools
1. Facets
2. Results
3. Autosuggest
4. Suggestions

84. Search is UX
Reasons to focus on UX
1. Low-Intent Traffic
2. Leverage
3. Mobile
4. Lower-Stakes

85. EXERCISE

86. Exercise
Suggest 3 UX
enhancements.
Think about
1. Exploration
2. Disambiguation
3. Mobile
Tools
1. Facets
2. Results
3. Autosuggest
4. Suggestions

87. Query
Understanding
Beyond the 70s

88. Query Understanding
keywords ≠ ideas

89. f you wanted to make this one statement
as well.
Or another one.
Search results for “dress shirt”

90. Query Understanding
keywords ideas

91. Query Understanding
“dress shirt”
Category:
Clothing > Shirts > Dress Shirts

92. Query Understanding
Tokenization
Stemming
Synonyms
Language Detection
Spelling Correction
Entity Recognition
Query Expansion
Query Relaxation
Query Classification
Query Parsing
Query Segmentation
Knowledge Graphs

93. Query Understanding
Too many tools.
Too many theoretical
problems.

94. Query Understanding
Start with data:
1. What are people looking
for?
2. When are they not
successful?

95. Query Understanding
Search Exit

96. f you wanted to make this one statement
as well.
Or another one.
Search results for “dress”

97. Query Understanding
“fabric” Supply!Model

98. Query Understanding

99. Query Understanding

100. Query Understanding
“fabric” Supply!You

101. Search results for “fanny pack”

102. Search results for “bum bag”

103. Users and documents
speak different languages.

104. Hat?

105. Query Understanding
User Data Metadata

106. Query Understanding

107. Search results for “red nascar”

108. EXERCISE

109. Exercise
Optimize some important
or underperforming
queries.
Problems
1. Precision
2. Recall
3. Exploration
Tools
1. Analysis
2. Understanding
3. Metadata
4. UX

110. Search as a System

111. gio@relatedworks.io
www.relatedworks.io