Dawn Anderson, Move It Marketing As the amount of digital content search engines must make sense of increases, so too does the importance of clarity in the written and spoken word. In this session, you’ll learn about challenges and developments in NLP (natural language processing) and how to disambiguate vagueness to focus intent and content in your SEO tactics and strategy. Learn more at c3.conductor.com

1. “What&Happens&in&
Vagueness&Stays&in&
Vagueness’”
Dawn&Anderson

2. If#I#came#into#
your#hardware#
store…
Image Attribution: Acabashi [CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0)]

3. And$asked$for$“fork$handles”

4. It#kind#of#sounds#like#“four#candles”

5. What%about%if%I%said%“Got%
any%Os”?

6. It#reads#like#O’s#but#it#sounds#like#
other#things#as#well#as#…#0#(zero)’s#…#
for#the#gate

7. Homophones)* Examples)‘four)candles’)and)‘fork)handles’

8. What%about,%if,%in%the%very%next%sentence,%I%asked%“Got%any%P’s?”

9. You’d&
presume&I&
meant&P’s”

10. Because'of'the'
context'of'the'
previous'question

11. Not…%garden%
peas

12. The$Two$Ronnies –
‘British$Comedians’
Name%
Droppers
The%Confusing%
Library
Four%Candles
Crossed%Lines Mastermind

13. Almost'every'
other'word'in'the'
English'language'
has'multiple'
meanings

14. “The%meaning%of%a%word%is%
its%use%in%a%language”%
(Ludwig%Wittgenstein
,%1953)
Image attribution: Moritz Nähr [Public domain]

16. The$most$important$thing$to$remember$
is$I$have$a$Pomeranian$called$Bert

17. Disclaimer:+I+am+NOT+a+data+
scientist

18. But$I$will$be$talking$about$some$concepts$
covering:
Data$Sciene
01
Information$
Retrieval
02
Algorithms
03
Linguistics
04
Information$
Architecture
05
Library$
Science
Category$
theory

19. Since…'These'are'all'areas'
connected'to'how'search'
engines'(try'to)'find'the'
right'information,'for'the'
right'informational'need'at'
the'right'time'for'the'right'
user

20. ‘information*retrieval’*in*web*search
To#extract#informational#resources#to#meet#a#
search#engine#user’s#information#need#at#time#
of#query.

21. Let$us$first$take$a$very$
simplistic$look$at$how$we$
know$search$engines$work

22. It’s%just%like%gathering%&%organizing%
books%in%a%library%system%or%using%an%
old%card%index%system

23. But$instead$we$are$taking$
words$(or$phrases)$and$
recording$where$they$live

24. EXAMPLE
Inverted.Index:.
Text.to.Doc.ID.
Mapping

25. And$then$picking$one$(or$some)$of$
these$‘word$homes’$(documents)$to$
meet$a$query

26. The$hard$part$is$knowing$how$to$
choose$the$right$documents,$in$
the$right$order,$at$the$right$time

27. Since&‘relevance’&to&one&user&is&not&‘relevance&to&another

28. For$some$queries$there$
can$be$only$one$answer

29. And$even$' Zero'query$
queries$' The$user$is$the$
query

30. Where%the%user%
might%be%looking%for%
a%restaurant%whilst%
travelling%at%60%mph%
on%a%highway?

31. So…
Just%what%is%the%right%information%need%for%the%
right%user%at%the%right%time?

32. It#Depends#…

33. Who?
What?
When?
Where?
Why?

34. Relevance(Matching(to(Query(Requires:
Understanding+meaning+of+words+in+content+&+query+(What?)
Understanding+meaning+of+word's+context+in+content+&+query+(What?)
Understanding+of+user’s+context+(Who+/+Where+/+When+/+Why?)
Understanding+of+collaboration+(Past+queries+/+popularity+/+
reinforcement+/+learning+to+rank)

35. Matching)‘content’)with)
‘intent’)requires)increasing)
precision

38. A"lot"of"content"is"kind"of"unfocused

39. Each%document%(page)%is%largely%just%a%‘stream%of%words’

40. Every&day&there&are&huge&volumes&of&new&indexable&data

41. Image&credit:&Paul&Newbury

42. Since&every&Single&Tweet&is&a&
new&web&Page

43. Many%websites%
(and%
webpages)%are%
not%logically%
organized%at%all
Unstructured*data*is*voluminous
Filled*with*irrelevance
Lacks*focus
Riddled*with*nuance
Lots*of*meaningless*text*and*further*
ambiguating*jabber

44. Most%text(filled%web%pages%
could%be%considered%
unstructured,%noisy%data
Blog%==%Blah%Blah

45. Structured(versus(unstructured(data
• Structured(data(– high(
degree(of(organization
• Readily(searchable(by(
simple(search(engine(
algorithms(or(known(search(
operators((e.g.(SQL)
• Logically(organized
• Often(stored(in(a(relational(
database

46. When%we%compare%them%with%highly%organized%relational%database%systems

47. A"form"of"structured"(&"semi0structured)"data"– Entities,"Knowledge"
Graphs,""Knowledge"Bases"&"Knowledge"Repositories

48. “Entities(help(to(bridge(the(
gap(between(structured(
and(unstructured(data”(
(Krisztian(Balog,(ECIR2019(
Keynote)

49. Author'of'Entity-Oriented'Search'
– Free'on'Open'Access

50. Using&structured&data&is&an&obvious&way&to&
disambiguate&in&both&content&&&query&
understanding

51. Two$things$(entities)$are$similar$if$
they$have$a$not$so$distant$common$
ancestor

52. ‘Is$A’$Hierarchies

53. Knowledge)
Graphs)using)
triples)
(subject,)
predicate,)
object)

54. IsA Concepts+in+entities+
&+their+relationships+
can+be+mapped+&+
categorised

55. A"well"organized"
website"can"
resemble"a"
knowledge"graph

56. Since website*
is*NOT*ALL*
unstructured*
data*even*
before*
structured*
data*markup
It#can#have#a#hierarchy
It#can#have#weighted#sections
It#can#have#metadata
It#(often)#has#a#tree#like#structure

57. As#long#as#there#is#
understanding#of##
notions#of#
categorical#
‘inheritance’

59. Categories*&*subcategories’

60. Semi%
structured,
data
• Hierarchical)nature)of)a)
website
• Tree)structure
• Well)sectioned)and)
including)clear)containers)
and)meta)headings
• An)ontology)map)between)
semi)and)structured

61. Internal(linking(can(be(
as(much(about(
ontology(mapping(as(
crawl(optimisation

62. And$many$pages$lack$the$things$that$emphasise important$topics$and$structure

65. Ontology(Driven(Natural(Language(Processing
Image credit: IBM
https://www.ibm.com/developerworks/community/blogs/nlp/entry/ontology_driven_nlp

66. But$even$named$
entities$can$be$
polysemic

67. Did$you$mean?
•Amadeus(Mozart(
(composer)
•Mozart(Street
•Mozart(Cafe

68. And$verbally…Who$
(what)$are$you$talking$
about?
”Lyndsey$Doyle”$or$
”Linseed$Oil”?

69. And$not$
everyone$or$
thing$is$mapped$
to$the$
knowledge$
graph

71. On#their#own#single#words#
have#no#semantic#meaning

72. Even%if%we%understand%the%
entity%(thing)%itself%we%need%
to%understand%word’s%context

73. Semantic)context)matters
•He#kicked#the#bucket
•I#have#yet#to#cross#that#off#my#
bucket#list
•The#bucket#was#filled#with#
water

74. Unfortunately…-when-things-lack-topical-focus-and-relevance

76. How$can$search$
engines$fill$in$the$
gaps$between$
named$entities?

77. When%they%can’t%even%tell%the%difference%between%Pomeranians%and%pancakes

78. They%need%
‘Text%
cohesion’
Cohesion is#the grammatical and#
lexical linking#within#a#text#
or sentence that#holds#a#text#
together#and#gives#it#meaning.#
Without#surrounding#words#the#
word#bucket#could#mean#
anything#in#a#sentence

79. If#I#said#to#you…
“I’ve#got#a#new#
jaguar”

80. “It’s&in&the&garage”&
(sidenote:&this&is&not&my&garage)

81. You$probably$
wouldn’t$expect$
to$see$this

82. Because'garage'and'car'go'together

83. The$‘jaguar’$(cat)$is$the$
odd$one$out$

84. Garage&and&car&and&jaguar&‘co.occur’&in&common&language&
together&&. ‘garage’&added&context&to&’jaguar’&the&‘car’

85. But$if$we$understood$a$topic$is$about$felines$we$
might$be$more$confident$of$a$jaguar$‘cat’

86. “You%shall%know%a%word%by%
the%company%it%keeps”%
(John%Rupert%Firth,%1957)

87. Natural'Language'
Disambiguation

89. Probabilistic+‘Guesstimation’

90. Teaching)machines)to)
understand)what)words)
live)nearby)each)other)in)
context

91. Then%we%can%disambiguate%
through%co3occurrence

92. Using&‘Distributional&
Similarity’&
(Relatedness)

93. Nearest'Neighbours (Similarity)'Evaluations
KNN#– K%Nearest%Neighbour

94. 2"words"are"similar"if"they"
co2occur"with"similar"words

95. 2"words"are"similar"if"they"occur"in"a"given"
grammatical"relation"with"the"same"words
Harvest Peel Eat Slice

96. First&Level&Relatedness&– Words&
that&appear&together&in&the&same&
sentence

97. Second'Level'Relatedness'– words'
that'co3occur'with'the'same'
‘other’'words

98. Coast&and&
Shore&
Example
Coast&and&shore&have&a&similar&
meaning
They&co4occur&in&first&and&second&
level&relatedness&documents&in&a&
collection
They&would&receive&a&high&score&in&
similarity

99. Language'models'are'trained'on'
very'large'text'corpora'or'
collections'(loads'of'words)'to'
learn'distributional'similarity

100. Vector'representations'of'words'(Word'Vectors)

101. Models'learn'the'
weights'of'the'
similarity'and'
relatedness'distances

102. An#important#
part#of#this#is#
‘Part#of#
Speech’#(POS)#
tagging

103. Continuous(Bag(of(Words((CBoW)(
(Method)(or(Skip8gram((Opposite(of(
CBoW)
Continuous(Bag(of(Words(!
Taking(a(continuous(bag(of(
words(with(no(context(utilize(a(
context(window(of(n(size(n!gram)(
to((ascertain(words(which(are(
similar(or(related(using(Euclidean(
distances(to(create(vector(
models(and(word(embeddings

104. A"Moving"Word"‘Context"Window’

105. And$build$vector$
space$models$for$
word$
embeddings
king$7 man$+$
woman$=$queen

106. Tensorflow*(tool)*
&*e.g.*Word2Vec*
or*Glove2Vec*
(language*models)

107. Layers'Everywhere

108. Concept2Vec
Ontological
concepts

109. Google’s(Topic(Layer(is(a(new(
Layer(in(the(Knowledge(Graph

110. Example(Microsoft(Concept(Distribution(Layer

111. Past%language%models%
(e.g.%Word2Vec%&%
Glove2Vec)%built%
context<free%word%
embeddings

112. Did$you$mean$“bank”?
Or$did$you$mean$“bank”?

113. Most%language%modellers%are%uni0directional
Source'Text
Writing a list of random sentences is harder than I Initially thought it would be
Writing a list of random sentences is harder than I Initially thought it would be
Writing a list of random sentences is harder than I Initially thought it would be
Writing a list of random sentences is harder than I Initially thought it would be
They can traverse over the word’s context window from only left to right or
right to left. Only in one direction, but not both at the same time

114. They%can%only%look%at%words%in%the%context%
window%before%and%not%the%words%in%the%rest%of%
the%sentence.%%Nor%sentence%to%follow%next

115. Often&the&next&
sentence&REALLY&
matters

116. I""remember"the"last"words"my"Grandpa"
said"before"he"kicked"the"bucket…
…How"far"do"you"reckon"I"could"kick"this"
bucket?

117. Meet$BERT

118. Not$the$pomeranian BERT

119. BERT%
(Bidirectional%
Encoder%
Representation%
from%
Transformers)

120. BERT%is%different.%%BERT%uses%bi1directional%
language%modelling.%%The%FIRST%to%do%this
Source'Text
Writing a list of random sentences is harder than I Initially thought it would be
Writing a list of random sentences is harder than I Initially thought it would be
Writing a list of random sentences is harder than I Initially thought it would be
Writing a list of random sentences is harder than I Initially thought it would be
Bert can see both the left and the right hand side of the target word

121. BERT%has%been%open%sourced%
by%Google%AI

122. Google’s(move(to(
open(source(BERT(
may(change(natural
language(processing(
forever

123. Bert%uses%‘Transformers’%&%
’Masked%Language%Modelling’

124. BERT%can%see%the%WHOLE%
sentence%on%either%side%of%a%
word%(contextual%language%
modelling)%and%all%of%the%
words%almost%at%once

125. BERT%has%been%pre.trained%on%a%
lot%of%words%…%on%the%whole%of%
the%English%Wikipedia%(2,500%
million%words)

126. BERT%can%identify%which%sentence%
likely%comes%next%from%two%choices

127. The$ML$&$NLP$Community$are$very$excited$about$
BERT

128. Vanilla&BERT&provides&a&pre2trained&
starting&point&layer&for&Neural&
Networks&in&machine&learning&&&
natural&language&diverse&tasks

129. Everybody)wants)to)‘Build4a4
BERT.))Now)there)are)loads)of)
algorithms)with)BERT

130. Whilst'BERT'has'been'pre2
trained'on'Wikipedia'it'is'fine2
tuned'on'‘questions'and'answer'
datasets’

131. Whilst'BERT'has'been'
pre2trained'on'
Wikipedia'it'is'fine2
tuned'on'‘questions'
and'answer'datasets’

132. Researchers(compete(over(Natural(Language(Understanding(with(e.g.(
SQuAD (Stanford(Question(&(Answering(Dataset)

133. BERT%now%even%beats%the%human%
reasoning%benchmark%on%SQuAD

134. Not$to$be$outdone$– Microsoft$also$extends$on$BERT$with$MT:DNN

135. In#GLUE#– It’s#
Humans,#MT3
DNN,#then#
BERT

136. It’s%not%just%words%in%content%
that%need%to%be%
disambiguated%though

138. How$can$search$
engines$
understand$
intents?

139. Query&Classifications&1
There&are&some&we&know&
of&already

140. We#need#to#understand#how#
queries#have#been#classified#
by#search#engines

142. Google’s(
Quality(Raters(
Guide(
simplifies(&(
extends(these
Know%query%==%Informational
Website%query%==%Navigational
Do%query%==%Transactional
Visit%in%person%==%Local%intent

143. There%are%also%
several%types%of%
queries%too
(Krisztian Balog,%
ECIR,%2019)
Keyword(queries((Normal(keyword(queries)
Keyword++(queries((Faceted(/(filtered(
queries)
Zero;Query(queries((User(is(the(query)
Natural(language(queries
Structured(queries((e.g.(SQL)

145. ‘Dresses’'is'clearly'classified'
as'a'transactional'query

146. Whilst'keyword'research'
tools'are'useful...'The'
SERPs'tell'us'some'
secrets'on'‘initial'intent’'
detection

147. But$If$I$searched$for$“fork$handles’

148. Would&I&mean&
“Handles&for&
forks”?

149. The$organic$results$are$NOT$for$fork$handles

150. The$Two$Ronnies – ‘Four$Candles’

151. No#high#organic#ranking#candles#or#forks

152. Apart&from…&Ebay selling&an&actual&fork&handle&in&position&8

153. Almost'completely'
‘informational'&'video'
results’'(not'
transactional)

154. The$overwhelming$‘intent’$
was$detected

155. Even%in%voice%search%&%assistant

156. Temporal)Dynamic)Intent)(Burstiness))is)a)huge)factor)for)intent

157. At#certain#times#far#more#intents#will#be#transactional

158. “dresses”,(“shoes”,(
“bags”
“buy(dresses”,(“buy(
shoes”,(“buy(bags”,(
“dress(sales”,(“shoe(
sales”
Really means

159. And$sometimes$only$
reasons$a$particular$
audience$would$
understand$spike$
temporal$queries

160. Sometimes(it(is(other(events(which(trigger(unexpected(queries

161. [Four&candles]&interest&over&time

162. [Fork&Handles]&interest&over&time

163. Often&intents&can&be&
modelled&according&to&
predicted&intent&shifts

164. Google&Trends&will&only&show&interest,&not&intent

165. The$exact$same$queries$
have$different$intent$at$
different$times$&$
different$locations

166. Let’s&Take&The&Query&[Easter]

167. What%did%you%really%mean%when%you%searched%for%‘Easter’?
When%did%
you%search%
for%‘Easter’?
A%few%weeks%
before%Easter
A%few%days%
before%Easter
During%Easter
What%you%
mostly%meant
When%is%
Easter?
Things%to%do%
at%Easter
What%is%the%
meaning%of%
Easter?
Radinsky, K., Svore, K.M., Dumais, S.T., Shokouhi, M., Teevan, J., Bocharov, A. and
Horvitz, E., 2013. Behavioral dynamics on the web: Learning, modeling, and
prediction. ACM Transactions on Information Systems (TOIS), 31(3), p.16.

168. “Easter”)Query)Intent)Shift

169. Predicting*the*future*
with*Web*Dynamics
• The$journey$to$predict$the$future:$Kira$
Radinsky$at$TEDxHiriya

170. This%is%‘Query%
Intent%Shift’

171. “When&users’&information&
needs&change&over&time,&the&
ranking&of&results&should&also&
change&to&accommodate&
these&needs.”&(Radinsky,&
2013)

175. Your%ranking%flux%might%well%be%shifting%query%intents%at%scale

176. The$passage$of$time$adds$new$meaning$
sometimes$too

177. Another(Great(‘Ronnies’(Sketch(BTW

178. The$rise$and$fall$
of$the$
Blackberry?

179. In#query#understanding#sometimes#users#don’t#know#what#they#want

180. Sometimes(the(
searcher(query(is(
a(‘cold(start’(
query

181. Broad&queries&might&call&for&
result&diversification&due&to&
lack&of&intent&detection

182. Search'
engines'may'
return'a'
blend'of'
results'to'
match'these
Freshness
Serendipity
Novelty
Diversity

183. The$searcher$has$to$click$
around$to$provide$
feedback$on$their$intent$
or$reformulate$the$query$
by$entering$something$
else$(‘query$refinement’)

184. To#then#deliver#sequential#
queries#with#greater#intent#
understanding

186. Query&Refinement&says…&“Your&move&next”

187. Sometimes(there(
are(not(enough(
precise(results(
either

188. And$result$precision$is$not$possible

189. And$this$can$increase$recall$due$to$query$expansion$or$relaxation

190. Precision)versus)recall)in)search)results

191. The$intent$tied$to$the$
page$type$matters$too

192. Different(features(matter(to(users(
more(dependent(on(the(domain
News%(freshness)
Jobs%(salary,%job%title,%location)
Restaurants%(location,%cuisine)
Shopping%(price)

193. In#theory…#a#consolidated#page#should#rank#higher…#but…

194. Who?
What?
When?
Where?
Why?

195. Mixing&‘intent’&
on&target&pages&
can&be&like&oil&
and&water

196. So#watch#out#for#
random#informational#
blurb#on#ecommerce#
pages

197. Watch&out&for&both&topical&&&
intent&drift

198. And$watch$out$you$don’t$lose$a$featured$snippet$by$changing$intent

199. Focus&&&
disambiguate

200. One$oar$on$topic$– the$other$on$intent

201. To#keep#the#boat#going#straight

202. But$wait…$Understanding$word’s$context$
more$is$NOT$understanding$‘The$Whole$
Context’

203. Where%the%
user%is%truly%
‘the%query’

204. Since&humans&are&unique&individuals

205. Truly&PERSONAL&AI&is&not&
possible&without&a&
PERSONAL&KNOWLEDGE&
GRAPH&(Krisztian&Balog,&
ECIR&2019)

206. Assistant'+'Home'+'Discover'
+'Search'App'+'Desktop

208. A"Recent"Microsoft"Personal"Knowledge"Graph"Patent

209. Semantic)
Query)
Understanding)
Example
Source'&'Image'Attribution'
NTent

210. That is'a'whole'different'
‘kettle'of'fish’

211. And$that$is$for$another$
time…

212. In#the#
meantime…#
remember…