SlideShare uma empresa Scribd logo
1 de 42
Baixar para ler offline
Tetherless World Constellation

Why Watson Won: A
cognitive perspective
Jim Hendler
and Simon Ellis
Tetherless World Professor of Computer,Web and Cognitive Sciences
Director, Rensselaer Institute for Data Exploration and Applications

Rensselaer Polytechnic Institute (RPI)
http://www.cs.rpi.edu/~hendler
@jahendler (twitter)
IBM Watson
Tetherless World Constellation
How’d I get into it? Watson and Semantic Web
Tetherless World Constellation

IBM
Watson and Semantic Web
Tetherless World Constellation

IBM
Is Watson cognitive?

???

“The computer‟s techniques for unraveling Jeopardy! clues sounded
just like mine. That machine zeroes in on key words in a clue, then
combs its memory (in Watson‟s case, a 15-terabyte data bank of
human knowledge) for clusters of associations with those words. It
rigorously checks the top hits against all the contextual information it
can muster: the category name; the kind of answer being sought; the
time, place, and gender hinted at in the clue; and so on. And when it
feels „sure‟ enough, it decides to buzz. This is all an instant, intuitive
process for a human Jeopardy! player, but I felt convinced that under
the hood my brain was doing more or less the same thing.”
— Ken Jennings
Outline
Tetherless World Constellation

• Is Ken right?
– How Watson Works
– Watson as a cognitive architecture??
– Beyond Watson
Inside Watson

???

Watson pipeline as published by IBM; see IBM J Res & Dev 56 (3/4), May/July 2012, p.
Question Analysis

???
???

Question analysis


What is the question asking for?



Which terms in the question refer to the answer?



Given any natural language question, how can Watson
accurately discover this information?

Who is the president of
Rensselaer Polytechnic Institute?

Question
Analysis

Focus Terms:
“Who”, “president of Rensselaer
Polytechnic Institute”
Answer Types: Person, President
Parsing and semantic analysis


What information about a previously unseen piece of
English text can Watson determine?



???

How is this information useful?

Natural Language Parsing

Semantic Analysis

- grammatical structure

- meanings of words, phrases, etc.

- parts of speech

- synonyms, entailment

- relationships between words

- hypernyms, hyponyms

- ...etc.

- ...etc.
Question analysis pipeline

Unstructured
Question Text

Parsing
&
Semantic
Analysis

Structured Annotations
of Question:
Focus, answer types,
Useful search queries

Machine
Learning
Classifiers

???
Search Result Processing and Candidate
???
Generation
Primary Search

???



Primary Search is used to generate the corpus of
information from which to take candidate answers,
passages, supporting evidence, and essentially all textual
input to the system



It formulates queries based on the results of Question
Analysis



These queries are passed into a (cached) search engine
which returns a set number of highly relevant documents
and their ranks.


on the open Web this could be a regular search engine (our
Candidate Generation

???



Candidate Generation generates a wide net of possible
answers for the question from each document.



Using each document, and the passages created by
Search Result Processing, we generate candidates using
three techniques:


Title of Document (T.O.D.): Adds the title of the document as a
candidate.



Wikipedia Title Candidate Generation: Adds any noun phrases
within the document‟s passage texts that are also the titles of
Wikipedia articles.



Anchor Text Candidate Generation: Adds candidates based on
the hyperlinks and metadata within the document.
Search Result Processing and
Candidate Generation

???
Scoring & Ranking

???
Scoring


Analyzes how well a candidate answer relates to the
question



Two basic types of scoring algorithm


Context-independent scoring



Context-dependent scoring

???
Types of scorers


Context-independent


Question Analysis



Ontologies (DBpedia, YAGO, etc)





???

Type hierarchy reasoning

Context-dependent


Analyzes feature of the natural language environment where
candidates were found




Relies on “passages” found during search

Many special purpose ones used in Jeopardy
Scorers


Passage Term Match



Textual Alignment



???

Skip-Bigram


Each of these scores supportive evidence



These scores are then merged to produce a single candidate
score
Example:Textual Alignment

???



Finds an optimal alignment of a question and a passage



Assigns “partial credit” for close matches




“Who is the President of RPI?”
Who

President of RPI.

Shirley Ann Jackson is the President of RPI.
Skip-Bigram


Constructs a graph


Nodes represent terms (syntactic objects)





???

Edges represent relations

Extracts skip-bigrams






A skip-bigram is a pair of nodes either directly connected or
which have only one intermediate node
Skip-bigrams represent close relationships between terms

Scores based on number of common skip-bigrams
Example


Who authored
“The Good Earth”?



“Pearl Buck, author of
the good earth…”

???
Watson Summary
Tetherless World Constellation

• Watson works by
– Analyzing the question
• natural language parsing
• text extraction

– Generating a large number of
candidates
• mostly search heuristics

– Scoring each
• through multiple scorers
• with weights adjusted by learning algorithm

– Returning top candidate
MiniDeepQA (Not Watson!)


???

RPI students implementing a DeepQA pipeline to explore
the principles underlying this kind of Q/A system

(THIS IS NOT WATSON!)


Pipeline development



Data caching



Graphical and command line interfaces



Parsing



Scoring
???

Examples

Right answer
???

Examples

Right answer?
???

Examples

Right answer
???

Examples

Right
answer??
???

Examples

had to get this
one right!
Scoring

???



One of DeepQA‟s main strengths is aggregating a
number of different scoring algorithms capable of
running in parallel.



RPI scorers are primitive compared to IBM‟s, but


allow us to explore the principles



allow us to explore different algorithms for computing
scores



allow us to create new ones not tried by IBM
Scoring Principles: combine
evidence


He was the Prime Minister of Canada in 1993.




candidates could include Trudeau, Harper, Campbell, Chretien,
Mulroney…

Try (Research):


Trudeau was Prime Minister of Canada in 1993 (doesn‟t match)



Campbell was Prime Minister of Canada in 1993 (MATCH)



Chretien was Prime Minister of Canada in 1993 (MATCH)



Scoring Research & type match


Trudeau: Re-search NO; Type: Yes



Campbell: Re-search YES; Type: No



Chretien: Re-search YES; Type: Yes

WHO WAS CHRETIEN?

???
New Scoring types


???

We can explore how new kinds of information can be
added to the Watson scoring pipeline


Example: new NLP extraction techniques




Example: Specialized Web Sources




Adding a ML-based extractor built by Heng Ji
Database advisor project

Example: More complex inferencing


Jeopardy questions are unambiguous, real world questions aren‟t

•
•


Where is Montreal?
Who is Jim Hendler?

Example: Special purpose reasoning…
Special purpose reasoning

???

• Can we match simulation (or steer) large scale
simulations to help answer NL questions?

-

eg. Answer questions such as “Why” and “How”
integrated with large scale simulations
Alternate Universe Reasoning
(Contexts)


How can a Watson reasoner appropriately use Q/A
contexts?


Where was Yoda born?




Very little is known about Yoda's early life.
He was from a remote planet,
but which one remains a mystery.

Where was Yoda made?



designed and built by Stuart Freeborn





The Yoda puppet was originally
for LucasFilm and Industrial Light & Magic.

Where did Yoda live?




Jedi Master Yoda went into voluntary exile on Dagobah

Where did Yoda live in the Phantom Menace?

???
But back to the original question
Tetherless World Constellation

• Q: How does Watson fare as a
cognitive model?
• A: Poorly
– no conversational ability
– no concept of self
– no deeper reasoning

…

• Q: How does Watson fare as a model
of question answering?
Watson and Q/A
Tetherless World Constellation

• Watson’s feed-forward pipeline has
the following properties
– lots of candidates generated
• the more the better

– “ad hoc” filtering pipelines
• domain independent usually score lower
than domain dependent

– no “counter-reasoning” between
answers
• separately scored, only comparison is
numbers
Production rules, modules, etc
Tetherless World Constellation

Production Rule style Architectures cf ACT-R (Anderson 1974; …2012)
- modularization, but not Watson style
- parallelization, but in rule productions (procedural memory)
- declarative memory is fact based
Watson is not well correlated, except for using search for declarative
memory
Network based
Tetherless World Constellation

Network based architectures (cf. spreading activation (Collins 75),
marker-passing (Hendler 86) … Microsaint 2006)
- positive activations
- inhibitory nodes (or other negative enforcers)
Watson has no negative inhibition, does use network-based scorers
MAC/FAC
Tetherless World Constellation

MAC/FAC (Gentner & Forbus, 1991)
Many are chosen, few are called model of analogic reasoning
Strong correspondence in performance, not in mechanism
New work by Forbus (SME) uses a more feed-forward mechanism
(Discussions in progress)
Cognitive Architecture? Watson as “component”

Decision Making

Memory
Reasoning

Watson, Cogito, and Clarion

Office of Research
Summary
Tetherless World Constellation

• Watson won by a combination of
–
–
–
–
–

natural language processing
search technologies
semantic typing (minimal reasoning)
scoring heuristics
machine learning (scorer tuning)

• Watson Q/A has some interesting analogies to
cognitive architectures of the past
– but mainly at a “level of abstraction”

• Watson as a memory component in a more
complex cognitive system is a very intriguing
possibility
Questions?
Tetherless World Constellation

Mais conteúdo relacionado

Mais procurados

Broad Data (India 2015)
Broad Data (India 2015)Broad Data (India 2015)
Broad Data (India 2015)James Hendler
 
Social Machines: The coming collision of Artificial Intelligence, Social Netw...
Social Machines: The coming collision of Artificial Intelligence, Social Netw...Social Machines: The coming collision of Artificial Intelligence, Social Netw...
Social Machines: The coming collision of Artificial Intelligence, Social Netw...James Hendler
 
Watson at RPI - Summer 2013
Watson at RPI - Summer 2013Watson at RPI - Summer 2013
Watson at RPI - Summer 2013James Hendler
 
Digital Archiving, The Semantic Web, and Modern AI
Digital Archiving, The Semantic Web, and Modern AIDigital Archiving, The Semantic Web, and Modern AI
Digital Archiving, The Semantic Web, and Modern AIJames Hendler
 
Social Machines - 2017 Update (University of Iowa)
Social Machines - 2017 Update (University of Iowa)Social Machines - 2017 Update (University of Iowa)
Social Machines - 2017 Update (University of Iowa)James Hendler
 
Data Big and Broad (Oxford, 2012)
Data Big and Broad (Oxford, 2012)Data Big and Broad (Oxford, 2012)
Data Big and Broad (Oxford, 2012)James Hendler
 
The Semantic Web: It's for Real
The Semantic Web: It's for RealThe Semantic Web: It's for Real
The Semantic Web: It's for RealJames Hendler
 
Facilitating Web Science Collaboration through Semantic Markup
Facilitating Web Science Collaboration through Semantic MarkupFacilitating Web Science Collaboration through Semantic Markup
Facilitating Web Science Collaboration through Semantic MarkupJames Hendler
 
Tragedy of the Data Commons (ODSC-East, 2021)
Tragedy of the Data Commons (ODSC-East, 2021)Tragedy of the Data Commons (ODSC-East, 2021)
Tragedy of the Data Commons (ODSC-East, 2021)James Hendler
 
"Why the Semantic Web will Never Work" (note the quotes)
"Why the Semantic Web will Never Work"  (note the quotes)"Why the Semantic Web will Never Work"  (note the quotes)
"Why the Semantic Web will Never Work" (note the quotes)James Hendler
 
The Future(s) of the World Wide Web
The Future(s) of the World Wide WebThe Future(s) of the World Wide Web
The Future(s) of the World Wide WebJames Hendler
 
HyperMembrane Structures for Open Source Cognitive Computing
HyperMembrane Structures for Open Source Cognitive ComputingHyperMembrane Structures for Open Source Cognitive Computing
HyperMembrane Structures for Open Source Cognitive ComputingJack Park
 
The Semantic Web: 2010 Update
The Semantic Web: 2010 Update The Semantic Web: 2010 Update
The Semantic Web: 2010 Update James Hendler
 
Towards An Improvement Community Platform for Service Innovation
Towards An Improvement Community Platform for Service InnovationTowards An Improvement Community Platform for Service Innovation
Towards An Improvement Community Platform for Service InnovationJack Park
 
CHIIR2017 - Tetris Model of Resolving Information Needs
CHIIR2017 - Tetris Model of Resolving Information NeedsCHIIR2017 - Tetris Model of Resolving Information Needs
CHIIR2017 - Tetris Model of Resolving Information NeedsMax L. Wilson
 
Federating Cultures: Human Knowledge, Teachers, Students
Federating Cultures: Human Knowledge, Teachers, StudentsFederating Cultures: Human Knowledge, Teachers, Students
Federating Cultures: Human Knowledge, Teachers, StudentsJack Park
 
SolrSherlock: Linkfinding among Biomolecules with Literature-based Discovery
SolrSherlock: Linkfinding among Biomolecules with Literature-based DiscoverySolrSherlock: Linkfinding among Biomolecules with Literature-based Discovery
SolrSherlock: Linkfinding among Biomolecules with Literature-based DiscoveryJack Park
 
The HCI Perspective on IR (DIR2016 Keynote)
The HCI Perspective on IR (DIR2016 Keynote)The HCI Perspective on IR (DIR2016 Keynote)
The HCI Perspective on IR (DIR2016 Keynote)Max L. Wilson
 
DMTM Lecture 01 Introduction
DMTM Lecture 01 IntroductionDMTM Lecture 01 Introduction
DMTM Lecture 01 IntroductionPier Luca Lanzi
 

Mais procurados (20)

Broad Data (India 2015)
Broad Data (India 2015)Broad Data (India 2015)
Broad Data (India 2015)
 
Social Machines: The coming collision of Artificial Intelligence, Social Netw...
Social Machines: The coming collision of Artificial Intelligence, Social Netw...Social Machines: The coming collision of Artificial Intelligence, Social Netw...
Social Machines: The coming collision of Artificial Intelligence, Social Netw...
 
Watson at RPI - Summer 2013
Watson at RPI - Summer 2013Watson at RPI - Summer 2013
Watson at RPI - Summer 2013
 
Digital Archiving, The Semantic Web, and Modern AI
Digital Archiving, The Semantic Web, and Modern AIDigital Archiving, The Semantic Web, and Modern AI
Digital Archiving, The Semantic Web, and Modern AI
 
Social Machines - 2017 Update (University of Iowa)
Social Machines - 2017 Update (University of Iowa)Social Machines - 2017 Update (University of Iowa)
Social Machines - 2017 Update (University of Iowa)
 
Data Big and Broad (Oxford, 2012)
Data Big and Broad (Oxford, 2012)Data Big and Broad (Oxford, 2012)
Data Big and Broad (Oxford, 2012)
 
The Semantic Web: It's for Real
The Semantic Web: It's for RealThe Semantic Web: It's for Real
The Semantic Web: It's for Real
 
Facilitating Web Science Collaboration through Semantic Markup
Facilitating Web Science Collaboration through Semantic MarkupFacilitating Web Science Collaboration through Semantic Markup
Facilitating Web Science Collaboration through Semantic Markup
 
Tragedy of the Data Commons (ODSC-East, 2021)
Tragedy of the Data Commons (ODSC-East, 2021)Tragedy of the Data Commons (ODSC-East, 2021)
Tragedy of the Data Commons (ODSC-East, 2021)
 
"Why the Semantic Web will Never Work" (note the quotes)
"Why the Semantic Web will Never Work"  (note the quotes)"Why the Semantic Web will Never Work"  (note the quotes)
"Why the Semantic Web will Never Work" (note the quotes)
 
The Future(s) of the World Wide Web
The Future(s) of the World Wide WebThe Future(s) of the World Wide Web
The Future(s) of the World Wide Web
 
HyperMembrane Structures for Open Source Cognitive Computing
HyperMembrane Structures for Open Source Cognitive ComputingHyperMembrane Structures for Open Source Cognitive Computing
HyperMembrane Structures for Open Source Cognitive Computing
 
The Semantic Web: 2010 Update
The Semantic Web: 2010 Update The Semantic Web: 2010 Update
The Semantic Web: 2010 Update
 
Broad Data
Broad DataBroad Data
Broad Data
 
Towards An Improvement Community Platform for Service Innovation
Towards An Improvement Community Platform for Service InnovationTowards An Improvement Community Platform for Service Innovation
Towards An Improvement Community Platform for Service Innovation
 
CHIIR2017 - Tetris Model of Resolving Information Needs
CHIIR2017 - Tetris Model of Resolving Information NeedsCHIIR2017 - Tetris Model of Resolving Information Needs
CHIIR2017 - Tetris Model of Resolving Information Needs
 
Federating Cultures: Human Knowledge, Teachers, Students
Federating Cultures: Human Knowledge, Teachers, StudentsFederating Cultures: Human Knowledge, Teachers, Students
Federating Cultures: Human Knowledge, Teachers, Students
 
SolrSherlock: Linkfinding among Biomolecules with Literature-based Discovery
SolrSherlock: Linkfinding among Biomolecules with Literature-based DiscoverySolrSherlock: Linkfinding among Biomolecules with Literature-based Discovery
SolrSherlock: Linkfinding among Biomolecules with Literature-based Discovery
 
The HCI Perspective on IR (DIR2016 Keynote)
The HCI Perspective on IR (DIR2016 Keynote)The HCI Perspective on IR (DIR2016 Keynote)
The HCI Perspective on IR (DIR2016 Keynote)
 
DMTM Lecture 01 Introduction
DMTM Lecture 01 IntroductionDMTM Lecture 01 Introduction
DMTM Lecture 01 Introduction
 

Destaque

IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!
IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!
IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!Tony Pearson
 
World of Watson 2016 - Architecting your Analytics House
World of Watson 2016 - Architecting your Analytics HouseWorld of Watson 2016 - Architecting your Analytics House
World of Watson 2016 - Architecting your Analytics HouseKeith Redman
 
Attitude Changes Everything
Attitude Changes EverythingAttitude Changes Everything
Attitude Changes Everythingpudding37
 
Every Crisis is Global, Social, Viral
Every Crisis is Global, Social, ViralEvery Crisis is Global, Social, Viral
Every Crisis is Global, Social, ViralGaurav Mishra
 
Ch. 2 -_the_biological_perspective
Ch. 2 -_the_biological_perspectiveCh. 2 -_the_biological_perspective
Ch. 2 -_the_biological_perspectivemcolon344
 
An Evolutionary Perspective on Approximate RDF Query Answering
An Evolutionary Perspective on Approximate RDF Query AnsweringAn Evolutionary Perspective on Approximate RDF Query Answering
An Evolutionary Perspective on Approximate RDF Query AnsweringChristophe Guéret
 
Drugan Notes- Biological Perspective
Drugan Notes- Biological PerspectiveDrugan Notes- Biological Perspective
Drugan Notes- Biological PerspectiveKim Drugan
 
Learning styles from a multicultural perspective
Learning styles from a multicultural perspectiveLearning styles from a multicultural perspective
Learning styles from a multicultural perspectiveNamchalla LSS
 
Valuing ecosystem services: a biological perspective
Valuing ecosystem services: a biological perspectiveValuing ecosystem services: a biological perspective
Valuing ecosystem services: a biological perspectiveKent Holsinger
 
The bee effect: Action to effect change
The bee effect: Action to effect changeThe bee effect: Action to effect change
The bee effect: Action to effect changeAllen McClinton
 
Managing Multicultural Individuals
Managing Multicultural IndividualsManaging Multicultural Individuals
Managing Multicultural IndividualsMargareta Heidt
 
C:\Multicultural Education Powerpoint
C:\Multicultural Education PowerpointC:\Multicultural Education Powerpoint
C:\Multicultural Education Powerpointnkiogima
 
Evolutionary perspective
Evolutionary perspectiveEvolutionary perspective
Evolutionary perspectivempape
 
IBM Watson & Open Source Software - LinuxCon 2012
IBM Watson & Open Source Software - LinuxCon 2012IBM Watson & Open Source Software - LinuxCon 2012
IBM Watson & Open Source Software - LinuxCon 2012iportilla
 
our behaviour is the foundation of our attitude and self perception
our behaviour is the foundation of our attitude and self perceptionour behaviour is the foundation of our attitude and self perception
our behaviour is the foundation of our attitude and self perceptionParveen Bano
 

Destaque (20)

IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!
IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!
IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!
 
World of Watson 2016 - Architecting your Analytics House
World of Watson 2016 - Architecting your Analytics HouseWorld of Watson 2016 - Architecting your Analytics House
World of Watson 2016 - Architecting your Analytics House
 
2010 1 materialism1
2010 1 materialism12010 1 materialism1
2010 1 materialism1
 
Attitude Changes Everything
Attitude Changes EverythingAttitude Changes Everything
Attitude Changes Everything
 
Every Crisis is Global, Social, Viral
Every Crisis is Global, Social, ViralEvery Crisis is Global, Social, Viral
Every Crisis is Global, Social, Viral
 
Ch. 2 -_the_biological_perspective
Ch. 2 -_the_biological_perspectiveCh. 2 -_the_biological_perspective
Ch. 2 -_the_biological_perspective
 
An Evolutionary Perspective on Approximate RDF Query Answering
An Evolutionary Perspective on Approximate RDF Query AnsweringAn Evolutionary Perspective on Approximate RDF Query Answering
An Evolutionary Perspective on Approximate RDF Query Answering
 
Drugan Notes- Biological Perspective
Drugan Notes- Biological PerspectiveDrugan Notes- Biological Perspective
Drugan Notes- Biological Perspective
 
Learning styles from a multicultural perspective
Learning styles from a multicultural perspectiveLearning styles from a multicultural perspective
Learning styles from a multicultural perspective
 
Valuing ecosystem services: a biological perspective
Valuing ecosystem services: a biological perspectiveValuing ecosystem services: a biological perspective
Valuing ecosystem services: a biological perspective
 
The bee effect: Action to effect change
The bee effect: Action to effect changeThe bee effect: Action to effect change
The bee effect: Action to effect change
 
Managing Multicultural Individuals
Managing Multicultural IndividualsManaging Multicultural Individuals
Managing Multicultural Individuals
 
Inculcate Self Confidence & Self Belief
Inculcate Self Confidence & Self Belief Inculcate Self Confidence & Self Belief
Inculcate Self Confidence & Self Belief
 
Watson System
Watson SystemWatson System
Watson System
 
C:\Multicultural Education Powerpoint
C:\Multicultural Education PowerpointC:\Multicultural Education Powerpoint
C:\Multicultural Education Powerpoint
 
IBM's watson
IBM's watsonIBM's watson
IBM's watson
 
Watson on bluemix
Watson on bluemixWatson on bluemix
Watson on bluemix
 
Evolutionary perspective
Evolutionary perspectiveEvolutionary perspective
Evolutionary perspective
 
IBM Watson & Open Source Software - LinuxCon 2012
IBM Watson & Open Source Software - LinuxCon 2012IBM Watson & Open Source Software - LinuxCon 2012
IBM Watson & Open Source Software - LinuxCon 2012
 
our behaviour is the foundation of our attitude and self perception
our behaviour is the foundation of our attitude and self perceptionour behaviour is the foundation of our attitude and self perception
our behaviour is the foundation of our attitude and self perception
 

Semelhante a Why Watson Won: A cognitive perspective

Ibm cognitive seminar march 2015 watsonsim final
Ibm cognitive seminar march 2015  watsonsim finalIbm cognitive seminar march 2015  watsonsim final
Ibm cognitive seminar march 2015 watsonsim finaldiannepatricia
 
Using and learning phrases
Using and learning phrasesUsing and learning phrases
Using and learning phrasesCassandra Jacobs
 
Iulia Pasov, Sixt. Trends in sentiment analysis. The entire history from rule...
Iulia Pasov, Sixt. Trends in sentiment analysis. The entire history from rule...Iulia Pasov, Sixt. Trends in sentiment analysis. The entire history from rule...
Iulia Pasov, Sixt. Trends in sentiment analysis. The entire history from rule...IT Arena
 
Introduction to Natural Language Processing
Introduction to Natural Language ProcessingIntroduction to Natural Language Processing
Introduction to Natural Language ProcessingPranav Gupta
 
Text Analytics for Semantic Computing
Text Analytics for Semantic ComputingText Analytics for Semantic Computing
Text Analytics for Semantic ComputingMeena Nagarajan
 
Extracting Relevant Questions to an RDF Dataset Using Formal Concept Analysis
Extracting Relevant Questions to an RDF Dataset Using Formal Concept AnalysisExtracting Relevant Questions to an RDF Dataset Using Formal Concept Analysis
Extracting Relevant Questions to an RDF Dataset Using Formal Concept AnalysisMathieu d'Aquin
 
Factoid based natural language question generation system
Factoid based natural language question generation systemFactoid based natural language question generation system
Factoid based natural language question generation systemAnimesh Shaw
 
Web & text mining lecture10
Web & text mining lecture10Web & text mining lecture10
Web & text mining lecture10Houw Liong The
 
Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A compreh...
Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A compreh...Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A compreh...
Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A compreh...Amit Sheth
 
Question Focus Recognition in Question Answering Systems
Question Focus Recognition in Question  Answering Systems Question Focus Recognition in Question  Answering Systems
Question Focus Recognition in Question Answering Systems Waheeb Ahmed
 
Epistemic networks for Epistemic Commitments
Epistemic networks for Epistemic CommitmentsEpistemic networks for Epistemic Commitments
Epistemic networks for Epistemic CommitmentsSimon Knight
 
Text mining introduction-1
Text mining   introduction-1Text mining   introduction-1
Text mining introduction-1Sumit Sony
 
Using topic modelling frameworks for NLP and semantic search
Using topic modelling frameworks for NLP and semantic searchUsing topic modelling frameworks for NLP and semantic search
Using topic modelling frameworks for NLP and semantic searchDawn Anderson MSc DigM
 

Semelhante a Why Watson Won: A cognitive perspective (20)

Ibm cognitive seminar march 2015 watsonsim final
Ibm cognitive seminar march 2015  watsonsim finalIbm cognitive seminar march 2015  watsonsim final
Ibm cognitive seminar march 2015 watsonsim final
 
Eacl 2006 Pedersen
Eacl 2006 PedersenEacl 2006 Pedersen
Eacl 2006 Pedersen
 
Eurolan 2005 Pedersen
Eurolan 2005 PedersenEurolan 2005 Pedersen
Eurolan 2005 Pedersen
 
Using and learning phrases
Using and learning phrasesUsing and learning phrases
Using and learning phrases
 
Iulia Pasov, Sixt. Trends in sentiment analysis. The entire history from rule...
Iulia Pasov, Sixt. Trends in sentiment analysis. The entire history from rule...Iulia Pasov, Sixt. Trends in sentiment analysis. The entire history from rule...
Iulia Pasov, Sixt. Trends in sentiment analysis. The entire history from rule...
 
Introduction to Natural Language Processing
Introduction to Natural Language ProcessingIntroduction to Natural Language Processing
Introduction to Natural Language Processing
 
Text Analytics for Semantic Computing
Text Analytics for Semantic ComputingText Analytics for Semantic Computing
Text Analytics for Semantic Computing
 
NLP & DBpedia
 NLP & DBpedia NLP & DBpedia
NLP & DBpedia
 
Extracting Relevant Questions to an RDF Dataset Using Formal Concept Analysis
Extracting Relevant Questions to an RDF Dataset Using Formal Concept AnalysisExtracting Relevant Questions to an RDF Dataset Using Formal Concept Analysis
Extracting Relevant Questions to an RDF Dataset Using Formal Concept Analysis
 
Factoid based natural language question generation system
Factoid based natural language question generation systemFactoid based natural language question generation system
Factoid based natural language question generation system
 
Web & text mining lecture10
Web & text mining lecture10Web & text mining lecture10
Web & text mining lecture10
 
Relation Extraction
Relation ExtractionRelation Extraction
Relation Extraction
 
Aaai 2006 Pedersen
Aaai 2006 PedersenAaai 2006 Pedersen
Aaai 2006 Pedersen
 
Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A compreh...
Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A compreh...Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A compreh...
Semantic Integration of Citizen Sensor Data and Multilevel Sensing: A compreh...
 
Knowledge acquisition using automated techniques
Knowledge acquisition using automated techniquesKnowledge acquisition using automated techniques
Knowledge acquisition using automated techniques
 
Question Focus Recognition in Question Answering Systems
Question Focus Recognition in Question  Answering Systems Question Focus Recognition in Question  Answering Systems
Question Focus Recognition in Question Answering Systems
 
Epistemic networks for Epistemic Commitments
Epistemic networks for Epistemic CommitmentsEpistemic networks for Epistemic Commitments
Epistemic networks for Epistemic Commitments
 
Text mining introduction-1
Text mining   introduction-1Text mining   introduction-1
Text mining introduction-1
 
What is AI ML NLP and how to apply them
What is AI ML NLP and how to apply themWhat is AI ML NLP and how to apply them
What is AI ML NLP and how to apply them
 
Using topic modelling frameworks for NLP and semantic search
Using topic modelling frameworks for NLP and semantic searchUsing topic modelling frameworks for NLP and semantic search
Using topic modelling frameworks for NLP and semantic search
 

Mais de James Hendler

Knowing what AI Systems Don't know and Why it matters
Knowing what AI  Systems Don't know and Why it mattersKnowing what AI  Systems Don't know and Why it matters
Knowing what AI Systems Don't know and Why it mattersJames Hendler
 
Exploring the Boundaries of Artificial Intelligence (or "Modern AI")
Exploring the Boundaries of Artificial Intelligence (or "Modern AI")Exploring the Boundaries of Artificial Intelligence (or "Modern AI")
Exploring the Boundaries of Artificial Intelligence (or "Modern AI")James Hendler
 
Tragedy of the (Data) Commons
Tragedy of the (Data) CommonsTragedy of the (Data) Commons
Tragedy of the (Data) CommonsJames Hendler
 
Knowledge Graph Semantics/Interoperability
Knowledge Graph Semantics/InteroperabilityKnowledge Graph Semantics/Interoperability
Knowledge Graph Semantics/InteroperabilityJames Hendler
 
Enhancing Precision Wellness with Personal Health Knowledge Graphs
Enhancing Precision Wellness with Personal Health Knowledge Graphs Enhancing Precision Wellness with Personal Health Knowledge Graphs
Enhancing Precision Wellness with Personal Health Knowledge Graphs James Hendler
 
Capacity Building: Data Science in the University At Rensselaer Polytechnic ...
Capacity Building: Data Science in the University  At Rensselaer Polytechnic ...Capacity Building: Data Science in the University  At Rensselaer Polytechnic ...
Capacity Building: Data Science in the University At Rensselaer Polytechnic ...James Hendler
 
Enhancing Precision Wellness with Knowledge Graphs and Semantic Analytics: O...
Enhancing Precision Wellness with  Knowledge Graphs and Semantic Analytics: O...Enhancing Precision Wellness with  Knowledge Graphs and Semantic Analytics: O...
Enhancing Precision Wellness with Knowledge Graphs and Semantic Analytics: O...James Hendler
 
Artificial Intelligence: Existential Threat or Our Best Hope for the Future?
Artificial Intelligence: Existential Threat or Our Best Hope for the Future?Artificial Intelligence: Existential Threat or Our Best Hope for the Future?
Artificial Intelligence: Existential Threat or Our Best Hope for the Future?James Hendler
 
The Science of Data Science
The Science of Data Science The Science of Data Science
The Science of Data Science James Hendler
 
Big Data and Computer Science Education
Big Data and Computer Science EducationBig Data and Computer Science Education
Big Data and Computer Science EducationJames Hendler
 
The Rensselaer IDEA: Data Exploration
The Rensselaer IDEA: Data Exploration The Rensselaer IDEA: Data Exploration
The Rensselaer IDEA: Data Exploration James Hendler
 
Future of the World WIde Web (India)
Future of the World WIde Web (India)Future of the World WIde Web (India)
Future of the World WIde Web (India)James Hendler
 

Mais de James Hendler (12)

Knowing what AI Systems Don't know and Why it matters
Knowing what AI  Systems Don't know and Why it mattersKnowing what AI  Systems Don't know and Why it matters
Knowing what AI Systems Don't know and Why it matters
 
Exploring the Boundaries of Artificial Intelligence (or "Modern AI")
Exploring the Boundaries of Artificial Intelligence (or "Modern AI")Exploring the Boundaries of Artificial Intelligence (or "Modern AI")
Exploring the Boundaries of Artificial Intelligence (or "Modern AI")
 
Tragedy of the (Data) Commons
Tragedy of the (Data) CommonsTragedy of the (Data) Commons
Tragedy of the (Data) Commons
 
Knowledge Graph Semantics/Interoperability
Knowledge Graph Semantics/InteroperabilityKnowledge Graph Semantics/Interoperability
Knowledge Graph Semantics/Interoperability
 
Enhancing Precision Wellness with Personal Health Knowledge Graphs
Enhancing Precision Wellness with Personal Health Knowledge Graphs Enhancing Precision Wellness with Personal Health Knowledge Graphs
Enhancing Precision Wellness with Personal Health Knowledge Graphs
 
Capacity Building: Data Science in the University At Rensselaer Polytechnic ...
Capacity Building: Data Science in the University  At Rensselaer Polytechnic ...Capacity Building: Data Science in the University  At Rensselaer Polytechnic ...
Capacity Building: Data Science in the University At Rensselaer Polytechnic ...
 
Enhancing Precision Wellness with Knowledge Graphs and Semantic Analytics: O...
Enhancing Precision Wellness with  Knowledge Graphs and Semantic Analytics: O...Enhancing Precision Wellness with  Knowledge Graphs and Semantic Analytics: O...
Enhancing Precision Wellness with Knowledge Graphs and Semantic Analytics: O...
 
Artificial Intelligence: Existential Threat or Our Best Hope for the Future?
Artificial Intelligence: Existential Threat or Our Best Hope for the Future?Artificial Intelligence: Existential Threat or Our Best Hope for the Future?
Artificial Intelligence: Existential Threat or Our Best Hope for the Future?
 
The Science of Data Science
The Science of Data Science The Science of Data Science
The Science of Data Science
 
Big Data and Computer Science Education
Big Data and Computer Science EducationBig Data and Computer Science Education
Big Data and Computer Science Education
 
The Rensselaer IDEA: Data Exploration
The Rensselaer IDEA: Data Exploration The Rensselaer IDEA: Data Exploration
The Rensselaer IDEA: Data Exploration
 
Future of the World WIde Web (India)
Future of the World WIde Web (India)Future of the World WIde Web (India)
Future of the World WIde Web (India)
 

Último

Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 

Último (20)

Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 

Why Watson Won: A cognitive perspective

  • 1. Tetherless World Constellation Why Watson Won: A cognitive perspective Jim Hendler and Simon Ellis Tetherless World Professor of Computer,Web and Cognitive Sciences Director, Rensselaer Institute for Data Exploration and Applications Rensselaer Polytechnic Institute (RPI) http://www.cs.rpi.edu/~hendler @jahendler (twitter)
  • 3. How’d I get into it? Watson and Semantic Web Tetherless World Constellation IBM
  • 4. Watson and Semantic Web Tetherless World Constellation IBM
  • 5. Is Watson cognitive? ??? “The computer‟s techniques for unraveling Jeopardy! clues sounded just like mine. That machine zeroes in on key words in a clue, then combs its memory (in Watson‟s case, a 15-terabyte data bank of human knowledge) for clusters of associations with those words. It rigorously checks the top hits against all the contextual information it can muster: the category name; the kind of answer being sought; the time, place, and gender hinted at in the clue; and so on. And when it feels „sure‟ enough, it decides to buzz. This is all an instant, intuitive process for a human Jeopardy! player, but I felt convinced that under the hood my brain was doing more or less the same thing.” — Ken Jennings
  • 6. Outline Tetherless World Constellation • Is Ken right? – How Watson Works – Watson as a cognitive architecture?? – Beyond Watson
  • 7. Inside Watson ??? Watson pipeline as published by IBM; see IBM J Res & Dev 56 (3/4), May/July 2012, p.
  • 9. ??? Question analysis  What is the question asking for?  Which terms in the question refer to the answer?  Given any natural language question, how can Watson accurately discover this information? Who is the president of Rensselaer Polytechnic Institute? Question Analysis Focus Terms: “Who”, “president of Rensselaer Polytechnic Institute” Answer Types: Person, President
  • 10. Parsing and semantic analysis  What information about a previously unseen piece of English text can Watson determine?  ??? How is this information useful? Natural Language Parsing Semantic Analysis - grammatical structure - meanings of words, phrases, etc. - parts of speech - synonyms, entailment - relationships between words - hypernyms, hyponyms - ...etc. - ...etc.
  • 11. Question analysis pipeline Unstructured Question Text Parsing & Semantic Analysis Structured Annotations of Question: Focus, answer types, Useful search queries Machine Learning Classifiers ???
  • 12. Search Result Processing and Candidate ??? Generation
  • 13. Primary Search ???  Primary Search is used to generate the corpus of information from which to take candidate answers, passages, supporting evidence, and essentially all textual input to the system  It formulates queries based on the results of Question Analysis  These queries are passed into a (cached) search engine which returns a set number of highly relevant documents and their ranks.  on the open Web this could be a regular search engine (our
  • 14. Candidate Generation ???  Candidate Generation generates a wide net of possible answers for the question from each document.  Using each document, and the passages created by Search Result Processing, we generate candidates using three techniques:  Title of Document (T.O.D.): Adds the title of the document as a candidate.  Wikipedia Title Candidate Generation: Adds any noun phrases within the document‟s passage texts that are also the titles of Wikipedia articles.  Anchor Text Candidate Generation: Adds candidates based on the hyperlinks and metadata within the document.
  • 15. Search Result Processing and Candidate Generation ???
  • 17. Scoring  Analyzes how well a candidate answer relates to the question  Two basic types of scoring algorithm  Context-independent scoring  Context-dependent scoring ???
  • 18. Types of scorers  Context-independent  Question Analysis  Ontologies (DBpedia, YAGO, etc)   ??? Type hierarchy reasoning Context-dependent  Analyzes feature of the natural language environment where candidates were found   Relies on “passages” found during search Many special purpose ones used in Jeopardy
  • 19. Scorers  Passage Term Match  Textual Alignment  ??? Skip-Bigram  Each of these scores supportive evidence  These scores are then merged to produce a single candidate score
  • 20. Example:Textual Alignment ???  Finds an optimal alignment of a question and a passage  Assigns “partial credit” for close matches    “Who is the President of RPI?” Who President of RPI. Shirley Ann Jackson is the President of RPI.
  • 21. Skip-Bigram  Constructs a graph  Nodes represent terms (syntactic objects)   ??? Edges represent relations Extracts skip-bigrams    A skip-bigram is a pair of nodes either directly connected or which have only one intermediate node Skip-bigrams represent close relationships between terms Scores based on number of common skip-bigrams
  • 22. Example  Who authored “The Good Earth”?  “Pearl Buck, author of the good earth…” ???
  • 23. Watson Summary Tetherless World Constellation • Watson works by – Analyzing the question • natural language parsing • text extraction – Generating a large number of candidates • mostly search heuristics – Scoring each • through multiple scorers • with weights adjusted by learning algorithm – Returning top candidate
  • 24. MiniDeepQA (Not Watson!)  ??? RPI students implementing a DeepQA pipeline to explore the principles underlying this kind of Q/A system (THIS IS NOT WATSON!)  Pipeline development  Data caching  Graphical and command line interfaces  Parsing  Scoring
  • 29. ??? Examples had to get this one right!
  • 30. Scoring ???  One of DeepQA‟s main strengths is aggregating a number of different scoring algorithms capable of running in parallel.  RPI scorers are primitive compared to IBM‟s, but  allow us to explore the principles  allow us to explore different algorithms for computing scores  allow us to create new ones not tried by IBM
  • 31. Scoring Principles: combine evidence  He was the Prime Minister of Canada in 1993.   candidates could include Trudeau, Harper, Campbell, Chretien, Mulroney… Try (Research):  Trudeau was Prime Minister of Canada in 1993 (doesn‟t match)  Campbell was Prime Minister of Canada in 1993 (MATCH)  Chretien was Prime Minister of Canada in 1993 (MATCH)  Scoring Research & type match  Trudeau: Re-search NO; Type: Yes  Campbell: Re-search YES; Type: No  Chretien: Re-search YES; Type: Yes WHO WAS CHRETIEN? ???
  • 32. New Scoring types  ??? We can explore how new kinds of information can be added to the Watson scoring pipeline  Example: new NLP extraction techniques   Example: Specialized Web Sources   Adding a ML-based extractor built by Heng Ji Database advisor project Example: More complex inferencing  Jeopardy questions are unambiguous, real world questions aren‟t • •  Where is Montreal? Who is Jim Hendler? Example: Special purpose reasoning…
  • 33. Special purpose reasoning ??? • Can we match simulation (or steer) large scale simulations to help answer NL questions? - eg. Answer questions such as “Why” and “How” integrated with large scale simulations
  • 34. Alternate Universe Reasoning (Contexts)  How can a Watson reasoner appropriately use Q/A contexts?  Where was Yoda born?   Very little is known about Yoda's early life. He was from a remote planet, but which one remains a mystery. Where was Yoda made?   designed and built by Stuart Freeborn   The Yoda puppet was originally for LucasFilm and Industrial Light & Magic. Where did Yoda live?   Jedi Master Yoda went into voluntary exile on Dagobah Where did Yoda live in the Phantom Menace? ???
  • 35. But back to the original question Tetherless World Constellation • Q: How does Watson fare as a cognitive model? • A: Poorly – no conversational ability – no concept of self – no deeper reasoning … • Q: How does Watson fare as a model of question answering?
  • 36. Watson and Q/A Tetherless World Constellation • Watson’s feed-forward pipeline has the following properties – lots of candidates generated • the more the better – “ad hoc” filtering pipelines • domain independent usually score lower than domain dependent – no “counter-reasoning” between answers • separately scored, only comparison is numbers
  • 37. Production rules, modules, etc Tetherless World Constellation Production Rule style Architectures cf ACT-R (Anderson 1974; …2012) - modularization, but not Watson style - parallelization, but in rule productions (procedural memory) - declarative memory is fact based Watson is not well correlated, except for using search for declarative memory
  • 38. Network based Tetherless World Constellation Network based architectures (cf. spreading activation (Collins 75), marker-passing (Hendler 86) … Microsaint 2006) - positive activations - inhibitory nodes (or other negative enforcers) Watson has no negative inhibition, does use network-based scorers
  • 39. MAC/FAC Tetherless World Constellation MAC/FAC (Gentner & Forbus, 1991) Many are chosen, few are called model of analogic reasoning Strong correspondence in performance, not in mechanism New work by Forbus (SME) uses a more feed-forward mechanism (Discussions in progress)
  • 40. Cognitive Architecture? Watson as “component” Decision Making Memory Reasoning Watson, Cogito, and Clarion Office of Research
  • 41. Summary Tetherless World Constellation • Watson won by a combination of – – – – – natural language processing search technologies semantic typing (minimal reasoning) scoring heuristics machine learning (scorer tuning) • Watson Q/A has some interesting analogies to cognitive architectures of the past – but mainly at a “level of abstraction” • Watson as a memory component in a more complex cognitive system is a very intriguing possibility