The lecture was given in a Cognitive and Analytics workshop at Indian Institute of Management. Topics covered was -
1) Understanding Natural Language Processing, Classification, Watson & its modules
2) Industry applications of Cognitive Computing
3) Understanding Cognitive Architecture
4) Understanding the disciplines / tools being used in Cognitive Science
Introduction to Cognitive Computing the science behind and use of IBM Watson
1. Workshop on Cognitive and Advanced Analytics
Introduction to Cognitive
Computing, the science behind
and use of IBM Watson
AN INDUSTRY PERSPECTIVE
Subhendu Dey | Senior Solution Architect, Cognitive Business Solutions, IBM
PGDHRM | Indian Institute of Ranchi | August 26-29, 2016
2. Workshop on Cognitive and Advanced Analytics
What we want to
cover today
Understand Natural Language Processing, Classification,
Watson & its modules
Industry applications of Cognitive Computing
Understanding Cognitive Architecture
Understanding the disciplines / tools being used in Cognitive
Science
PGDHRM | Indian Institute of Ranchi | August 26-29, 20168/28/2016 2
3. Workshop on Cognitive and Advanced Analytics
Cognitive
computing is built
upon two main*
pillars of
computing
science -
Natural Language
Processing (NLP)
& Machine
Learning
In the world of cognitive computing, we expect the IT systems
to have one or more of the following capabilities
Understand both structured as well as unstructured content
Extract meaning out of it
Relate ingested information with own data scheme
Apply Reasoning capability
Learn with usage and
React accordingly
What is the difference between Cognitive Computing and
Artificial Intelligence? Are they same?
What all tasks from the above list leverage NLP?
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 3
4. Workshop on Cognitive and Advanced Analytics
Let us first
understand the
need of NLP
techniques with an
example
Question: In May
1898 Portugal
celebrated the 400th
anniversary of this
explorers’ arrival in
India. Who is he?
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 4
Celebrated
In May 1898
400th anniversary
Portugal
Arrival in
India
Explorer
In May, Gary arrived in India
after he celebrated his
anniversary in Portugal
Arrived in
Celebrated
In May
anniversary
In Portugal
India
Gary
Keyword matching
Keyword matching
Keyword matching
Keyword matching
Keyword matching
5. Workshop on Cognitive and Advanced Analytics
Let us first
understand the
need of NLP
techniques with an
example
Question: In May
1898 Portugal
celebrated the 400th
anniversary of this
explorers’ arrival in
India. Who is he?
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 5
• Search far and wide
• Explore many
hypothesis
• Find and rank evidence
Celebrated
In May 1898
400th anniversary
Portugal
Arrival in
India
Explorer
On the 27th of May, 1498
Vasco da Gama landed in
Kappad Beach
Landed in
27th May 1498
Kappad beach
Vasco da Gama
Geo-Spatial Reasoning
Statistical Paraphrasing
Temporal Reasoning
6. Workshop on Cognitive and Advanced Analytics
NLP as a science
has evolved over
time, however
there are still few
challenges, and
we have made
decent progress
in many.
Spam detection
Part of Speech
tagging –
identification of
nouns, verbs,
adjectives etc.
Named Entity
Recognition –
identification of
person, location,
organization etc.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 6
Sentiment
Analysis. E.g.
movie or book
review.
Co-reference
resolution – i.e.
mapping pronoun
to a noun
mentioned before.
Word sense
disambiguation
Machine
translation (MT)
Information
Extraction (IE)
Question and
Answering
Paraphrase
Summarization
Dialog
Can we think of some pre-processing of text that would be essential
for any of these analysis?
7. Workshop on Cognitive and Advanced Analytics
Techniques of
working with NLP
Following the Markov
Assumption, we can
convert chain rule to
unigram or bi-gram
model or tri-gram
model.
Regular Expressions (deterministic rules)
Similarity analysis – minimum edit distance (simple and
weighted), typically used for spell correction
N-gram model – calculating the probability of a sentence or
sequence of words. Typically very useful for
Understanding the quality of machine translation
Spell / word Correction in a sentence
Speech recognition
In mathematical terms it looks like
P(x1 , x2 ,…. xn ) = P(x1)P(x2 |x1) P(x3 |x1 , x2)…P(xn | x1 ,.., xn )
i.e.
Can we realistically calculate this?
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 7
8. Workshop on Cognitive and Advanced Analytics
Further usage of
NLP can be seen
in the areas of
classification of
text / document
Text classification is useful in the areas of
Assigning subject categories
Spam detection
Authorship identification
Age / gender identification
Language identification
Sentiment analysis
--- many more
Hand-written rules is the best as well as simplest but since
that is not scalable supervised machine learning is adopted
Various kinds of classifiers are
Naïve Bayes
Logistic Regression
Support Vector machine (SVM)
k-Nearest neighbors
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 8
9. Workshop on Cognitive and Advanced Analytics
Further usage of
NLP can be seen
in the areas of
classification of
text / document
Text classification is useful in the areas of
Assigning subject categories
Spam detection
Authorship identification
Age / gender identification
Language identification
Sentiment analysis
--- many more
Hand-written rules is the best as well as simplest but since
that is not scalable supervised machine learning is adopted
Various kinds of classifiers are
Naïve Bayes
Logistic Regression
Support Vector machine (SVM)
k-Nearest neighbors
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 9
To avoid problem of floating point underflow this is
practically used as
10. Workshop on Cognitive and Advanced Analytics
Sentiment
Analysis: as we
move to
unstructured
content beyond
the boundary of
the enterprise
(e.g. news) this
becomes all the
more important
Sentiment analysis is kind of text classification. It’s like
detection of attitude.
Holder of attitude (source)
Target of attitude (aspect)
Type of attitude (simple – positive | negative, complex – scoring)
Applicable for –
Product review (find the aspects attributes like ease of use, value etc.
and assign value to them)
Consumer confidence ( people have proved that twitter sentiment
correlates with polling result, i.e. public opinion )
Traits in twitter (e.g. calmness) has been proven to be predictor of
financial performance
Polarity analysis (positive | negative | neutral )
Again Naïve Bayes classification algorithm can be used, which is the
classifier algorithm we are covering today.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 10
11. Workshop on Cognitive and Advanced Analytics
Information
Extraction - basics
Regular expressions
Smart notes: create calendar event based on notes
NER
Often indexed for search
Sentiment can be associated to them
Typically done through Machine learned classifier algorithm,
however some of the entities are popularly identified through
exhaustive dictionary.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 11
12. Workshop on Cognitive and Advanced Analytics
Relationship
Extraction - basics
What relations to extract
Automated Content Extraction (ACE)
UMLS – unified medical language system, used for Biomedical Information
extraction
Database of Wikipedia relations (taken from Wikipedia info box), called DBpedia
Typically RDF triples (subject – predicate (i.e. relations) – object) – there are ~ 1B such
RDF triples
Ontological relation: is-a (hypernym) and / or instance-of and many others
How to build - Hand-written rules | Supervised machine learning | Semi-supervised /
un-supervised (bootstrapping, distant supervision, unsupervised learning from web)
Intuitions:
Use patterns to build relation (start with hand-written pattern, e.g. “such as”,
“including”, “especially” works for “IS-A” relation.
Start with NER and then extract relation, because a pair of named entities can
have a finite set of relations.
We can combine these two and make patterns (possible to be built from English
thesaurus)
However all these hand-written patterns are often have low recall though
may have high precision.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 12
13. Workshop on Cognitive and Advanced Analytics
Information
Retrieval:
The Feast or Famine model
does not work well most of
the times, calling for a
Ranked retrieval model
which is more realistic.
There are several techniques
in the Ranked retrieval.
Use of Jaccard Coefficient – however, this does not take into
account the term frequency, hence could be misleading
Even if we use term frequency, taking it in linear proportion is
not quite right.
So we use the term frequency as
This carries the problem of bag of words concept
Essentially the score becomes
General observation is rare terms are more significant
So we use document frequency (inverse) along with term frequency
Use of Cosine similarity
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 13
14. Workshop on Cognitive and Advanced Analytics
What we want to
cover today
Understand Natural Language Processing, Classification,
Watson & its modules
Industry applications of Cognitive Computing
Understanding Cognitive Architecture
Understanding the disciplines / tools being used in Cognitive
Science
PGDHRM | Indian Institute of Ranchi | August 26-29, 20168/28/2016 14
15. Workshop on Cognitive and Advanced Analytics
We can detect
Cognitive
Opportunities by
keeping top three
things in mind
These are the items those
are fundamentally
changing computing in
this era.
Intuitive / Value added / Pervasive System of Engagement – where the IT
component can be engaged more like human in natural means (natural
language, visual recognition, voice commands).
Mine huge set of structured and/or unstructured data to generate
hypothesis – essentially analyzing content in a reasonable time which is
humanly impossible. There could be two types of the mining of data
Industry / Org specific content mining (e.g. mining of claims data for future insight)
or
Industry agnostic content mining (e.g. complaints / feedback analysis for
sentiment analysis)
Evidence based decisions instead of static rules – this depends on
supervised or unsupervised machine learning capability to derive decisions
based on the evidence. The evidences are obtained from the structured and
unstructured data analysis (as mentioned above).
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 15
16. Workshop on Cognitive and Advanced Analytics
The fundamental
differentiators
have generated
some business
patterns.
These are the
patterns of
applying cognitive
techniques.
Engagement - helps build stronger relationships with constituents.
Conversational agents of human-computer communication
Human-human communication is now mediated by computers
Pictures, Videos are acting as sensors to react to event
Discovery – helps create new insights by synthesizing information.
Discovery of knowledge (personality, sentiment etc.) from natural language text
Discovery of unknown risks / opportunities from unstructured (textual) notes
Decision – helps users make more informed, evidence based decision
Manual intervention based on evidence and not rules
Move away from one-size-fit all rules and continuous update
Policy – helps users evaluate compliance of a decision to policies
De-codification of long / complex policy documents to knowledge graphs
Check Adherence to the policy
Exploration – visually depict and analyze data for clear advice
Explore the exposure to certain commodity from annual reports
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 16
17. Workshop on Cognitive and Advanced Analytics
We expect
cognitive
technologies to
change the way
business is done
Direct-to-customer agents
Advisor-facing apps
Employee-facing apps
Conversational style of interaction using text &
speech
Create an individualized experience to make
personalized recommendations
Invoke transactions specific to the appropriate
business process
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 17
TRANSFORMED
INTERACTIONS
EMPOWERED
ADVISORS
Front-office applications allowing advisors to
readily find pertinent information for customer
interaction
Integration with backend systems to provide “total
view of customer”
Analytically-driven recommendations to improve
customer interactions
OPTIMIZED
OPERATIONS
Leverage experience & best practices for
improved decision-making
Access disparate systems to provide holistic view
of the risk
Shift time spent finding information to making
decisions & recommendations
18. Workshop on Cognitive and Advanced Analytics
Now, intelligent
machines simulate
human brain
capabilities to help
solve society’s most
vexing problems.
Cognitive computing
has indeed arrived,
and its potential to
transform industries
around the globe is
enormous.
To explore future opportunities and determine how
cognitive computing is already being utilized in
various industries, the IBM Institute for Business
Value conducted follow up research to its initial
research study.
Through a survey conducted by the Economist
Intelligence Unit, we gained insights from more than
800 executives from around the world in a variety of
industries, including healthcare, banking, insurance,
retail, government, telecommunications, life
sciences, consumer products, and oil and gas, and
from supplemental desk research and interviews
with subject matter experts across IBM.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 18
Find these reports @
http://www-935.ibm.com/services/us/gbs/thoughtleadership/cognitiveindustry/
19. Workshop on Cognitive and Advanced Analytics
Banking and
Financial Services
Industry:
67% of banking executives believe
personalization is driving customer
expectations
Almost $1 trillion lost in the
subprime mortgage crisis due to poor
credit decisions
89% of those familiar with cognitive
computing believe it will play a
disruptive role in the banking industry
The financial services industry is experiencing multiple challenges –
declining return on equity, expanding regulatory requirements, relentless
security threats, demanding customer requirements, and growing non-
traditional competition
At the same time, banks and other financial institutions are confronted with
an ever-expanding deluge of internal and external data that might help
redress challenges
Given constraints of traditional algorithm-based analytics and peoples’ ability
to process information, banks have been generally unable to exploit
maximum value from data
Cognitive computing expands the ability of computing exponentially,
unleashing an entirely new range of business opportunities:
Organizations are able to scale and accelerate human expertise in new, powerful
ways
People are able to make much better use of complex data
Bankers are able to leverage new insights to change behavior and transform their
organizations
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 19
Can we think of one use case from our Banking experience?
20. Workshop on Cognitive and Advanced Analytics
Communication
Industry:
71% of the CSP executives believe
that customers demand a
personalized experience today
49% of the CSP executives believe
that customers demand a more
seamless and consistent
experience
69% of the CSP executives are
actively pursuing industry model
innovation
46% of the CSP executives are
actively pursuing product and service
innovation
The communications industry is facing multiple forces of change - evolving
customer expectations, increasing OTT (over-the-top) services, rapid
increase of data-intensive apps, accelerated pressure on cost reduction and
higher privacy & security issues
To be successful, CSPs (communications service provider) need to develop
deep capabilities around engagement of customers and other
stakeholders, effective decision making, discovery of new products and
services, and management of profitability
These capabilities must be able to deal with both structured and
unstructured data, such as call center transcripts, and to include machine
learning in M2M communications
But many CSPs lack the analytical tools and other assets needed to be a
market leader. They struggle to meet customer expectations for seamless
and complete service, to provide innovative services and products, and to
make timely, accurate decisions
Cognitive computing can address these challenges and open up fresh
opportunities for CSPs by harnessing insights hidden in data from across the
organization and beyond.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 20
21. Workshop on Cognitive and Advanced Analytics
Consumer
Products Goods
Industry:
57% of consumer product leaders
believe they are not competent
enough in delivering personalized
experience across all touch points
The consumer products industry is facing multiple forces of change –
digitally empowered consumer, changing demographics, volatile commodity
prices, disruptive competition and changing regulations
To be successful, consumer products companies need to develop deep
capabilities around engagement of consumers and other stakeholders,
discovery of new ideas and effective decision making
But many of them lack the analytic tools and other assets needed to be a
market leader. Most of them struggle to meet consumer expectations for
engagement and personalization.
Cognitive computing can address these challenges. Example being:
Helps in improving supply chain and procurement decisions
Helps develop campaign messaging for specific targeting
Helps to discover unexpected flavors which were never thought to put together
before
Helps in predicting the demand for hottest products in the season
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 21
22. Workshop on Cognitive and Advanced Analytics
Healthcare
Industry:
Outperformers are 43% more
competent in consumer engagement
than underperformers
167% more outperformers make
innovation a major priority than
underperformers
44% more outperformers are strong in
decision making than underperformers
# 3 killer in USAis preventable
medical error in 2013
• The healthcare industry is undergoing significant change driven by six
disruptive forces - rapid digitization, changing consumer expectations,
regulatory complexities, increasing healthcare cost and demand, shortage
of skilled resources and elevating cost pressure
• To meet the implication of these forces, healthcare organizations must excel
in engaging with consumers, discovering new ideas and taking effective
decisions
• Currently, traditional analytics capabilities are unable to exploit maximum
value from the ever increasing data resource constraining organization’s
achievements and performance. But cognitive computing has the ability to
bridge this gap and can open up fresh opportunities for the healthcare
industry. It is already helping healthcare organizations to provide
personalized care, effective decisions and more innovative solutions.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 22
23. Workshop on Cognitive and Advanced Analytics
Insurance
Industry:
Outperformers are 65% more
competent in customer engagement
than underperformers
285% more outperformers make
innovation a major priority
66% more outperformers are strong in
decision making
98% of the insurance executives,
familiar with cognitive computing,
believe that it will play a disruptive role
in the insurance industry
The insurance industry is facing multiple forces of change - rapid
digitization, changing demographics, rising customer expectations,
challenging economic environment and expanding risk of sophisticated
fraud
To be successful, insurers need to develop deep capabilities around
engagement of customers and other stakeholders, effective decision
making, discovery of new ideas and management of profitability
But many insurers lack the analytic tools and other assets needed to be a
market leader. Many insurers struggle to meet customer expectations for
engagement and personalization, to calculate risk and profitability at a
granular level, and to make timely, accurate decisions
Cognitive computing can address these challenges and open up fresh
opportunities for insurers by creating machines that can
learn new problem domains
reason through the hypotheses
resolve ambiguity
evolve towards more accuracy and
interact in natural ways to engage discover and decide better
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 23
24. Workshop on Cognitive and Advanced Analytics
Pharma and Life
Science Industry:
145% increase in the cost of
developing a drug from 2004-2014
167% more outperformers make
innovation a major priority than
underperformers
Outperformers are 43% more
competent in consumer engagement
than underperformers
A new healthcare ecosystem is emerging in which life sciences
organizations will play a key role across the continuum from health and
wellness to preventative medicine.
The life sciences industry is facing multiple forces of change including
erosion of traditional industry boundaries; rapid digitization; continued
pressure on productivity as well as the need to prove the value of their
drugs.
To meet the implication of these forces, life sciences organizations must
excel in discovering new ideas, taking effective decisions and engaging
with payer and providers and most importantly, the patient.
Currently, traditional analytics capabilities are unable to exploit maximum
value from the ever increasing data resource constraining organization’s
achievements and performance. But cognitive computing has the ability –
in combinations with data driven analytics - to bridge this gap and can open
up fresh opportunities for the life sciences industry such as accelerating
discovery, modernizing clinical trials, transforming pharmacovigilance and
improving adherence.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 24
25. Workshop on Cognitive and Advanced Analytics
Retail Industry:
>2.5 PB of data is collected by
Walmart every hour from customer
transactions
69% of retail CXOs agree that
customers demand more personalized
experiences
59% of retail CXOs are actively
pursuing industry model innovation
More than half of retail executives are
expected to make big decisions in
strategic areas, in next 12 months
The retail industry is undergoing significant change driven by five
disruptive forces – expanding customer expectations, increasing self-serve
retail, rising technological advancements, falling margins and rising security
breaches
To meet the implication of these forces, retailers must excel in engaging with
customers, discovering new ideas and making effective decisions
Cognitive computing is helping face the challenge. Examples being –
Retailers use cognitive to understand shoppers behavior, search intent and guide
them with personalized advice and accurate recommendation
Retailers use cognitive in constructing 360°view of customers and finding
personality insights that helps them in designing effective campaigns
A major digital imaging company uses cognitive computing to understand
customer trend and help company in making concrete improvements in products
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 25
26. Workshop on Cognitive and Advanced Analytics
What we want to
cover today
Understand Natural Language Processing, Classification,
Watson & its modules
Industry applications of Cognitive Computing
Understanding Cognitive Architecture
Understanding the disciplines / tools being used in Cognitive
Science
PGDHRM | Indian Institute of Ranchi | August 26-29, 20168/28/2016 26
27. Workshop on Cognitive and Advanced Analytics
It is very difficult to
think of an
generalized
architecture for
cognitive
applications, as
the applicability
differs based on
the business
pattern
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 27
Security
Infrastructure (Exploratory Sandbox and Runtime Cluster)
Data (annotated, searchable, indexed)
Contentselection,
provisioning
Ingestion &
maintain
CognitiveVisualization (search, explore, feedback)
Crawl
Convert
Index
Content
Classification
Rule Based
Metadata
Maintenance
Metadata
Workflow
Relationship
and
Reasoning
Feedback&Learn
Content
Summary
Formation
Probabilistic
Model based
Ontology
Mapping
Temporal
Reasoning
Geospatial
Reasoning
Other
Reasoning
Text
Similarity
Analysis
Topic
Cluster
Content Search
and Retrieve
SearchAPI
Context
Management
Orchestration
Cognitive Exploration and Analysis
Integration(in/out
ofenterprise)
28. Workshop on Cognitive and Advanced Analytics
What we want to
cover today
Understand Natural Language Processing, Classification,
Watson & its modules
Industry applications of Cognitive Computing
Understanding Cognitive Architecture
Understanding the disciplines / tools being used in
Cognitive Science
PGDHRM | Indian Institute of Ranchi | August 26-29, 20168/28/2016 28
29. Workshop on Cognitive and Advanced Analytics
There are some
clear disciplines of
study / work in the
world of cognitive
computing
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 29
Typically if you are
from mathematics /
statistics
background. Or the
world of math/stat
excites you.
However, often
there is also
domain specific
knowledge
embedded.
Typically if you
are software
geek and
making of an
interconnected,
intelligent and
instrumented
world through
real life project
excites you.
In case you have a strong domain of
interest (may be because of your past
experience).
30. Workshop on Cognitive and Advanced Analytics
There are some
clear disciplines of
study / work in the
world of cognitive
computing
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 30
31. Workshop on Cognitive and Advanced Analytics
Tools from IBM.
The Watson suite
of products and
associated
services.
IBM Watson is a technology platform that uses natural language
processing and machine learning to reveal insights from large
amounts of unstructured data.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 31
Watson
Developer
Cloud
Watson DeveloperCloud enables developers and businesses of all sizes to build new cognitive
applications, and add cognitive capabilities to existing applications. It provides a growing
set of API’s and SDK’s, and is accessible to anyone through the Bluemix cloud environment.
Watson
Engagement
Advisor
Watson Engagement Advisor is a technology service that interacts with customers, listens to
questions and offers solutions. Engagement Advisor learns with every human interaction and
grows its collection of knowledge, quickly adapting to the way humans think.
Watson
Explorer
Watson Explorer is a technology platform that accesses and analyzes structured and
unstructured content. Explorer presents data, analytics and cognitive insights in a single view.
Explorer gives you the information you’re looking for while uncovering trends, patterns and
relationships.
Watson
Knowledge
Studio
Watson Knowledge Studio is a tool that enable subject matter experts and developers to teach
Watson the linguistic nuances of industries and knowledge domains.
Watson
Company
Analyzer
Watson Company Analyzer helps you reduce the time and effort to collect, digest and
synthesize information for building strategic business relationships
Watson
Ecosystem
A breakthrough partner program to join the tens of thousands of developers who are building
withWatson. From gaining insights from text to analyzing images and video, you can tap into
the power of Watson APIs to build cognitive apps.
32. Workshop on Cognitive and Advanced Analytics
It is very important
to understand the
implications of
regulatory
compliance and
economy of
choice before
investing over
tools and
technology
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 32
• Mostly Structured few
unstructured
• Comparatively low
volume
• More control on
Cognitive techniques
• Typically high precision
Data of
Interest
Internal
Data
External
Data
• Mostly unstructured /
semi-structured data
• Typically massive
infrastructure needed
except for a few cases.
• Typically high precision
• Overhead on data
masking and re-
formation.Typically
seen in Q&A type
solution.
• Typically high recall.
Data Residence
Restriction,
computing
platformOn-premise Cloud
• Look for Aggregator
APIs, e.g. News on a
company with Positive
Sentiment.
• Mostly machine
learned techniques,
easily scalable.
• Typically high recall.
33. Workshop on Cognitive and Advanced Analytics
Watson
DeveloperCloud
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 33
Relationship
Extraction
Questions
&
Answers
Language
Detection
Personality
Insights
Keyword
Extraction
Image Link
Extraction
Feed
Detection
Visual
Recognition
Concept
Expansion
Concept
Insights
Dialog
Sentimen
t Analysis
Text to
Speech
Tradeoff
Analytics
Natural
Language
Classifier
Author
Extraction
Speech to
Text
Retrieve
&
Rank
Watson
News
Language
Translatio
n
Entity
Extraction
Tone
Analyzer
Concept
Tagging
Taxonomy
Text
Extraction
Message
Resonance
Image
Tagging
Face
Detection
Answer
Generation
Usage
Insights
Fusion
Q&A
Video
Augmentation
Decision
Optimization
Knowledge
Graph
Risk
Stratification
Policy
Identification
Emotion
Analysis
Decision
Support
Criteria
Classification
Knowledge
Canvas
Easy
Adaptation
Knowledge
Studio
Service
Statistical
Dialog
Q&A
Qualification
Factoid
Pipeline
Case
Evaluation
The Waston that competed on
Jeopardy! in 2011 comprised what
is now a single API—Q&A—built
on five underlying technologies.
Since then, Watson has grown to
a family of 28 APIs.
By the end of 2016, there will
be nearly 50 Watson APIs—
with more added every year.
Natural Language
Processing
Machine Learning
Question Analysis
Feature Engineering
Ontology Analysis
34. Workshop on Cognitive and Advanced Analytics
Foundational
Technologies
behind Watson
Fifty (50) foundational
technologies draw upon
five (5) distinct field of
study:
Big Data & Analytics
Artificial Intelligence
Cognitive Experience
Cognitive Knowledge
Computing Infrastructure
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 34
AnaphoricCo-referencing Feature Engineering Learn to Rank Question Analysis
Colloquialism Processing Feature Normalization Linguistic Analysis
Question-answering
Reasoning Strategies
Content Management --
Versioning
Focus and Spurious Phrase
Resolution
Logical Reasoning Analysis
Recursive Neural
Networks
Convolutional Neural
Networks
HTML Page Analysis Logistical Regression Rules Processing
Curation Image Management Machine Learning Scalable Search
Deep Learning Information Retrieval
Multi-dimensional
Clustering
SimilarityAnalysis
Dialog Framing
Knowledge (Property)
Graphs
MultilingualTraining
Statistical Language
Parsing
Ellipses KnowledgeAnswering
N-gram analysis (word
combinations & distance)
SupportVector Machines
EmbeddedTable
Processing
Knowledge Extraction
Annotators
OntologyAnalysis SyllableAnalysis
Ensembles and Fusion
KnowledgeValidation and
Extrapolation
ParetoAnalysis TableAnswering
Entity Resolution Language Modeling Passage Answering VisualAnalysis
FactoidAnswering Latent Semantic Analysis
PDF Conversion Visual Rendering
Phoneme Aggregation Voice Synthesis
35. Workshop on Cognitive and Advanced Analytics
Alchemy Language
AlchemyLanguage is a
collection of APIs that
offer text analysis through
natural language
processing. The
AlchemyLanguageAPIs
can analyze text and help
you to understand its
sentiment, keywords,
entities, high-level
concepts and more.
Entity Extraction
Sentiment Analysis
Emotion Analysis
Keyword Extraction
Concept Tagging
Relation Extraction
Taxonomy Classification
Author Extraction
Language Detection
Linked Data Support
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 35
36. Workshop on Cognitive and Advanced Analytics
Conversation
Add a natural language
interface to your
application to automate
interactions with your
end users. Common
applications include
virtual agents and chat
bots that can integrate
and communicate on
any channel or device.
Watson combines a number of cognitive techniques to help
you build and train a bot - defining intents and entities and
crafting dialog to simulate conversation. The system can then
be further refined with supplementary technologies to make
the system more human-like or to give it a higher chance of
returning the right answer. Watson Conversation allows you
to deploy a range of bots via many channels, from simple,
narrowly focused Bots to much more sophisticated, full-blown
virtual agents across mobile devices, messaging platforms
like Slack, or even through a physical robot.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 36
37. Workshop on Cognitive and Advanced Analytics
Personality
Insights
Uncover a deeper
understanding of
people's personality
characteristics, needs,
and values to drive
personalization.
Personality Insights extracts and analyzes a spectrum of personality
attributes to help discover actionable insights about people and
entities, and in turn guides end users to highly personalized
interactions. The service outputs personality characteristics that are
divided into three dimensions: the Big 5, Values, and Needs. While
some services are contextually specific depending on the domain
model and content, Personality Insights only requires a minimum of
3500+ words of any text.
Some usages:
Targeted marketing
Customer acquisition
Customer care
Personal connections
Resume writing
……….
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 37
38. Workshop on Cognitive and Advanced Analytics
Visual
Recognition
Understand the
contents of images.
Create custom
classifiers to develop
smart applications.
Visual Recognition allows users to understand the contents of
an image or video frame, answering the question: “What is in
this image?”
Submit an image, and the service returns scores for relevant
classifiers representing things such as objects, events and
settings.
What types of images are relevant to your business? How
could you benefit from understanding and organizing those
images based on their contents?
With Visual Recognition, users can automatically identify
subjects and objects contained within the image and organize
and classify these images into logical categories.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 38
39. Workshop on Cognitive and Advanced Analytics
Alchemy Data
News
AlchemyData provides
news and blog content
enriched with natural
language processing to
allow for highly targeted
search and trend
analysis. Now you can
query the world's news
sources and blogs like
a database.
AlchemyData News indexes 250k to 300k English language
news and blog articles every day with historical search
available for the past 60 days.
One can query the News API directly with no need to acquire,
enrich and store the data yourself - enabling you to go
beyond simple keyword-based searches.
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 39
40. Workshop on Cognitive and Advanced Analytics
Tradeoff Analytics
Helps users make
better choices to best
meet multiple
conflicting goals.
Tradeoff Analytics is a service that helps people make decisions
when balancing multiple objectives. The service uses a
mathematical filtering technique called “Pareto Optimization,” that
enables users to explore tradeoffs when considering multiple criteria
for a single decision.
It can help bank analysts or wealth managers select the best
investment strategy based on performance attributes, risk, and cost.
It can help consumers purchase the product that best matches their
preferences based on attributes like features, price, or warranties.
Additionally, Tradeoff Analytics can help physicians find the most
suitable treatment based on multiple criteria such as success rate,
effectiveness, or adverse effects
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 40
41. Workshop on Cognitive and Advanced Analytics
Questions
8/28/2016 PGDHRM | Indian Institute of Ranchi | August 26-29, 2016 41