Exploring the Future Potential of AI-Enabled Smartphone Processors
Â
What Business Innovators Need to Know about Sentiment Analysis, Claire Cardie
1. What Business Innovators Need to
Know about Sentiment Analysis
Claire Cardie
Department of Computer Science
Chair, Information Science Department
Cornell University
Co-founder
Chief Scientist
2. Plan for the Talk
Subjectivity and sentiment in language
Continuum of capabilities
â Surface-level in-depth understanding
â Document-level phrase-level
Next stepsâŠ
3. Subjective Language
Subjective text expresses speculations,
beliefs, emotions, evaluations, goals,
opinions, judgments, âŠ
âą Jill said, "I hate Bill."
âą John thought about whom to vote for.
âą Seth knew his symposium would go well.
4. Subjectivity vs. Sentiment
Sentiment-bearing text expresses
positive and negative speculations, beliefs,
emotions, evaluations, goals, opinions,
judgments,âŠ
âą Jill said, "I hate Bill." -
âą John thought about whom to vote for. ~
âą Seth knew his symposium would go well.
+
sentiment analysis tome [Pang & Lee, 2008]
5. A Word on Polarity (tone, valence)
Positive âI love NY.â
Negative âI hate NY.â
Neither positive nor negative
â Objective?
âI thought about NY.â
â Neutral?
âIâm ambivalent about NY.â
â Mixed polarity?
âSometimes I love NY; other times I hate it.â
6. And What About Intensity?
Strength/intensity
âI love NY.â
âI absolutely adore NY!â
â Low, medium, high, very high, extreme
â ratings
â rotten tomatoes
7. Plan for the Talk
Subjectivity and sentiment in language
Continuum of capabilities
â Surface-level in-depth understanding
â Document-level phrase-level
Next stepsâŠ
9. Identifying Tone of a Collection
Sentiment (w.r.t. a topic)
â Example: Tone on âeconomic stimulusâ
10. Detecting âchatterâ or âbuzzâ
Chatter (w.r.t. a topic)
â Example: Buzz on âeconomic stimulusâ
11. Keyword-based Approaches
Search the text for the presence of
specific terms from a manually created
âsentiment lexiconâ
â +: âgreatâ, âpraiseâ, âpeaceâ, âsuperbâ, âŠ
â -: âwarâ, âdullâ, âmessyâ, âcriticizeâ, âŠ
Sentiment is based on the counts
â E.g.,
If more positive terms than negative terms,
then return +,
else return â
12. Keyword-based Approaches
Complications
â Inherent ambiguities of languageâŠ
â This laptop is a great deal.
â A great deal of media attention surrounded the
release of the new laptop model.
â If you think this laptop is a great deal, Iâve got
a nice bridge for you to buy.
[Examples from Lillian Lee]
[Pang & Lee, 2008]
13. Machine-learning Approaches
Learn from training data
Are better able to take advantage of
context to disambiguate terms
examples
ML Algorithm
statistical model
(novel) examples class
(program)
14. Measuring Performance
Precision: #correct / #attempted
Recall: #correct / #possible
F-measure: harmonic mean of P and R
1. _______
P = 3 / 4 = .75
2. _______
P = 3 / 3 = 1.00
3. _______
R = 3 / 4 = .75
4. _______
accuracy
15. Measuring Performance
How well do document-level sentiment
analysis systems work?
It dependsâŠ
â Product reviews easier than Movie reviews,
easier than News/editorials
â Shorter documents harder than longer ones
â Messy documents harder than clean ones
~75 F - ~85 F
16. This is actually quite goodâŠ
Comparison is not vs. 100% P/RâŠbut vs.
human sentiment analysis accuracy
â Cohenâs kappa
Machine-learning methods for sentiment
analysis approach human agreement
levels
â ~85 F: for positive/negative
â ~75 F: when neutrals are included
17. Sentiment Analysis at Passage Level
Passage tone The suggestion that the White
House never took seriously an
â Optionally w.r.t. a issue that infuriated millions of
topic Americans was supported by
â E.g., AIG or Geithner Senator Robert Menendez, a
New Jersey Democrat who
claimed that several weeks
earlier he warned Timothy
Geithner, the Treasury
secretary, that AIG was
planning to use taxpayer funds
to pay out $165m in bonusesâŠ
speculation that Obama will
have to replace him, despite
the presidentâs insistence to
Leno that Geithner is doing "an
outstanding jobâ.
18. Sentiment Analysis at Phrase Level
Fine-grained opinion analysis
Identify who is saying what about what
19.
20. Fine-Grained Sentiment Extraction
The suggestion that the White House never took
seriously an issue that infuriated millions of Americans
was supported by Senator Robert Menendez, a New
Jersey Democrat who claimed that several weeks
earlier he warned Timothy Geithner, the Treasury
secretary, that AIG was planning to use taxpayer
funds to pay out $165m in bonuses⊠speculation that
Obama will have to replace him, despite the
presidentâs insistence to Leno that Geithner is doing
"an outstanding job".
21. Fine-Grained Sentiment Extraction
âŠthe president insisted to Leno that Geithner is doing "an
outstanding job".
â Opinion trigger
â Polarity Opinion Frame
â Intensity Polarity: positive
â Opinion holder Intensity: high
Opinion Holder: âthe presidentâ
â Target (topic) Target: âGeithnerâ
22. Example â fine-grained opinions
opinion frame
opinion frame
opinion frame
opinion frame opinion frame
opinion frame
The suggestion that the White House never took
seriously an issue that infuriated millions of
Americans was supported by Senator Robert
Menendez, a New Jersey Democrat who claimed
that several weeks earlier he warned Timothy
Geithner, the Treasury secretary, that AIG was
planning to use taxpayer funds to pay out $165m in
bonusesâŠthe president insisted to Leno that
Geithner is doing "an outstanding job".
opinion frame
24. Example â Opinion Summary
Summarize thoughts and views across
documents
â Critical addition: opinion holder
AIG
25. What makes this hard?
Same issues of ambiguity as before plusâŠ
Need to associate opinion with topic and
with opinion holder
Requires different machine learning
methods
Requires many language-processing
modules
26. Noun Phrase Coreference Resolution
The suggestion that the White House never took
seriously an issue that infuriated millions of Americans
was supported by Senator Robert Menendez, a New
Jersey Democrat who claimed that several weeks
earlier he warned Timothy Geithner, the Treasury
secretary, that AIG was planning to use taxpayer
funds to pay out $165m in bonusesâŠspeculation that
Obama will have to replace Geithner, despite
the presidentâs insistence to Leno that he is
doing "an outstanding job".
Ng & Cardie [2002, 2003]; Stoyanov & Cardie [2006, 2008]
28. Plan for the Talk
Subjectivity and sentiment in language
Continuum of capabilities
â Surface-level in-depth understanding
â Document-level phrase-level
Next stepsâŠ