Mais conteúdo relacionado
Semelhante a Converseon 2012 CASRO Technology Conference (20)
Converseon 2012 CASRO Technology Conference
- 1. Distilling Actionable
Insights from the Deluge of
Social Media Data
Jasper Snyder
VP, Converseon
© 2012 Converseon Inc. Proprietary and Confidential
- 2. From Data Deluge to Insights
© 2012 Converseon Inc. Proprietary and Confidential 2
- 3. The vast scope of social media data available today
requires scalable tech solutions. Human-machine
collaboration is the only way to deal with this deluge.
Social Media Channel Approx. Monthly Volume Furthermore…
On-site comments and
Blogs 30 million new posts social cues and sharing
Social cues (e.g., “likes”)
Facebook 1.8 billion status updates and comments
Social cues like favoriting
Twitter 4 billion tweets and flagging other users
240 years of video
YouTube 400 million social actions content uploaded each
month
© 2012 Converseon Inc. Proprietary and Confidential 3
- 4. Social-media research can support both traditional
market research goals and PR use cases.
Traditional Market Research Communications Functions through Social
through Social Media Listening Media Monitoring
• Consumer • Consumer complaints and
Segmentation product malfunctions
• Purchase triggers • Adverse reactions for
pharmaceutical companies
• Thoughts and opinions
about products and • Crisis monitoring and
brands response
• Market awareness of • Reputation management
products or brands
© 2012 Converseon Inc. Proprietary and Confidential 4
- 5. These two use cases – market research and
communications – closely align with two services.
Social Listening Social Media Monitoring
When what matters most is When what matters most is delivering
understanding a consumer segment or customer service, navigating a crisis
market. situation or detecting reputation threats.
Goal is to acquire just enough data to Goal is comprehensive, real time
understand a population “out there” in coverage.
the world.
Higher tolerance for missing content. Higher tolerance for irrelevant content.
Lower tolerance for irrelevant content. Lower tolerance for missing content.
© 2012 Converseon Inc. Proprietary and Confidential 5
- 6. The Social Media Research Process: From Raw Data to
Insights
1. Data 2. Data
Collection Enrichment
3. Analysis
& Insight
Generation
© 2012 Converseon Inc. Proprietary and Confidential 6
- 7. Stage 1: Social Data Collection
Primary Goal:
Identify and acquire the data
1. Data 2. Data
that can answer your business
Collection Enrichment
questions.
Primary Challenges:
1. Pull in relevant data and
3. Analysis & metadata
Insight
Generation 2. Coverage of appropriate social
media channels
3. Eliminate spam and irrelevant
content.
© 2012 Converseon Inc. Proprietary and Confidential 7
- 8. Stage 2: Data Enrichment
Primary Goal:
2. Data Implement document- and sub-
1. Data Collection Enrichment document-level enrichments like
topic, consumer segment,
emotion and sentiment.
Primary Challenges:
3. Analysis & 1. Data normalization
Insight
Generation
2. Classification
3. Scalability
© 2012 Converseon Inc. Proprietary and Confidential 8
- 9. Stage 3: Analysis & Insight Generation
Primary Goal:
2. Data Connect the dots between a
1. Data Collection Enrichment suite of metrics and data points
in order to reach sound strategic
conclusions.
Primary Challenges:
3. Analysis 1. Reliability
& Insight
Generation 2. Strategic Value
© 2012 Converseon Inc. Proprietary and Confidential 9
- 10. Social media is a massive compendium of documents…
© 2012 Converseon Inc. Proprietary and Confidential 10
- 11. Harvesting Data and Metadata from Social Media
Documents: A Tweet Dissected
© 2012 Converseon Inc. Proprietary and Confidential 11
- 12. Harvesting Data and Metadata from Social Media
Documents: A Tweet Dissected
Datapoints:
• Author Name
• Text
• Publication Date
• Some hashtags
© 2012 Converseon Inc. Proprietary and Confidential 12
- 13. Harvesting Data and Metadata from Social Media
Documents: A Tweet Dissected
Metadata:
• Person or tweet that a
tweet is in reply to
• Follower count of author
• Times retweeted
• Times favorited
• Author description
© 2012 Converseon Inc. Proprietary and Confidential 13
- 14. Sorting Social Metadata
A
B
C
Tweets that contain
#Ford in the text.
© 2012 Converseon Inc. Proprietary and Confidential 14
- 15. Relevancy as a Sorting Task…
Irrelevant Documents
All Social Media Documents • Spam
• Documents not
in target
All Documents language (e.g.,
not English)
Containing Your
Boolean Query • Contain
keyword but not
relevant to
client question
Relevant
Documents
© 2012 Converseon Inc. Proprietary and Confidential 15
- 16. Data Enrichment: What Should We Measure?
Metric Explanation
Sentiment Does the author make a negative or positive
point about a product or brand?
Topics What topic is the author talking about the
product or brand in relation to?
Purchase Stage Has the author of a document already
purchased the product when writing about it
online?
Consumer Segmentation What segment is the document’s author
from?
Emotions What emotions do authors express toward
the target brand or product?
© 2012 Converseon Inc. Proprietary and Confidential 16
- 17. Data Enrichment: What Should We Measure?
Metric Sorting Categories
Sentiment Positive, negative, neutral
Topics Pre-selected topic and unexpected topics
Purchase Stage Before making a purchase or after.
Consumer Segmentation Young male, middle-aged woman, etc.
Emotions Joy, anticipation, surprise, fear, etc.
© 2012 Converseon Inc. Proprietary and Confidential 17
- 18. How can we implement the sorting tasks we’ve
discussed so far?
Machine Sorters Human Sorters
Sorting Tasks
© 2012 Converseon Inc. Proprietary and Confidential 18
- 19. Q: How do you know when a computer is correct?
A: The same way you know that a human is correct:
“I know it when I see it…”
© 2012 Converseon Inc. Proprietary and Confidential 19
- 20. Establishing A Basis for How Well Humans Agree With
One Another
Example 1: Inter-Coder Agreement on Sentiment Example 2: Inter-Coder Agreement on Emotion
Item Coder 1 Coder 2 Tweet Coder 1 Coder 2
I do not like the cats with Disgust Anger
1 Positive Positive thumbs “advert”
2 Positive Neutral I say that video is real, Trust No Emotion
definitely. Expressed
3 Neutral Neutral
4 Negative Positive
etc. … …
© 2012 Converseon Inc. Proprietary and Confidential 20
- 21. Using Human Parallel Coding to Establish Gold
Standards
Confusion Matrix: Human as Gold Standard
POSITIVE NEGATIVE NEUTRAL TOTAL
POSITIVE 365 24 159 548
NEGATIVE 57 81 65 203 Raw Accuracy:
61.5%
NEUTRAL 274 60 415 749
TOTAL 696 165 639 1500
© 2012 Converseon Inc. Proprietary and Confidential 21
- 22. Using A Credit Matrix to Create Improved Measurement
Credit Matrix
POSITIVE NEGATIVE NEUTRAL
POSITIVE 100% 0% 50%
NEGATIVE 0% 100% 50%
NEUTRAL 50% 50% 100%
Partial Credit Figure of Merit:
82.3%
Confusion Matrix: Human 1 as Gold Standard
POSITIVE NEGATIVE NEUTRAL
POSITIVE 365 24 159
NEGATIVE 57 81 65
NEUTRAL 274 60 415
© 2012 Converseon Inc. Proprietary and Confidential 22
- 23. But how does the machine learn?
1. Collection of Human 2. Machine ingests coded data 3. Machine applies model from
and finds patterns in each
Annotated Data category classification step two on raw data. Results
are compared to human
coding of same material.
© 2012 Converseon Inc. Proprietary and Confidential 23
- 25. Thank You!
Jasper Snyder,
VP, Converseon
jsnyder@converseon.com
Converseon Inc.
53 West 36th Street, 8th Floor,
New York, NY 10018
t: 212.213.4279 | f: 646.304.2364
www.converseon.com
25
© 2012 Converseon Inc. Proprietary and Confidential