Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Open Analytics: Building Effective Frameworks for Social Media Analysis
1. Open Analytics NYC – 11/08/2012
Building Effective Frameworks
for Social Media Analysis
2. Agenda
• Social Media: An Intelligence perspective
• Common Analytic Pitfalls
• An Analytic Framework
• Case Study: Brand Management
– Problem Definition
– Source Selection
– Data Capture
– Data Reporting
– Data Analysis
• Ways Forward, Future Analysis
• Questions?
3. Intelligence
• Intelligence is information that has been
transformed to meet an operational need
Data Intelligence
Operational Lens
4. Intelligence Cycle
• No matter what methodology you use…
Collect
Distribute Store
Analyze
intelligence analysis is an iterative process.
5. Social Media: Intelligence Perspective
• Social Media Intelligence is a combination of the
best and worst features of:
– HUMINT
– OSINT
– SIGINT HUMINT
OSINT SIGINT
6. Social Media Analysis Goals
• Provide value to the organization – turn data into
intelligence using an “operational lens”
• Ensure cyclical feedback occurs during collection,
processing, analysis, and consumption
• Validate that a particular network is the right source
of data for the questions you need answered
7. Common Misconceptions
• Social media is not a panacea
– Not everyone uses social media
– Users of social media use it unevenly
– User behavior changes based on situations
• Just because people can talk about anything does
not mean they talk about everything all the time.
8. Common Pitfalls
• Analyzing What Instead of Why:
The important thing is often not what people are
saying… but why they are saying it.
• Using the Wrong Analysis Tools:
Reporting tools rarely help dig into the why. Many
common tools, reports, and metrics are actually
misleading:
– Word clouds atomize message context
– Sentiment metrics are often highly inaccurate
– Information in aggregate hides more than it reveals
11. Dangers of Disintegration
Source: Matthew Auer, Policy Studies Journal,
Volume 39, Issue 4, pages 709–736, Nov 2011
12. Analytic Framework
• Data Capture (DC) Capture
• Data Reporting (DR)
• Data Analysis (DA)
– What to measure Analyze Report
– What the data is saying
– What should be done based on the data
Source: Avinash Kaushik, Occam’s Razor Blog http://www.kaushik.net/avinash/web-analytics-consulting-
framework-smarter-decisions/
13. Choosing a Platform
• Social media, and the ways that it is used, is
relatively new and evolving rapidly:
– Static approaches to social media are flawed from
the outset
– No one metric or set of metrics will always let you
know what is happening
• Platforms need to be open and highly
adaptable to facilitate data capture, reporting,
and analysis
14. Case Study: Brand Management
• Industry: Gaming
– Experiencing 10% growth annually
– Overall revenue expected to exceed $80 billion by
2014
• In May, Zenimax Online Studios announced
Elder Scrolls Online
– Elder Scrolls V: Skyrim 2nd largest game of 2011
15. Problem Definition
• Question: How can brand managers use social
media to track and understand public
attitudes toward a product?
• Challenge: Capture relevant information for
social media sources.
– Query too large = false positives
– Query too small = miss potential information
16. Twitter
• Twitter has excellent analytical potential:
– Enormous volume, 400 million+ tweets per day
– Large user base, 140 million+ accounts
– Open API
• But its not without its limitations:
– 140 characters
– Limited historical (lookback) capacity without
using a 3rd party provider like DataSift or GNIP
17. Data Capture: Initial Query
• Twitter search for “Elder Scrolls Online”
– Simplest possible way to access information
– RSS feed for 10 days (Jun 27 – July 6 2012)
18. Data Capture: Entities & Associations
Hashtag TwitterHandle URL
Unstructured Keywords
Time / Date Stamp
Who What When Where
TwitterHandle Hashtags, Keywords, Time, Date Geo (if Available)
URLs
21. Data Analysis
• Analysis needs to be rooted in the operational
need:
“How can I use social media to track and
understand public attitudes toward my
product”
• Emphasis on hypothesis generation, testing,
and experimentation
22. Data Analysis: Hashtags
• Top hashtags were almost all generic or
abstract
– Undermines tracking and understanding
– Top hashtags tied to franchise, not to the game
Hashtags
#ElderScrolls #concept
#games #nerd
#online #geek
#MMO #gamer
#skyrim #ScreenShot
23. Data Analysis: Expanding the Query
• Hash tags from an initial subset of Tweets fed
back into the initial query
Initial Query Expanded Query
Results Results
Twitter Stream
24. Data Analysis: Sentiment
• Sentiment analysis on small snippets of text
like Tweets is generally poor
• Follow and convert linked URLs into derivative
sources
• Larger text sources offer potential value
with sentiment analysis
that tweets alone cannot
offer
25. Data Analysis: Sentiment
• Top negative and positive sentiment scores
can provide a glimpse into aggregate attitudes
• Provide starting points for additional analysis
26. Next Steps: Shape the Conversation
• Create and promote hashtags that help shape
the conversation and make it easier to collect
and analyze the Twitter stream
27. Next Steps: Segment the Data
• Segment, or cluster, your data by:
– User name or handle
– Hashtags
– Keywords
– Geographic region
to explore patterns and trends at the micro level
versus the entire dataset
30. Lessons Learned
• Don’t:
– Try drinking from a fire hose, sometimes less
really is more;
– Use metrics you can’t tie to actions;
– Use visualizations or reports that strip the data
from its context.
31. Lessons Learned
• Do:
– Segment data rather than attempting to work in
the aggregate;
– Look for the why behind the message;
– Always return to the source material;
– Explore alternative explanations;
– Always consider the ultimate goal.
Intelligence is information (data) that has been transformed to meet an operational need.There are a lot of ways to move from raw data to usable intelligence.
No matter what methodology you use…intelligence analysis is an iterative processYou Collect the data, Store it, Analyze it, and Distribute the end results to your organization in some usable format.
HUMINT, Human Intelligence: intelligence gathering by means of interpersonal contact. Pros: Can reveal intentions Cons: Can be unreliableOSINT, Open Source Intelligence: intelligence collected from publicly available sources. Pros: Fast and accessible Cons: NoiseSIGINT, Signals Intelligence: intelligence-gathering by interception of signals. Pros: High volume Cons: Noise
My mom does not Tweet or have a FaceBook profile. It only seems like your friends post or tweet every 30 seconds.For example, people use different networks for different reasons so tracking individuals consistently can be difficult
Why is someone tweeting or posting? If some checks in from a store is it really because the store is so incredible that they need to share that information or is because they are trying to form an impression about their lifestyle (i.e. image shaping)?Why is much harder than What.
http://apps.washingtonpost.com/politics/transcripts/2012/presidential/live/737/Washington Post and Votertide collaboration to analyze how viewers reacted to Clinton’s speech at the DNC Convention a few months ago.They captured 496,222 tweets and generated what amounts to a very basic word cloud that really provides limited value from an analysis perspective.
What can you learn from this type of experiment with the wrong tools?A lot of people were tweeting when Clinton was speaking but not many were really tweeting about what Clinton was saying.People like funny tweets
Word clouds can tell you something about the language used but not the meaning behind the language. What you see in the cloud is what but not why.
So how do you avoid some of these pitfalls and get useful intelligence from social media? The answer to that question, or partial answer at least is the focus of the remainder of the presentation.The key ingredients for a framework include Data Capture, Data Reporting and Data Analysis components. All of which are important but the Data Analysis components are the most interesting. :>
There are a lot of platforms that you could use to do social analysis but a few key issues to consider before making a commitment:
This is where I repeat that I shamelessly stole most of this presentation from a coworker Andrew who is a huge gaming fan as well as being a super bright analyst.
Almost all NLP/text extraction/unstructured data analysis tools perform poorly on small blobs of text
Negative: Users weren’t impressed by the game’s teaser and graphics suggesting that the trailer hadn’t been well received.Positive: Other hashtags showed that fans still had positive sentiment towards the Elder Scrolls franchise in general.
Use Graph Analysis to explore the links between entities extracted from your data, for example:Identify Key InfluencersView links between tweets, websites, and blogs