1. Detection and Extracting of Emergency
Knowledge from Twitter Streams
Bernhard Klein, Xabier Laiseca, Diego Casado-
Mansilla, Diego Lopez-de-Ipiña and Alejandro Prada
Nespral
6th International Conference on Ubiquitous Computing and Ambient Intelligence
Session 10: Key application domains: eEmergency, eLearning, eTraining
5. December, 2012
Social
Awareness
Based
Emergency
Situation
Solver
UCAmI 2012 B. Klein 1/17

2. Outline
1. Problem Description
2. Research Field
3. Architecture of Analysis Tool
4. Semantic Social Network Analysis
5. Recent Advances
6. Conclusions
UCAmI 2012 B. Klein 2/17

3. Objective
 Trends Detection  Event Knowledge Extraction
≠ Counting of Keywords
 Aggregation + Interpretation of post content!
 Problems:
 Big data
 Noisy + short posts
 Real-time support
UCAmI 2012 B. Klein 3/17

4. Twitter Examples
► Good examples:
► Bad examples:
► Crawling reality:
UCAmI 2012 B. Klein 4/17

5. Research Field
• SensePlace2
• Hacer and Muraki, 2011 • TweetTracker
• Sudha et al., 2011 • Twitcident
Emergency
Corpus Analysis Support
Tools
Microblogging
SNA-Techniques Clustering-Techniques
• Mendozza et al., 2010
• Becker et al, 2011
• Marcus et al, 2011
NLP-Techniques • Pohl et al, 2012
• Sudha et al, 2011
• Abel et al, 2011
UCAmI 2012 B. Klein 5/17

6. SABESS Web system
UCAmI 2012 B. Klein 6/17

7. Opensource Implementation
• Emergency message filter based on emergency taxonomy
• Language filter e.g. english or spanish
• Slang reduction (punctation + letter repititions)
UCAmI 2012 B. Klein 7/17

8. Social Network Analysis
► Objective: Filtering after tweet credibility
UCAmI 2012 B. Klein 8/17

9. Observed Problems
 “Slow” Graph Calculations
 Replace betweeness centrality with user data
a) followers count ~ influence
b) friend count ~ knowledge access
c) number of posts ~ experience
 “Sparse” Social Network
 Replace SNA with Sentiment Analysis:
Punctation-, letter- and word repititions
Tweet credibility < Informative tweet!
(see also Sudha et al., 2011)
UCAmI 2012 B. Klein 9/17

10. Natural language procesing
► Objective: Content enrichment
• Big Improvement with “slang reduction” !!
UCAmI 2012 B. Klein 10/17

11. Other Knowledge Sources
 Hierarchical Knowledge Structure
1. Textual location
a) Named Entity Location
b) Regular Expression e.g. address
(Requires reverse coding!)
2. Tweet metadata
a) GPS tagged tweets
b) Place tagged tweets
(Author location can be different!)
3. User profile data
a) Home location Increasing reliability!
UCAmI 2012 B. Klein 11/17

12. Recent Advances: Event Detection
► Objective: Group tweets into emergency events
 How to describe an emergency event?
 Emergency type, location (range), time (progress),
person/organization data, text descriptions, number of
tweets
 Global reporting standard “Common Alert Protocol”.
 Example:
UCAmI 2012 B. Klein 12/17

13. Recent Advances: Clustering
► Incremental DBSCAN
SANDY HURRICANE RELIEF VOLUNTEER EFFORTS
SANDY HURRICANE VICTIMS VOLUNTEER
SANDY HURRICANE VICTIMS VOLUNTEER EFFORT GRASSROOTS
SANDY HURRICANE VICTIMS VOLUNTEER NEWS EFFORT GRASSROOTS
Sandy, 180 SANDY HURRICANE VICTIMS POLICE
Fukuschima, 170 SANDY HURRICANE VICTIMS RELIEF
…. SANDY HURRICANE VICTIMS Locations:
Ambulance, 80 SANDY HURRICANE VICTIMS RELIEF DISASTER PrincetonHall
e.g.
….. SANDY HURRICANE RELIEF
SANDY HURRICANE RELIEF
SANDY HURRICANE RELIEF DISASTER
SANDY VOLUNTEER
Online
Conversations:
Dictionary ConversationID=83
Hashtags:
#TylerPerryFire
Attachments:
http://t.co/kqF7Xy8t
UCAmI 2012 B. Klein 13/17

14. Common Alert Protocol
Whenever clusters become modified,
generate new alert message??
Alert
CAP
Info
Place
Urls,
Figs
Cluster of tweets
UCAmI 2012 B. Klein 14/17

15. Conclusions
 Real-time analysis of noisy tweets
► Big data problem, 2 phase analysis
 Emergency message filtering
 Slang and language filtering
► Semantic Social Network Analysis
 POS/Noun tags, NER/Location tags
 Community centrality/follower count tags
► Tweet clustering
 Group tweets after hashtags, attachments and
conversations
 Group tweets after emergency specific keywords
► Common Alert Protocol
UCAmI 2012 B. Klein 15/17

16. Contact: Bernhard Klein,
Email: bernhard.klein@deusto.es
Deusto Intitute of Technology,
University of Deusto,
th International Conference on Ubiquitous Computing and Ambient Intelligence
6
Avda. Universidades, 24 | 48007 Bilbao |
Session 10: Key application domains: eEmergency, eLearning, eTraining
Spain 5. December, 2012
UCAmI 2012 B. Klein 16/17