O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Damage Assessment from Social Media
Imagery Data During Disasters
Dat T. Nguyen, Ferda Ofli, Muhammad Imran, Prasenjit Mit...
Types of Information on Twitter
- Twitter data from 13
recent crises
- Over 100,000 tweets
- Information types
- Types of ...
The Value of Timely Information
During Disasters
Based on FEMA large-scale survey among emergency management professionals...
The Value of Timely Information
During Disasters
Based on FEMA large-scale survey among emergency management professionals...
2013 Pakistan Earthquake
September 28 at 07:34 UTC
2010 Haiti Earthquake
January 12 at 21:53 UTC
Social Media Data and Opp...
“A picture is worth a thousand words.”
Images from 3 Different Disasters
Time-Critical Events and Information Gaps
Info. Info. Info.
Disaster event (earthquake, flood) Destruction, Damage
Informa...
Tweet4Act: Automatic Image
Processing Pipeline
Presented at ASONAM’17 as demo
Damage Severity Assessment from Images
Task: Our Task is to classify each incoming image
Into one of the three classes.
Challenges
• Task complexity: lack of labeled data, ill-defined
objects
• Poor signal-to-noise ration: social media data i...
Images Datasets: Twitter + Google
Twitter messages
collected using
- Damaged building
- Damaged road
- Damaged bridge
Quer...
Human Annotations
We used AIDR (volunteers) and Crowdflower (paid workers)
The purpose of this task is to assess the sever...
Human Annotations
We used AIDR (volunteers) and Crowdflower (paid workers)
Crowdflower annotations
AIDR was used during th...
Learning Schemes
1. Baseline (PHOW + SVM):
Pyramid Histogram of Visual Words (PHOW) features
with linear SVM
2. Pre-traine...
Learning Settings
1. Event-specific setting:
Training, development, and test sets are form the same event
Train: 60%, Dev ...
Event-Specific Results
Cross-Event using
Ecuador and Matthew as Test
Ecuador earthquake (20%) as fixed test set and all sources with 60%
Hurrican...
Event-Specific Precision-Recall Curves
and AUC
Cross-Event Precision-Recall Curves
and AUC
Conclusions
• We presented results for the task of damage
assessment from social media images
• We used real world dataset...
Thanks – Q & A
@aidr_qcri
Próximos SlideShares
Carregando em…5
×

Damage Assessment from Social Media Imagery Data During Disasters

205 visualizações

Publicada em

Rapid access to situation-sensitive data through social media networks creates new opportunities to address a number of real-world problems. Damage assessment during dis- asters is a core situational awareness task for many humanitarian organizations that traditionally takes weeks and months. In this work, we analyze images posted on social media platforms during natural disasters to determine the level of damage caused by the disasters. We employ state-of-the-art machine learning techniques to perform an extensive experimentation of damage assessment using images from four major natural disasters. We show that the domain-specific fine-tuning of deep Convolutional Neural Networks (CNN) outperforms other state-of-the-art techniques such as Bag-of-Visual-Words (BoVW). High classification ac- curacy under both event-specific and cross-event test settings demonstrate that the proposed approach can effectively adapt deep-CNN features to identify the severity of destruction from social media images taken after a disaster strike.

Publicada em: Ciências
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Damage Assessment from Social Media Imagery Data During Disasters

  1. 1. Damage Assessment from Social Media Imagery Data During Disasters Dat T. Nguyen, Ferda Ofli, Muhammad Imran, Prasenjit Mitra Qatar Computing Research Institute, Qatar The Pennsylvania State University, University Park, PA, USA Partners & Clients: New York (Suffolk) Emergency Management Dept.
  2. 2. Types of Information on Twitter - Twitter data from 13 recent crises - Over 100,000 tweets - Information types - Types of sources Source: Qatar Computing Research Institute - Published in World Humanitarian Data and Trends 2014 (UN OCHA)
  3. 3. The Value of Timely Information During Disasters Based on FEMA large-scale survey among emergency management professionals across the US. Informationvalue When information is too late
  4. 4. The Value of Timely Information During Disasters Based on FEMA large-scale survey among emergency management professionals across the US. Informationvalue When information is too late
  5. 5. 2013 Pakistan Earthquake September 28 at 07:34 UTC 2010 Haiti Earthquake January 12 at 21:53 UTC Social Media Data and Opportunities Social Media Platforms Availability of Immense Data: Around 16 thousands tweets per minute were posted during the hurricane Sandy in the US. Opportunities: - Early warning and event detection - Situational awareness - Actionable information - Rapid crisis response - Post-disaster analysis Disease outbreaks
  6. 6. “A picture is worth a thousand words.” Images from 3 Different Disasters
  7. 7. Time-Critical Events and Information Gaps Info. Info. Info. Disaster event (earthquake, flood) Destruction, Damage Information gathering Humanitarian organizations and local administration Need information to help and launch response Information gathering, especially in real-time, is the most challenging part Relief operations & reconstruction Disaster Government orgs.
  8. 8. Tweet4Act: Automatic Image Processing Pipeline Presented at ASONAM’17 as demo
  9. 9. Damage Severity Assessment from Images Task: Our Task is to classify each incoming image Into one of the three classes.
  10. 10. Challenges • Task complexity: lack of labeled data, ill-defined objects • Poor signal-to-noise ration: social media data is extremely noisy. E.g., duplicates, irrelevant • Task subjectivity: confusion between damage severity classes “severe” and “mild” • Cold-start issue: first few hours of a disaster are critical, learning ML classifiers needs labeled data
  11. 11. Images Datasets: Twitter + Google Twitter messages collected using - Damaged building - Damaged road - Damaged bridge Queries we used:
  12. 12. Human Annotations We used AIDR (volunteers) and Crowdflower (paid workers) The purpose of this task is to assess the severity of damage shown in an image… 1. Severe damage Substantial destruction, a non-livable Or non-useable building, a non- crossable Bridge, a non-drivable road 2. Mild damage Damage generally exceeding minor (e.g., 50% of a building is damaged), partial loss of amenity/roof, part of bridge is unusable or needs repairs 3. Little-to-no damage Images that show damage-free infrastructure Or small cracks, wear and tear due to age Three classes: Instructions:
  13. 13. Human Annotations We used AIDR (volunteers) and Crowdflower (paid workers) Crowdflower annotations AIDR was used during the actual event.
  14. 14. Learning Schemes 1. Baseline (PHOW + SVM): Pyramid Histogram of Visual Words (PHOW) features with linear SVM 2. Pre-trained CNN as feature extractor: We used VGG-16 network trained on the ImageNet dataset 1.2M images and 1000 classes. We used fc7 layer i.e., removed the last layer to get a 4097-dimensional vector for every image. 3. Fine-tuning a pre-trained CNN: Used existing weights of a pre-trained CNN as an initialization for our dataset Where last layer representing our task (3 classes)
  15. 15. Learning Settings 1. Event-specific setting: Training, development, and test sets are form the same event Train: 60%, Dev = 20%, Test = 20% 2. Cross-event setting: Scenario: no labeled data for the target event. Labeled data from past events is abundant. Cross-event: train on past events (source) and test on current event (target) For example: Train: Nepal earthquake + Ecuador earthquake Test: Typhoon Ruby We use Google data assuming no past event data is available
  16. 16. Event-Specific Results
  17. 17. Cross-Event using Ecuador and Matthew as Test Ecuador earthquake (20%) as fixed test set and all sources with 60% Hurricane Matthew (20%) as fixed test set and all sources with 60%
  18. 18. Event-Specific Precision-Recall Curves and AUC
  19. 19. Cross-Event Precision-Recall Curves and AUC
  20. 20. Conclusions • We presented results for the task of damage assessment from social media images • We used real world datasets • Compared non-deep learning, deep learning and transfer learning approaches • In the event-specific case, transfer learning approach performs better • In the cross-event case, we observed the more the data the better, same event data always helps
  21. 21. Thanks – Q & A @aidr_qcri

×