Presentation of Kathleen Moore, Andrea H. Tapia and Christopher Griffin on the topic "Understanding How Emergency Managers Evaluate Crowdsourced Data: A Trust Game-Based Approach" at ISCRAM2013

1. Understanding How Emergency Managers
Evaluate Crowdsourced Data:
A Trust Game-Based Approach
Kathleen Moore
Andrea H. Tapia
Christopher Griffin
The Pennsylvania State University
College of Information Sciences & Technology
ISCRAM 2013 – Baden-Baden, Germany

2. Problem Statement
• Crowdsourced data by Emergency Managers
(EM) has been a significant topic of scholarly
discussion
• Strongest barrier to use identified as data quality
• Focus on data quality is a small part of problem
space
• Need to understand info behavior

3. Background
• Role of EMs
• Avoid and mitigate risk
• Prepare, respond, recover from disasters
• EM decision making
• Hampered by stress of time
• Cognitive limits of EM
• Amount and quality of information
• Not necessarily problematic as research has shown that
even with all available information, managers do not
always make optimal decision regarding events and can
misinterpret information

4. From Data to Decisions
• EMs overcome through satisficing
• Choosing good-enough solutions over perfect
ones
• Social media a technical solution to
enhance dissemination
• Problem: flow, quality, usability,
trustworthiness, verifiability
• Overwhelms the process/EM
• Any technical solutions must account for
information culture of the organization/EM
receiving the data

5. Assessing Trust in Data
• Trust: acceptance of a certain amount
of risk when lacking full knowledge and
lacking the ability to fully control a
situation (Alpern 1997)
• Ability, benevolence, integrity
• Fluid, dynamic, reiterative
• Trust, mistrust, distrust, untrust
• Better understood in static Web,
ecommerce, old social media, less so in
new social media
• Primarily studied from perspective of
information provider and not receptor

6. Trust & Social Media
• Old rules/understandings do not apply
• Trust IS NOT one, two, or even three data points
• One Tweet may yield OVER 30 pieces of information!
• Meredith Morris @ Microsoft
• Social Cues are less understood
• What is ability, benevolence, integrity in 140 characters?
• To study trust in microblogging:
• Person providing information
• Quality, consistency of that information
• ACT of the person ACCEPTING that information

7. Research Goal
• Study EM as a trustworthy data analyst
• Propose: develop model for capturing trust-
analytical behavior through game theory and
semantic content

8. Research Design
• Phase I
• Measuring trustworthy data, modeling trust behavior, and
building a game
• Non-cooperative, and Game Theory perspective
• Game Theory (loosely)
• System for analysis of behavior where consequences of
actors’ decisions depend on information provided by others
for EM to act upon
• System to organize, capture and learn from future experience
• Game setting creates sense of competitiveness/urgency to
mimic stress of pressing events facing EM

9. The Math!
• Beautiful! Elegant!
• It will change what you have for breakfast!
• Refer to paper in ISCRAM Proceedings
• PLEASE email any questions!!

10. The Game
• Multi-turn 2 player game
• Player 1 – Teller (microblogger)
• Player 2 – Actor (EM)
• Player 2 does or does not act on info by Player 1 in a finite set of sentences
• Every piece of information is either consistent with what we know, inconsistent
or could be consistent, but no way to tell
• Halting play occurs when Player 2 no longer wishes to accept information from
Player 1 (deemed untrustworthy)
• The objective of game is to obtain largest net payoff possible
• Component of experimentation, develop tool to help EM detect less than
credible information
• Determine impact information has on quality of a player’s responses to
stimuli

11. Game Format

13. Implications/Future Work
• Develop models of how trust works in a
microblogged environment providing
crowdsourced data during crisis events
• Will need to address veracity of trust factors in
crowdsourced data as viewed by analysts rather
than everyday persons/technical specialists
• Also reconcile potential discrepancies between
what is perceived as trustworthy versus what is
acted upon given perception of trustworthy.

14. Thank You
ISCRAM 2014
Kathleen Moore
kam6015@psu.edu