We present PolemicTweet a system with an encompassing, economic, and engaging approach to video tagging and analysis. Annotating and tag- ging videos manually is a boring and time-consuming process. Yet, in the last couple of years the audiences of events—such as academic conferences—have begun to produce unexploited metadata in the form of micropost activities. With PolemicTweet we explore the use of tagged microposts for both video annotation and browsing aid. PolemicTweet is a system 1) to crowd source conference video tagging with structured sentiment metadata, 2) to engage audiences in a tagging process, and 3) to visualize these annotations for browsing and analyzing a video. We describe the the system and its components as well as the results from a one-year live deployment in 27 different events.
1. PolemicTweet
Video Annotation and Analysis
through tagged Tweets
Samuel Huron, Petra Isenberg, Jean Daniel Fekete
06/09/13 IFIP Interact 2013 – Polemic tweet
7. 05/09/2013 IFIP Interact 2013 – Polemic tweet
Previous process
1. Videotape
the seminar
2. Manually
annotate and
tag
3. Publish the
video online and
let user annotate
10. 05/09/2013 IFIP Interact 2013 – Polemic tweet
Problem
2. Annotating a
video manually
is boring and
time-consuming
1. Browsing a
video after a
conference is
difficult
3. Tagging and
structuring
annotations are
even more
boring
12. Motivation
05/09/2013 IFIP Interact 2013 – Polemic tweet
3. Provide an easily
accessible system
from annotation to
video analysis.
2. Provide a visual
backchannel to
incite people to
tag their tweets.
1. Engage the
audience:
crowdsource video
annotation and
tagging by tweet.
13. A certain type of tag: Sentiments
06/09/13 IFIP Interact 2013 – Polemic tweet
18. Design Goals:
05/09/2013 IFIP Interact 2013 – Polemic tweet
1. To crowdsource
conference video
tagging with
structured
sentiment metadata.
2. To engage
audiences in the
tagging process.
3. To visualize
annotations for
browsing and
analyzing the video.
19. 0. Service Design
1. A special syntax to
enhance tweets
4. A video player with a
visualization
2. A social protocol to
organize the event over time
3. A real time visualization to
read and write tweets
20. 05/09/2013 IFIP Interact 2013 – Polemic tweet
1. Syntax properties and description:
GreenAgreement++
RedDisagreement--
YellowReferences==
BlueQuestions??
Tag Semantic Color
Simple
Brief
Neutral
Easy to parse
Expressive
Unambiguous
Design Rational
21. 2. Social Protocol
05/09/2013 IFIP Interact 2013 – Polemic tweet
A. Before the
talk: Instruct
the crowd.
B. During the talk:
Crowdsource the
video tagging with
a backchannel.
C. After the
talk: Publish
on a web video
player.
22. Instruct the crowd
05/09/2013 IFIP Interact 2013 – Polemic tweet
The network
access policy
The Twitter
hashtag
The website
URL
The tags
37. 05/09/2013 IFIP Interact 2013 – Polemic tweet
Questions
1. Is the system
sufficiently
engaging?
3. What is the
impact of
presenting tweets
and tags on a web
video player?
2. Does the visual
backchannel
incite people to
add tags?
38. Data Collection
05/09/2013 IFIP Interact 2013 – Polemic tweet
Record tweets sent
during the events
Log the user
interaction on the
web video player
Two web surveys,
one for
participants, one
for organizers
40. Question 1:
Is the system sufficiently engaging?
05/09/2013 IFIP Interact 2013 – Polemic tweet
0.2 to 7.94Tweets
per minute
0.12 to 2.13 Tags
per minute
40% Tweets using
our syntax
35 to 45 % Of all individuals
used our syntaxe
41. 05/09/2013 IFIP Interact 2013 – Polemic tweet
Question 1:
Is the system sufficiently engaging?
• Type of Audience. (size, culture, equipment, incentive, distribution)
• Type of Speaker and Twitter Reaction.
• Type of Content.
• Observance of the PT Protocol.
Influential factors for engagement
42. Question 2:
Does the visual backchannel
incite people to add tags?
05/09/2013 IFIP Interact 2013 – Polemic tweet
43. 05/09/2013 IFIP Interact 2013 – Polemic tweet
Question 3:
What is the impact of presenting tweets and
tags on a web video player?
44. 05/09/2013 IFIP Interact 2013 – Polemic tweet
Question 3:
What is the impact of presenting tweets and
tags on a web video player?
Visualization User seek activity
?
45. 05/09/2013 IFIP Interact 2013 – Polemic tweet
Question 3:
What is the impact of presenting tweets and
tags on a web video player?
TODO: Figuresp < 0.1 p < 0.1 p < 0.1
0.1 < p < 0.5 0.1 < p < 0.5P > 0.1
47. The Evaluation was challenging
05/09/2013 IFIP Interact 2013 – Polemic tweet
Deployed in the wild
Long period of time
Design change over time
Real life users
Real life setting
Because
48. Result!
1.The system
engaged the
audience to provide
data for
crowdsourced video
tagging.
2. The visual
backchannel incited
people to tag their
tweets and thus
provided an
annotation
structure.
3. The media player
augmented with the
annotated tweet
visualization was
considered a
significant
improvement over
traditional video
players.
49. Next steps
05/09/2013 IFIP Interact 2013 – Polemic tweet
As classroom
presentations.
Synchronous
web seminar.
Public debates,
and social TV.
51. Subsequent work:
05/09/2013 IFIP Interact 2013 – Polemic tweet
http://goo.gl/c2iqg0
Bubble-TV: Live Visual
Feedback for Social
TV Broadcast
Visual Sedimentation:
Data stream
visualization technique
http://goo.gl/KvllnO
User supplied
sentiment in tweets
http://goo.gl/XXVzJb
52. MORE INFOS:
+ open source
+ details
THANKS: To people who used the system and people who fund this work:
Thank you !
Contact :
Samuel.Huron@inria.fr
http://www.cybunk.com
Team :
http://www.aviz.fr
http://iri.centrepompidou.fr
goo.gl/wKG9Sg
54. Survey Result
96% (25) of respondents attended at least one
conference had already used twitter at conferences
( 7% only for reading)
86% (24) reported familiarity with the PT syntax
21% (6) with the LiveTweet interface
35% (14) with the web video player visualization
timeline.
05/09/2013 WIID: Who Is Interaction Design?
55. Survey Result
In the survey 14 users reported to know
what the Tweet Timeline Visualization is.
05/09/2013 WIID: Who Is Interaction Design?
• 64% (9) of these found that
provides useful information
• 79% (11) agree or strongly agree
that the visualization helps them to
browse the video
• 86% (12) would recommend PT to
a friend.
# to view a part of video with :
– many tweets: 57% (8)
– positive opinions (++): 29% (4)
– negative opinions (–): 43% (6)
– questions (??): 21% (3)
– references (==): 21% (3)
56. Cost and benefit of tagging technics
06/09/13 IFIP Interact 2013 – Polemic tweet
- Time consuming
- Cost is proportional
- Hight recall (100%)
- Mostly targeted at English
- Not “off the shelves”
- Precision problem
- Cost is proportional
- Language agnostic
- Accurate
- Fast
Cost Benefit
Mechanical turk
- Language agnostic
- Accurate
- Fast
- Cheap, cost is distributed
- Engage user
- Low recall
Natural Language
Processing