'Pitch' type presentation for the SocialZap demonstrator at the SEALincMeida Symposium on 23 November 2012. Also includes an update of the current status of the R&D with the various technical partners

1. SocialZap
Enriching Television Content with
Social Media
Bouke Huurnink
SealincMedia
Symposium
23-11-2012

2. COMMIT
kickoff
20 March 2012

3. Imagine you are a television
producer

5. You increasingly put content
online

7. But someone is taking your viewers
– and your ad revenue…

9. Why are they going to
YouTube?

10. The Power of Social
The crowd decides what’s
interesting

11. How can we use the power of
social?

12. Introducing…
SocialZap

13. SocialZap
1. Give it a television broadcast, it finds
popular topics in online chatter
2. It then identifies the ‘Zap Points’ in the
broadcast for those topics
3. Shares the social Zap Points on your
website, identifying popular content for
more viewers
Goodbye YouTube!

14. Zap Points go
here
(or we can make you a
cool new website)

15. Who are we?

16. The SocialZap partners
Language analysis
pros
Computer vision
experts
Software
pros
The keepers of Dutch
television
Multimedia
interfacers
Social mavens
all

17. How will we do it?

18. SocialZap – how it works
1. Collect tweets for your broadcasts
2. Analyse them for popular (visual) topics
3. Learn a visual model for popular topic
4. Find that topic in your broadcast
5. Display in your (Uitzending Gemist) player
6. Use Zap Points for sharing, commenting, and
getting more viewers

19. SocialZap
Using the crowds to find the crowds
(and get back your ad revenue)

20. That was the pitch
• Now for the nitty-gritty stuff

21. System module dataflow
Central Server
Tweet Caption Video Video
Front-End Users
Analyzer Analyzer Miner Indexer
Closed Captions
(optional) Uitzending
Beeld en Geluid
Gemist
User-Generated Content

22. System module dataflow
Central Server
Tweet Caption Video Video
Front-End Users
Analyzer Analyzer Miner Indexer
Closed Captions
(optional) Uitzending
Beeld en Geluid
Gemist
User-Generated Content

23. System module dataflow
Central Server
Tweet Caption Video Video
Front-End Users
Analyzer Analyzer Miner Indexer
Closed Captions
(optional) Uitzending
Beeld en Geluid
Gemist
User-Generated Content

24. System module dataflow
Central Server
Tweet Caption Video Video
Front-End Users
Analyzer Analyzer Miner Indexer
Closed Captions
(optional) Uitzending
Beeld en Geluid
Gemist
User-Generated Content

25. System module dataflow
Central Server
Tweet Caption Video Video
Front-End Users
Analyzer Analyzer Miner Indexer
Closed Captions
(optional) Uitzending
Beeld en Geluid
Gemist
User-Generated Content

26. System module dataflow
Central Server
Tweet Caption Video Video
Front-End Users
Analyzer Analyzer Miner Indexer
Closed Captions
(optional) Uitzending
Beeld en Geluid
Gemist
User-Generated Content

27. Status
• System-level
– Designed blueprint of the system architecture
– Determined role division among partners
– Defined data exchange
• Data-level
– Small dataset of DWDD and POWNews video and tweets created
• Component-level
– Tweet analyzer provided first results
– Video miner provided first results
– UvA software is up-and-running on NISV server

28. Way Forward
• Version 1 of the demo 2013 Q1
• Several extensions (shoud have's) are considered, e.g.,
– Add speech recognition
– User-generated content for determining 'tweet intent’
– Training data from tweets
– Linking archive elements to identified fragments in Uitzending Gemist
content.
• We believe link with Use-Case 2 is most prevalent in interface that
will be developed
– Once interface is more concrete we should make the link explicit.
• We foresee synergy with the COMMIT Infinity project, for speech
recognition and twitter analysis.

29. On a side note
• A positive development for Sound and Vision
– It is time to start integrating automatic techniques
in the archive
– This is the first time computer vision software is
being integrated into Sound and Vision workflow
• Also, Twitter data as potentially valuable
source of contextual metadata?!

30. Conclusion
• SocialZap enables users to zap to popular points of
discussion
• Replaces uploading to YouTube and laborious manual
entry
• Challenging research questions around social media
analysis and video analysis
• Establishes computer vision workflow in audiovisual
archive
SocialZa
p!