The document discusses improving search and annotations for video fragments in cultural heritage collections. It notes that current annotations are too coarse-grained, refer only to entire videos, and use specialized vocabulary unknown to general audiences. It also observes that people frequently search for and request specific video fragments rather than entire videos. The document proposes improving support for fragment search, using time-based annotations, and bridging the vocabulary gap between searchers and cataloguers, possibly through crowdsourcing.

1. use cases with
semantics & user
interaction inTV &
cultural heritage
collections
Lora Aroyo
@laroyo
HUMANS & MACHINES TOGETHER
Friday, May 17, 13

2. Time-consuming:
5 time the video duration
Coarse grained:
refer to entire video
Professional vocabulary:
experts use a specific vocabulary
that is unknown to general
audiences
OBSERVATION I:
VIDEO ANNOTATIONS
Friday, May 17, 13

3. People request fragments:
broadcast (33%), stories (17%),
fragments (49%)
Finding fragments longer:
stories (2x), fragment (3x)
Not finding:
35% of clicked results
not found by title or term
OBSERVATION II:
SEARCH BEHAVIOR
Friday, May 17, 13

4. Improve support for
fragment search
time-based annotations
bridging vocabulary
gap between searcher
& cataloguer
crowdsourcing?
OBSERVATION III:
CHANGE NEEDED
Friday, May 17, 13

5. CULTURAL HERITAGE WITH
INVOLVED CONSUMERS &
PROVIDERS
data: open, shared & accessible
infrastructures: interoperable
smart services: relevant
content in right context
PART I
Friday, May 17, 13

6. http://chip-project.org
http://agora.cs.vu.nl
Rijksmuseum
Amsterdam
Friday, May 17, 13

7. http://openimages.eu
http://academia.nl
MUNCH project
CHOICE project
Netherlands Institute
for Sound & Vision
Friday, May 17, 13

8. Europeana
Cultural Search
http://e-culture.multimedian.nl
http://europeana.eu/portal/
thoughtlab.html
http://europeana.eu
Friday, May 17, 13

9. How to use the Crowd
Friday, May 17, 13

10. http://waisda.nl
http://www.prestoprime.org/
@waisda
Friday, May 17, 13

11. Friday, May 17, 13

12. Friday, May 17, 13

13. Friday, May 17, 13

14. Friday, May 17, 13

15. Two Pilots
Pilot I Pilot II
months 8 2
videos 612 1544
players 2000 438
tags 420000 172000
Friday, May 17, 13

16. On the Role of User-Generated Metadata in A/V Collections
Riste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011
Friday, May 17, 13

17. time-basedbernhard
just “tags”
On the Role of User-Generated Metadata in A/V Collections
Riste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011
Friday, May 17, 13

18. user vocabulary
8% in professional vocabulary
23% in Dutch lexicon
89% found on Google
objects (57%)
westminsterabbey
abbey
priester
geestelijken
hek
paarden
tocht
aankomst
koets
kroning
mensenmassa
parade
kroon
regen
On the Role of User-Generated Metadata in A/V Collections
Riste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011
Friday, May 17, 13

19. user vocabulary
8% in professional vocabulary
23% in Dutch lexicon
89% found on Google
persons (31%)
bernhard
juliana
objects (57%)
On the Role of User-Generated Metadata in A/V Collections
Riste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011
Friday, May 17, 13

20. user vocabulary
8% in professional vocabulary
23% in Dutch lexicon
89% found on Google
locations (7%)
engeland
persons (31%)
objects (57%)
On the Role of User-Generated Metadata in A/V Collections
Riste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011
Friday, May 17, 13

21. user vocabulary
8% in professional vocabulary
23% in Dutch lexicon
89% found on Google
locations (7%)
persons (31%)
objects (57%) no events
no scenes
On the Role of User-Generated Metadata in A/V Collections
Riste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011
Friday, May 17, 13

22. just “tags”
On the Role of User-Generated Metadata in A/V Collections
Riste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011
Friday, May 17, 13

23. hye park just “tags”
typos
On the Role of User-Generated Metadata in A/V Collections
Riste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011
Friday, May 17, 13

24. just “tags”
typos
no unique references
bernhard
On the Role of User-Generated Metadata in A/V Collections
Riste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011
Friday, May 17, 13

25. just “tags”
typos
no unique references
beefeater
no synonyms
On the Role of User-Generated Metadata in A/V Collections
Riste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011
Friday, May 17, 13

26. THE QUALITY OF TAGS?
Friday, May 17, 13

27. Tag gardening
annotations for video, fragments & scenes
adding types & roles to user tags
mapping tags to concepts, disambiguation
spelling corrections
single & batch tags/video editing
Friday, May 17, 13

28. Linking user-generated video annotations to the web of data
Michiel Hildebrand et al. International Conference on Multimedia Modeling 2012
Friday, May 17, 13

29. Linking user-generated video annotations to the web of data
Michiel Hildebrand et al. International Conference on Multimedia Modeling 2012
group by type
(faceted)
Friday, May 17, 13

30. Linking user-generated video annotations to the web of data
Michiel Hildebrand et al. International Conference on Multimedia Modeling 2012
Friday, May 17, 13

31. Linking user-generated video annotations to the web of data
Michiel Hildebrand et al. International Conference on Multimedia Modeling 2012
concepts
uniquely
identified
Friday, May 17, 13

32. Linking user-generated video annotations to the web of data
Michiel Hildebrand et al. International Conference on Multimedia Modeling 2012
Friday, May 17, 13

33. Linking user-generated video annotations to the web of data
Michiel Hildebrand et al. International Conference on Multimedia Modeling 2012
background
info
Friday, May 17, 13

34. Friday, May 17, 13

35. Friday, May 17, 13

36. Friday, May 17, 13

37. Install Your Own Waisda?
Friday, May 17, 13

38. TVWITH INVOLVED
CONSUMERS & PROVIDERS
42% UK adults discuss/
comment onTV programs
while watching (YouGov Deloitte report,Aug 2010)
49% of UKTwitter users use
Twitter regularly when
watchingTV (BBC Audience Research Twitter survey,Aug 2010)
PART II
Friday, May 17, 13

39. but ...
© VickyBuser, BBC
Friday, May 17, 13

40. lost in space
© RedBee Slides at MIPCube2012
Friday, May 17, 13

41. demand for new experience
Friday, May 17, 13

42. http://notube.tv
http://www.slideshare.net/NoTubeProject
@notubeproject
© Dan Brickley
Friday, May 17, 13

43. NoTube’s N-Screen
N-Screen
drag & drop, sharing & TV control
drag & drop, shuffle, sharing & TV control
Friday, May 17, 13

44. NoTube’s User Activities
aggregate
Friday, May 17, 13

45. NoTube’s User Interests
profile
Friday, May 17, 13

46. NoTube’s Beancounter
aggregate, analyze & profile
Friday, May 17, 13

47. Recommendations in NoTube
• surface interesting, new & relevant programs to individual and group users
• combine in a complementary way different statistical & semantic approaches
• define metrics for, e.g. serendipity, diversity, relevance
© Dan Brickley
Friday, May 17, 13

48. • access to basic metadata about programs
• links are the basic currency of social media
• URIs for the things you watch
• links to related Web entities
• even a small amount of metadata enables interesting apps
Open Web Standards & Data
Friday, May 17, 13

49. • The move to IP - opportunity to discover what people watch in greater breadth and depth
• gather consumers’ viewing behavior & video streams
• combine them with enhanced EPG as input for a holistic live-stream data mining analysis
• generate a high-quality linked open dataset (LOD) describing live TV programming
• market research on viewing behavior
Friday, May 17, 13

50. Michiel Hildebrand
Lotte Belice BaltussenJacco van Ossenbruggen
Marteen Brinkerink
Johan Oomen
Guus Schreiber
Riste Gligorov
Lora Aroyo
THE TEAM
@lottebelice
@johanoomen
@mbrinkerink
@laroyo
@jrvosse
@GuusSchreiber
@McHildebrand
Friday, May 17, 13

51. THE TEAM
Friday, May 17, 13

52. • http://waida.nl
• http://prestoprime.org
• http://notube.tv
• http://vista-tv.eu
• http://lora-aroyo.org
• @waisda
• @prestocenter
• @notubeproject
• @vistatv
• @laroyo
Questions
Friday, May 17, 13