This document discusses linking television programming to related online content through annotation and enrichment. It outlines the challenges of annotating and enriching television content and presents LinkedTV solutions. These solutions include annotating entities in television programs, linking them to related concepts and content online, and providing enriched experiences through secondary screens or applications. Proper annotation of content domains like news and cultural heritage is key to enabling relevant enrichments that add value for viewers.

1. http:/www.linkedtv.eu
Annotating TV programming and
linking to related content on the Web
Lyndon Nixon, MODUL University
18 November 2014, Potsdam, Germany

2. Structure
Introducing Linked Television
Challenges for TV Annotation
Challenges for TV Enrichment
LinkedTV Solutions
Summary

3. What is Linked Television?
„... a true TV ecosystem must functionally integrate the apps with the
television programming service and content being offered to the consumer.“
TVs can't be smart. Stop trying to make it happen. - WIRED Opinion, Gary Myer, Oct 2013

4. What is Linked Television?
„Television won't last because people will soon get tired of staring at a plywood
box every night.“
- Darryl Zanuck, movie producer, 20th Century Fox, 1946.

5. What is Linked Television?
Source: Tomorrow FOCUS Media „Smart-TV Effects 2014-1“
Source: Tomorrow FOCUS Media „Smart-TV Effects 2014-1“

6. What is Linked Television?
ARD/ZDF Online Study 2014 „Second Screen: Parallelnutzung von
Fernsehen und Internet“

7. 08.11.2012
What is Linked Television?
37-52% of TV viewers are using a
companion device alongside the TV
program.
http://www.linkedtv.eu
Ever saw something
on TV and wanted to
know more about it,
but didn‘t even know
how to search for it?

8. What is Linked Television?
Scenario 1:
LinkedNews app
 Easy access to
background to the news
 Focus on key concepts of
interest in each item
 hbbTV single screen and
dual screen apps
Scenario 2:
LinkedCulture app
 Easy access to
information on the art
objects discussed in the
program
 Secondary screen-orientation,
cast TV
program to main screen

9. Television annotations?
What we get?

10. Television annotations?
What we need?

11. Television enrichments?
What we get? What we want?

12. LinkedTV Solution
Examples of Paintings by Jan Sluijters
Video object and
text detection
dbpedia.org/resource/Jan_Sluyters
linkedtv.eu
“...schilderij van Jan Sluijters....”
Personalisation
engine
Expansion of
related concepts
Linking to
related content
Connection to
concepts
… and their digital images

13. Concept Detection
“...schilderij van Jan Sluijters....”
linkedtv.eu
Video object and
text detection
Scenario content specific video chapter
detection (News Items, Art Objects)
Aggregated REST service for all
analysis approaches in LinkedTV,
integrated with the LinkedTV Platform

14. Concept Connections
Keywords and text fragments can be connected to
uniquely identified, distinct named entities
using online entity extraction services (NERD,
http://nerd.eurecom.fr)
Introduction Silver Tea Jar
Nelleke van der Krogt Friesland
Hessel van Martena Silver (zilver)
Museum Martena Tea jar (theebus)
Franeker Emiel Aardewerk
Tin coffee decanter (art
object)
linkedtv.eu
Connection to
concepts
dbpedia.org/resource/Jan_Sluyters
Sondersitzung Blackout
BER Meesterteken
Berlin Brandenburg
airport
Zilversmid
Berlin Fire Brigade
Hartmut Mehdorn Frank Henkel
Jochen Großmann Vattenfall Europe

15. Linking to related content
linkedtv.eu
Digital images of Jan
Sluijters‘ paintings
Linking to
related content
• Information cards display relevant
properties of annotated entities by
type
• Links to related content are organised
by dimensions
• Access to related video chapters on
the LinkedTV Platform
• Specific enrichment services for
• News articles
• Cultural heritage objects

16. Linked Culture
Tussen Kunst en Kitsch Video
Chapter segmentation + tagging „Art Objects“
Entity extraction incl. Art Object annotation
Related
TKK
chapters
Related
art objects
(Europeana)
Related
White List
media
TVEnricher (TKK configuration)
Entity Proxy
Editor Tool LinkedTV Player

17. Art objects in TKK episodes

18. Art object: semantic model
http://data.linkedtv.eu/ob
ject/avro/8a8187f2-3fc8-
cb54-0140-
7dccd76f0001/2138
A silver tea jar
RDF Is-a Container http://vocab.getty.edu/aat
/300045611
CRM Consists-of Silver http://vocab.getty.edu/aat
/300010975
VRA locationCreationSite Friesland http://www.geonames.org
/2755812
DCT temporal Start: 1690, End: 1742

19. Mapping to Europeana API
http://data.linkedtv.eu/ob
ject/avro/8a8187f2-3fc8-
cb54-0140-
7dccd76f0001/2138
A silver tea jar
RDF Is-a Container
CRM Consists-of Silver
VRA locationCreationSite Friesland
what:(container+OR+houder+OR+b
ak+OR+tank+OR+blik)
proxy_dc_format:zilver
where:Friesland
DCT temporal Start: 1690, End: 1742
Leeuwarden
YEAR:[1690+TO+1742]

20. Enrichment results

21. Linked Culture
https://vimeo.com/108891238

22. Linked News
RBB Video
Chapter segmentation + tagging as „News Item“
Named Entity Expansion over news items
Related
news
items
Related
news articles
Related
White List
media
TVEnricher (RBB configuration)
Entity Proxy
Editor Tool LinkedTV Player

23. Linked News
Asparagus
Harvest
Poland Beelitz

24. Linked News
http://showcase.linkedtv.eu/showcase3.html

25. Summary
Rich annotation of AV content can enable added value services
but
• Consider the purpose of the annotation – this changes the
requirements on the annotation model and approach
• Consider the domain of the content to be annotated – every
domain has different requirements on how to annotate it
Enrichments are an example of added value on top of
annotations but
• Enrichment sources also need to be annotated
• Relevancy depends on the right choice of annotation entities
No perfect, automated approach: room for ongoing research!