Building the PoliMedia search system; data- and user-driven

www.polimedia.nl
Building the PoliMedia system;
data- and user-driven

Who are we?
Laura Hollink
• Assistant professor at VU
• Modeling, linking and enrichment
of data
• Data-driven research
• @laurahollink
Max Kemman
• Junior researcher at EUR
• Human-Computer Interaction
• User-driven research
• @MaxJ_K
eHumanities group - PoliMedia 2
PoliMedia team
Henri Beunders (EUR)
Jaap Blom (NISV)
Laura Hollink (VU)
Geert-Jan Houben (TU Delft)
Funded by CLARIN-NL
Damir Juric (TU Delft)
Max Kemman (EUR)
Martijn Kleppe (EUR)
Johan Oomen (NISV)

Linking Politics to Media

The research questions
• How is a person, subject or process covered & visualised by the media?
• How do debates and arguments develop over a longer period of time?
• Analysing the changing ideas, arguments and presentation in different
media

Issues with current approach

Goal: explicit links to different media
types in one system

PoliMedia system
PoliMedia
Portal
- Browse:
debate and
date
- Search:
debate and
person
Newspapers
KB
Television
Sound and Vision
Radio
KB
Staten
Generaal
Digitaal
KB
Data-driven (Laura) & user-driven (Max)

Data

Debate data
Handelingen der Staten-General or Dutch Hansard
from 1945-1995
Some provenance:
1. Transcripts are made of the complete debates of the Dutch
parliament.
2. Published online by the government on
http://www.statengeneraaldigitaal.nl/ (1818 1995) and
http://officielebekendmakingen.nl/ (from 1995)
3. PoliticalMashup project has translated government pdf and
txt files into XML, incl URI’s as identifiers, see
http://politicalmashup.nl/
4. We build on that.

Debate
Metadata
Topic 1
Topic 2
Speaker 1 / Content
Speaker 2 / Content
Speaker 3 / Content
Speaker 1 / Content
Structureof the
debate data
Including:
• who, when, what
• identifiers for subparts
of the debate
• chronological order of
speakers

Media data
• Newspaper articles
– at the National Library of the
Netherlands
– Many newspapers 1950- 1995
– Text + images of newspaper
layout
• Radio bulletins
– Transcripts of ANP news
• Newscasts
– in the Academia collection of the
Netherlands institute for Sound
and Vision

Semantic model
nl.proc.sgd.d.
194519460000002
nl.proc.sgd.d.
194519460000002.1
PartOfDebateDebate
http://resolver.politicalmashup.nl/nl.proc.sgd.d.194519460000002
http://statengeneraaldigitaal.nl/
http://resolver.kb.nl/resolve?urn=sgd:mpeg21:19451946:0000002:pdf
nl.proc.sgd.d.19720000002
Handelingen Verenigde
Vergadering...
Dutch
1945-11-20
rdf:type
dc:id
dc:source
dc:source
dc:publisher
dc:language
dc:date
hasPart
rdf:type
nl.proc.sgd.d.
194519460000002.1.1
hasPart
DebateContext
rdf:type
nl.proc.sgd.d.
194519460000002.1.2
Speech
rdf:type
hasPart
nl.proc.sgd.d.
194519460000002.1.3
hasSubsequentSpeech
"Mijnheer de
Voorzitter, de
Commissie
van …"
hasSpokenText
sem:hasActor
"De voorzitter
opent de
vergadering…"
hasText
http://resolver.kb.nl/resolve?urn=ddd:011198136:mpeg21:a0525:ocr
coveredIn
nl.proc.sgd.d.
194519460000002.2
hasSubsequentPartOfDebate

Semantic model
sem:hasActor
Speaker_0006
4
Party_kvp
hasParty
hasSpeaker
member_of
_parliament
Party
KVP
Katholieke Volkspartij
rdf:type
hasAcronym
hasFullName
Joannes Antonius James
Bargefoaf:ﬁrstName
foaf:lastName
Barge
rdfs:label
http://resolver.politicalmashup.nl/nl.m.00064
dc:source
Politician
rdf:type
hasRole
Reuse of vocabularies:
Simple Event Model (SEM),
Dublin Core, FOAF, links to
ISOCAT data categories.

Linked Data
• Data openly accessible in a semantic Web standard
• Easy to combine with other semantic Web data
• E.g. DBpedia data on politicians and parties.

Linking Debates to Newspaper
articles that cover them
• Challenges:
– How to link documents that are so different in
nature?
– Can we use the structure of the debates: people,
chronologic order of speeches, introductions to
each new topic, etc.
– How can we do this efficiently, using the access
mechanisms of the archives?

Linking approach
Detect
topics in
speeches
Create
queries
Search
newspaper
archive
Topics
Named
Entities
Name of
speaker
Detect
Named
Entities in
speeches
Candidate
articles
Queries
Rank
candidate
articles
Links
between
speeches
and articles
Debates
Date of
debate

Detect topics
The MALLET topic model package
• Unsupervised analysis of text
• “a Topic consists of a cluster of words that frequently occur together”
• [see http://mallet.cs.umass.edu/topics.php]
• Input:
– Text
– Number of iterations
– Number of topics/clusters
• Output:
– Words that cluster around one topic.
• Example:
– Text: a speech in a debate from 1975
– number of iterations: 2000
– number of topics: 1

Create Queries
Named
Entities from
the speech
Named
Entities from
the debate
intro
Topics from
the speech
Topics from
the debate
intro
Name of
speaker Date of debate
Named
Entities from
the speech
Named
Entities from
the debate
intro
Topics from
the speech
Topics from
the debate
intro

Evaluation
• Experiment 1: NEs in speech
• Experiment 2: NEs + topics in speech
• Experiment 3: NEs + topics in speech and debate

Results
• A linked open data set of Dutch parliamentary
debates.
• With links to URL’s of news paper articles and
radio bulletins at the Royal Library.
• A system that supports researchers in finding
the data to answer their questions.

User-driven
What do scholars want?
• Why user research?
• Understanding the user [1, 2]
– Acceptance
– Performance
– Capabilities
– Weaknesses
• Goal
– Creating a system that is intuitive and helpful to the users
[1] Y. Liu, A. Osvalder, and M. Karlsson, “Considering the importance of user profiles in
interface design,” no. May, 2010
[2] J. Preece, Y. Rogers, and H. Sharp, “Interaction Design: Beyond Human-Computer
Interaction,” Design, vol. 18, no. 1, pp. 68-68, 2002

User research in the development
process
• Examine search behaviour of users
– Survey regarding search strategies
– Interviews
• User wishes → user requirements
• Wireframes → Prototype
• Evaluation →New prioritization of remaining
user requirements
• Final version

Survey
General search strategies
• N=294
• Popular search engines
Very often
Often
Regularly
Sometimes
Never
Don’t
know it
Google
GoogleImages
GoogleScholar
YouTube
JSTOR
KB
Flickr
EBSCO
NationaalArchief
WebofKnowledge
UitzendingGemist
Yahoo!
Bing
Academia.nl
Europeana
Scopus
MicrosoftAcademicSearch
EUscreen
Arkyves
24

Survey
General search strategies
1. Keywords 4,75
2. Advanced search 3,36
3. Related terms 2,52
4. Boolean 2,42
5. Browsing subject
categories 2,29
6. Filters 2,19
7. Thesaurus 1,87
8. Visualization 1,22

Survey
Conclusions
• Google is the dominant search engine
• This has two consequences
1. People compare other search systems to their
experience with Google
2. The search task is mainly performed by using
keywords

Interviews
• N=5
• Quantitative (n=2) as well as qualitative (n=4)
• Main themes
– How do people search currently?
– What could be improved about current search systems?
– What should PoliMedia offer, given its goals?
• Results
– 39 user wishes
– Prioritized internally
• 19 user wishes deemed out of scope
• 20 user requirements

Interviews
Findings
• Key issue is to provide a good overview of data
– Why are search results retrieved
– How are search results ranked
• Assumptions of relevance
– Higher frequency of keywords indicated higher relevancy to
query?
– Longer segments (speeches and articles) indicate higher
importance?
• Many more or less out-of-scope wishes to make current
research easier
– Sentiment-metadata
– Context metadata
– Ability to export to own software

• Clear and
immediate
keyword-search
• Support for
Booleans and
(some) Google-
search operators
• Separate
advanced-search
Wireframes
Search interface

Wireframes
Search results
• Keyword search
remains
prominent
• User chosen
ranking of results
• Keyword
highlighting
• Overview of
related media
• Support for
filtering

Wireframes
Debate page
• Keyword search
remains
prominent
• Overview of
people in debate
• Easy access to
related material
31eHumanities group - PoliMedia

Prototype v1.0

Evaluation
• Eye tracking evaluation of the search system
– Search system was still in development
• N=24
– History
– Political communication
• Goals
– Gain understanding of distribution of attention
– Collect general feedback on interface

Evaluation
Eye tracking
• Viewing Duration
• Search bar received little attention after
search results were displayed
• Facets received a lot of attention
• Page-search (CTRL+F) mainly received
attention on debate page view
Tasks Search bar Facets Search results Page-search
Known Item 17% 22% 60% 2%
Exploratory 6% 12% 80% 2%

Evaluation
Usability feedback
• The ranking of search results was an issue for
users
• The year-filter should be a slider
• The debate page should be greatly improved
– Better identification for speaker, party, topic,
relevance to query
– Provide filters on debate-page as well

Prototype v2.0

Prototype v2.0 - query

Prototype v2.0 – filter speaker

Prototype v2.0 - filter role

Prototype v2.0 - debate

Prototype v2.0 - highlight speech

Prototype v2.0 - link newspaper

Prototype v2.0 - newspaper

Prototype v2.0 - link radio

Conclusion
• PoliMedia; data- or user-driven?
• Continuous interplay
– Users gave input for usefulness of links
– Data limits what features we can offer to users
• Collection quality and usability are both critical to
users [3]
[3] Xie, I. (2006). Evaluation of digital libraries: Criteria and problems from users’
perspectives. Library & Information Science Research, 28(3), 433–452.
doi:10.1016/j.lisr.2006.06.002

Building the PoliMedia search system; data- and user-driven

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (12)

Semelhante a Building the PoliMedia search system; data- and user-driven

Semelhante a Building the PoliMedia search system; data- and user-driven (20)

Mais de MaxKemman

Mais de MaxKemman (10)

Último

Último (20)

Building the PoliMedia search system; data- and user-driven

Notas do Editor