a presentation of research results from the steve.museum experiments, given at the NKOS workshop @ ECDL2008

1. Public and Professional
Vocabularies: Comparing user
tagging with museum documents
and documentation
J. Trant + D. Bearman
Archives & Museum Informatics

2. research question
Can social tagging and folksonomy
improve online access to art
collections?

3. Research Questions
gilded
animal
bejeweled
antler
rearing
deer
Artist
woman (Nationality, Dates)
Title, Date
Medium
action Dimensions
hunting
Ownership
Notes museum
...
luxury
documents
dog
bow
horse
object
leaping
huntress
users tag J. Trant, Jan 28, 2008
based on Wyman et al. 2006, Fig.2
a model of tagging

4. Research Questions
research questions:
social tagging and folksonomy analysis
users folksonomy vocabulary sources
who assigns what kinds of terms
are terms found in...
how many terms? are assigned?
Art and
Architecture
Thesaurus
terms
tag works of art
assigned 1 museum
(AAT)
records Union List
of Artists
Names
how does users' terms (ULAN)
tagging differ? assigned 2
searches of
art image
databases
works of art
J. Trant, February 2008
are terms applicable to
based on Trant 2006
Research Process

5. Research Questions
Who tagged What?

6. Research Questions
research questions:
social tagging and folksonomy analysis
users folksonomy vocabulary sources
who assigns what kinds of terms
are terms found in...
how many terms? are assigned?
Art and
Architecture
Thesaurus
terms
tag works of art
assigned 1 museum
(AAT)
records Union List
of Artists
Names
how does users' terms (ULAN)
tagging differ? assigned 2
searches of
art image
databases
works of art
J. Trant, February 2008
are terms applicable to
based on Trant 2006
What was tagged?

7. Research Questions: Tagging
Multi-Institutional Tagger: Works by Institution
[all works]
1,784 works of art
The Metropolitan Museum of
Art Boston Museum of Fine Arts
14% 13%
Cleveland Museum of Art
Tate Modern 7%
3%
Skirball Denver Art Museum
1%
9%
Indianapolis Museum of Art
13%
San Francisco Museum of
Modern Art
9%
Rubin Los Angeles County Museum
6% Minneapolis Institute of Arts of Art
14% 11%
J. Trant created from
Sept 8, 2008 tagsPerWork-080828.xls
Characterizing Tagging: Works Tagged

8. Research Questions: Tagging
Characterizing Tagging: Works Tagged

9. Research Questions: Tagging
research questions:
social tagging and folksonomy analysis
users folksonomy vocabulary sources
who assigns what kinds of terms
are terms found in...
how many terms? are assigned?
Art and
Architecture
Thesaurus
terms
tag works of art
assigned 1 museum
(AAT)
records Union List
of Artists
Names
how does users' terms (ULAN)
tagging differ? assigned 2
searches of
art image
databases
works of art
J. Trant, February 2008
are terms applicable to
based on Trant 2006
Characterizing Tagging

10. Research Questions: Tagging
research questions:
social tagging and folksonomy analysis
users folksonomy vocabulary sources
who assigns what kinds of terms
are terms found in...
how many terms? are assigned?
Art and
Architecture
Thesaurus
terms
tag
descriptive statistics:
works of art (AAT)
assigned 1 museum
records
tagger velocity Union List
of Artists
how does users' terms tag variation Names
(ULAN)
tagging differ? assigned 2
searches of
art image
databases
Tags per work works of art
# of tags # of taggers # tags per tagger
J. Trant, February 2008
are terms distinct
total applicabletotal
to total tags distinct tags
based on Trant 2006 high low average std. dev. high low average std. dev.
all works
work 1
work 2
work 3
Characterizing Tagging

11. Research Questions: Tagging
Multi-Institution Tagger Single-Institution Tagger
# of # of # of
# of tags # of users # of tags
works works users
total distinct total total distinct total
1,784 36,981 11,944 1,621 251 56,399 21,557 690
Characterizing Tagging: Basic Descriptive Statistics

12. Research Questions: Tagging
total tags distinct tags
90.0
67.5
45.0
22.5
0
multi-institution single institution
Characterizing Tagging: Basic Descriptive Statistics

13. Research Questions: Tagging
Characterizing Tagging: Compare Registered | Unregistered

14. Research Questions: Tagging
Multi-Institutional Tagger: Sessions per Registered User
4 or more sessions
6%
3 sessions
6%
2 sessions
20%
1 session
68%
J. Trant
Sept. 10, 2008 created from steve_termset4-view_user080909.xls
Characterizing Tagging: Sessions per Registered User

15. Research Questions: Tagging
Multi-Institutional Tagger: Tags per Work, Average by Object Type
number of tags per object type divided by number of works of that object type
highlighted values for samples of fewer than 10 works
45
41.0
40
Average Tags / Work
With Coverage: Average Tags / Work
35 Linear (Average Tags / Work)
33.4 29.8
30 28.1
27.0 26.3
24.9 25.1 24.4
25 27.1 27.0 22.3
22.2
25.1 21.2
23.4 23.7
20 22.2 21.6
20.7 17.3
20.1 16.0
15.0
15 16.4 16.0
14.5
13.4
10
5
1.5
0
s
rs
ks
ts
s
gs
o
ia
ns
s
s
ry
re
er
e
ile
h
t
ok
de
ur
ed
ec
lo
in
el
u
th
or
tin
ap
tio
xt
pt
ct
Bo
co
Vi
Pr
w
bj
O
M
lW
r
in
la
Te
ite
ul
Je
o-
og
er
O
d
al
Pa
Sc
Al
di
ch
at
e
ot
n
d
st
ix
ia
Au
an
W
Ph
Ar
In
M
ar
d
e
lit
an
m
tli
tu
s
U
s
g
Co
in
d
an
w
ra
ts
D
Ar
created from
e
iv
J. Trant tagsPerWork-080828.xls
at
or
Aug 28, 2008
ec
D
Characterizing Tagging: Tags Per Work by Object Type

16. Research Questions: Tagging
Characterizing Tagging: Vocabulary Stabilization?

17. Are tags new terms?

18. Research Questions: Museum Documentation
research questions:
social tagging and folksonomy analysis
users folksonomy vocabulary sources
who assigns what kinds of terms
are terms found in...
how many terms? are assigned?
Art and
Architecture
Thesaurus
terms
tag works of art
assigned 1 museum
(AAT)
records Union List
of Artists
Names
how does users' terms (ULAN)
tagging differ? assigned 2
searches of
art image
databases
works of art
J. Trant, February 2008
are terms applicable to
based on Trant 2006
Do user tags differ from museum documentation?

19. Research Questions: Museum Documentation
Museum Documentation: Catalogue Records On-line

20. Research Questions: Museum Documentation
Museum Data Definition Req. Example
Institution The display version of your museum's name y The Metropolitan Museum of Art
Image filename The name of the image file that matches this y aps854.jpg
object.
Institution An identifying number assigned to the object n 11479
ObjectID in your institution, other than an accession
number (e.g. the record number from your
collections management system).
Title/Object The display for of the work's preferred Title y The Gulf Stream
Name or Name.
Date The display form of the work's date. y 1899
Maker/Culture The display form of the creator's [or y Winslow Homer, 1836–1910
creators'] name[s] (and dates if desired)
Medium The display form of the material, support y oil on canvas
and/or techniques used to make the work
Dimensions The display form of the work's dimensions y 28 1/8 x 49 1/8 in. (71.4 x 124.8 cm)
Object Type The type of work, e.g. painting, sculpture, y painting
installation
Credit Line The display form of any credit to be shown y Catharine Lorillard Wolfe Collection, Wolfe Fund,
with the work of art. 1906
Copyright The Display form of any copyright statement n © Jasper Johns/Licensed by VAGA, New York, NY
for the work of art.
Accession The accession number assigned to the work y 06.1234
Number of art in your museum
Notes curatorial or other notes about the work. n The Gulf Stream was based upon studies made
during Homer's two winter trips to the Bahamas in
1884–85 and 1898–99. First exhibited at the
Pennsylvania Academy of the Fine Arts in
Philadelphia in 1900, the picture was subsequently
reworked and quot;improvedquot; by the artist. Early
photographs show changes to the sea and to the
back of the ship, making the composition more
dramatic and vivid. The painting was shown in this
state at the Carnegie Institute in Pittsburgh in
1900–1901, and then at M. Knoedler and Co. in
New York, where the artist placed on the picture the
record-asking price of $4,000. There were problems
selling the work because of either its high price or
its unpleasant subject matter. Homer may have
reworked the painting again in the face of this
criticism in order to add the rigger on the horizon
that signals hope and rescue from the perils of the
sea.
Museum Documentation: Catalogue Records On-line

21. Research Questions: Museum Documentation
Museum
New Tags
Tags ? Documentation
Redundant
tags
Do user tags differ from museum documentation?

22. Research Questions: Museum Documentation
Do user tags differ from museum documentation?

23. Research Questions: Museum Documentation
Do user tags differ from museum documentation?

24. Research Questions: Museum Documentation
Do user tags differ from museum documentation?

25. Research Questions: Museum Documentation
Multi-institutional Tagger:Documentation Found in Museum Documentation
Museum Terms Not
Multi-Institutional Tagger: Terms Not Found in
200
150
Times Assigned
100
50
0
an
s
gn
om
si
w
ic
da
zo
J. Trant created from
September 18, 2008 Term termsMuseumDocumentation.xls:musdocNULL-freq
Do user tags differ from museum documentation?

26. Research Questions: Museum Documentation
Multi-institutional Tagger: Terms Not Found inTerms Not Found in Museum Documentation
Multi-Institutional Tagger: Museum Documentation
[terms assigned more than 50 times, by occurrence]
200
150
Times Assigned
100
50
0
he ct
es
w al
an
om n
st e
en
an
on
re al
af e
y
m se
te
fe r
o e
n
n
e
ric
s
ul le
an
re rse
g
pe r
ee
n
d
it
en
n
ds
ye s
ue
nd lue
ld
co ck
e
de
rk
ew
ca re
pa w
in e
e
rio
er
u
tiv
sk
t
il
on
ap
ge der
ee
c
ia
in
re
er
io
ra
r
llo
et
a
ra
go
hi
hi
at
da
u
e
va
a
xt
fa
ou
io
tr
m
m
ric
et
rs
om
om
nu
at
flo
br
m
as
b
lig
rv
w
ho
tt
pt
tr
rt
bl
te
gr
st
sc
ra
m
w
w
st
w
lig
te
o
flo
cl
po
ab
w
d
sc
an
de
la
k
ac
bl
J. Trant
Term
September 10, 2008
Do user tags differ from museum documentation?

27. Research Questions: Extended Museum Documentation
Are Other Forms of Museum Documentation More
Useful?
Genres
✓ gallery label text
✓ docent tour talking points / tour scripts
✓ special exhibition catalogue entries
✓ permanent collection guidebook text
✓ curatorial analysis or notes
✓ scholarly article
✓ web site 'feature' texts
✓ audio tours or podcasts
✓ newspaper/popular article

28. Research Questions: Extended Museum Documentation
I. Text mining museum extended
documentation for the sorts of terms we get in
tagging cannot be algorithmic
II. Extended documentation is not better than
online museum documentation
III. Public genres of documentation are not
meaningfully better than scholarly and
internal for tag matches.
IV. False results from Inapplicable matches if
extended documentation was searched for

29. Do Museum
Professionals find tags
useful?

30. Research Questions: Usefulness
research questions:
social tagging and folksonomy analysis
users folksonomy vocabulary sources
who assigns what kinds of terms
are terms found in...
how many terms? are assigned?
Art and
Architecture
Thesaurus
terms
tag works of art
assigned 1 museum
(AAT)
records Union List
of Artists
Names
how does users' terms (ULAN)
tagging differ? assigned 2
searches of
art image
databases
works of art
J. Trant, February 2008
are terms applicable to
based on Trant 2006
Do Museum Professionals Find
Tagging Terminology Useful?

31. Research Questions: Usefulness
if you found this work, using this term in a query,
would you be surprised?

32. Research Questions: Usefulness
Would Terms Be Useful for Searching?

33. Research Questions: Usefulness
In-depth study of a sample

34. Research Questions: Usefulness
In-depth study of a sample
Sample Works: Useful terms

35. What kinds of words
are tags?

36. Research Questions: AAT/ULAN
research questions:
social tagging and folksonomy analysis
users folksonomy vocabulary sources
who assigns what kinds of terms
are terms found in...
how many terms? are assigned?
Art and
Architecture
Thesaurus
terms
tag works of art
assigned 1 museum
(AAT)
records Union List
of Artists
Names
how does users' terms (ULAN)
tagging differ? assigned 2
searches of
art image
databases
works of art
J. Trant, February 2008
are terms applicable to
based on Trant 2006
Are tags in Museum Vocabulary Sources?

37. Research Questions: AAT/ULAN
Are tags in Museum Vocabulary Sources?

38. Research Questions: AAT/ULAN
Are tags in Museum Vocabulary Sources?

39. Research Questions: AAT/ULAN
Multi-institutional Tagger: Full Matches to AAT by Hierarchy
Are tags in Museum Vocabulary Sources?

40. Research Questions: AAT/ULAN
Are tags in Museum Vocabulary Sources?

41. Research Questions: AAT/ULAN
ULAN Name: Green Terms are Unambiguous
Are tags in Museum Vocabulary Sources?

42. Are tags useful for
searching?

43. Research Questions: Searching
research questions:
social tagging and folksonomy analysis
users folksonomy vocabulary sources
who assigns what kinds of terms
are terms found in...
how many terms? are assigned?
Art and
Architecture
Thesaurus
terms
tag works of art
assigned 1 museum
(AAT)
records Union List
of Artists
Names
how does users' terms (ULAN)
tagging differ? assigned 2
searches of
art image
databases
works of art
J. Trant, February 2008
are terms applicable to
based on Trant 2006
Could tagging improve on-line searching for works of art?

44. Research Questions: Searching
New Tags
Tags could Tags Redundant
improve search Tags
? ?
Unsuccessful Search Museum Museum Documentation
Searches Terms Documentation not used in search
?
Successful
searches
Could tagging improve on-line searching for works of

45. Summary
85%+ Tags are Not Found in Museum Documentation
60%+ don’t Match Vocabularies
Most Tags Can’t Be Mined From Other Sources

46. Summary
Public Vocabulary is
Different.
Tagging does
Contribute.