A presentation at the ESF-COST symposium on the Networked Humanities - July 2010.

1. CROWD SOURCINGART HISTORY Research and Applications of Social Tagging in Museums Robert Stein, Indianapolis Museum of Art Robert Stein Chief Information Officer Indianapolis Museum of Art rstein@imamuseum.org @rjstein

2. MUSEUMS HAVE A PROBLEM

3. COLLECTIONS ARE TOO BIG TO BROWSE (even for small collections) Flickr Credit ~andrewhowson

5. 54,000 objects in collection 2,242 objects on display (4%) 26,268 objectimages (48%)

6. Unless you know what to look for… Descriptive meta-data to support browsing is limited Browsing is hard Flickr Credit ~andercismo

7. Search-only interfaces inhibit discovery

8. From: J. P. xxxxxx@xxxxxx.com Date: Sat, 26 Aug 2006 11:24:43 -0700 To: timeline@metmuseum.org Subject: Looking for a paintingPlease help:I have been looking on and off for years for this painting. The painting is of a very well dressed renaissance man standing in a room (a library) in front of him on a table is a large hour glass. The painting has very rich colors. I have talked to a lot of people and they have said they have seen this painting but can't remember its name or the name of the artist.Could you please use your resources to find this painting?

11. What “J. P.” knows: painting Renaissance standing man very well dressed library hourglass table rich colors What a Met curator knows: Portrait of a Man, ca. 1520–25Morettoda Brescia (Alessandro Bonvicino) (Italian, Brescian, born about 1498, died 1554)Oil on canvas; 34 1/4 x 32 in. (87 x 81.3 cm)Rogers Fund, 1928 (28.79) Provenance: Maffei, Brescia (by 1760, as "Ritrattod'uomo con carta in mano, edOrologio, diCallistoda Lodi"); by descent to contessa Beatrice ErizzoMaffeiFenaroli Avogadro, Palazzo Fenaroli, Brescia (by 1853–at least 1857, as by Moretto); her daughter, contessa Maria LiviaFenaroli Avogadro, later marchesaFassati, Brescia (in 1862); her son, marcheseIppolitoFassati, Milan (by 1878–at least 1912); [EliaVolpi, Florence, by 1915–16; sold to Knoedler]; [Knoedler, New York, 1916–28; sold to MMA]

12. How can visitors take part in powering their own experience? source ~ mindcaster-ezzolicious

13. Visitors As Data Visitors Havethe BrainPower We Want Credit: Benedict Campbell

14. Unfortunately, they aren’t clones who will do our bidding source ~donsolo

15. MUSEUM IMPACT VISITOR ENGAGEMENT Can we create a virtuous circle with visitors that clearly expresses the value and impact of their participation? source ~m-louis

16. Steve.Museum Exploring Applications of Social Tagging for Museums Founded in 2005 2006 Institute for Museum and Library Services (IMLS) National Leadership Research Grant 2008 IMLS NLG Steve In Action 2008 IMLS NLG Research Grant T3: Text, Tags, Trust Open Source software supporting tagging in museums

17. Steve.Museum Exploring Applications of Social Tagging for Museums Founded in 2005 2006 Institute for Museum and Library Services (IMLS) National Leadership Research Grant 2008 IMLS NLG Steve In Action 2008 IMLS NLG Research Grant T3: Text, Tags, Trust Open Source software supporting tagging in museums http://tagger.steve.museum

18. www.steve.museum steve@steve.museum Why study social tagging?Every participant had a different answer Can tagging help users find art more easily? Can tagging change the way users look at and engage with art? Can tagging help museums understand what visitors see and understand?

19. Uncomfortable

20. John Singleton Copley, Portrait of Paul Revere

21. Tag: Jack Black

22. Winslow Homer, The Gulf Stream Tag: dolphins, leisure

23. Winslow Homer, The Gulf Stream Tag: dolphins, leisure Accuracy?

24. 2006-08 Research ResultsDownload the 10MB file at http://www.steve.museum

25. Some stats from the research 11 Participating Museums 1,784 Works of Art in the Research 93,380 Tags collected* 2,275 Users who tagged* *Derived from the sum of statistics from single and multi-institutional deployments

26. A Few Highlights Museum professionalsfound most tags useful 88% of tags were useful If you found this work using this term would you be surprised?

27. A Few Highlights Tags are different than museum documentation 86% of all tags not found in label copy 62% of distinct tags not in AAT 85% of distinct tags not in ULAN

28. A Few Highlights Tags are almost always useful when they are assigned two or more times

29. A Few Highlights Institutional Affiliation Matters Users invited to tag by a single institution were 4 times as productive Multi-Institution Tagger: 22 tags / user Single-Institution Tagger: 82 tags / user

30. Photo Credit ~warzauwynn (Flickr) Putting Steveto work

31. Steve in Action Funded in 2008 by the IMLS A Few Project Goals Make Social Tagging Easy Generalize to all object collections Abstract Data for Tags Develop Innovative NewInterfaces Facilitate Cross-CollectionSearch / Browsing

32. Current Tagger Stats 18 Participating Institutions 65,708 Objects in the Tagger 427,624 Tags collected 4,159 Users who tagged

33. Steve in Action Features Simple Import (CSV, CDWA, Scraping) Hosted and Themable Data Collection Platform Powerful API Access Cut-n-Paste Tagging Widgets for Easy Integration

36. Drive user-experience with tagging Can tagging be fun?

45. LOST IN A CLOUD

46. Finding a Needle in the Haystack

48. Funded in 2008 by IMLS With the University of Maryland, and collaborative of museum partners Studying the relationships between social tags, scholarly text and resources, and the application of trust networks to improve access to museum collections.

49. Heirarchy for Tags

50. Can we create hierarchy automatically? Supporting Semantic Analysis…

51. TermProcessing Framework TokenStreamProcessor Performs an operation on a token TokenStreamPipeline A sequence of TokenStreamProcessors to apply in order Taggers A special TokenStreamProcessor that adds metadata to a term.

52. Token Stream May contain one or more terms/tags in sequence The fox jumped over the lazy dog. quick brown INPUT Tokenize

53. Token Stream Remove capitalization and punctuation the fox jumped over the lazy dog quick brown INPUT Tokenize RemoveCapitals RemovePunctuation

54. Token Stream Remove Stop Words fox jumped over lazy dog quick brown INPUT Tokenize RemoveCapitals RemovePunctuation RemoveStopWords

55. Token Stream Tag Part of Speech and Normalize morphology fox jump over lazy dog quick brown INPUT ADJ ADJ N V ADV ADJ N Tokenize RemoveCapitals RemovePunctuation RemoveStopWords PartOfSpeechTagger MergeMorphology TermContexts

56. Sample Chain import steve.proccessing as sp filter1 = sp.filters.RemoveWhiteSpace() filter2 = sp.filters.LowerCase() filter1.setInput([‘Star Wars’, ‘lightsaber’]) filter2.setInput(filter1) Filter2.getOutput() Result = [[‘starwars’], [‘lightsaber’]]

57. Raw Tags

58. Morphy Normalization Naïve Normalization yields 105,547 distinct terms Morphological Normalization yields 70,295 distinct terms 33% Reduction in the corpus

60. What About “New England” Idioms / lexicalize phrases are more difficult Heuristic comparison to Wikipedia Titles matched 46% (30% distinct) of multiword tags i.e. “Grapes of Wrath”, “Irish Wolfhound”, “Franco-Prussian War” *Klavans and Golbeck, 2010

61. Thank You! Questions?