Overview of my thesis in Wikipedia deletion discussions

1. Copyright 2011 Digital Enterprise Research Institute. All rights reserved.
Digital Enterprise Research Institute www.deri.ie
Enabling Networked Knowledge
Arguments about deleting Wikipedia content
Jodi Schneider
jschneider@pobox.com
Vendredi 19th April 2013
1
Télécom ParisTech

2. Is Wikipedia Sustainable?

3. Deletion threatens Wikipedia
• 1 in 4 new Wikipedia articles is deleted –
within minutes or hours
• Demotivating!
– 1 in 3 newcomers start by writing a new article
– 7X less likely to stay if their article is deleted!
• Can we support editor retention?

4. Ph.D. case study: argumentative dialogues
about deleting Wikipedia articles
• Goals:
– Understand collaboration & coordination
– Identify “pain points” & new IT support opportunities
• Approaches:
– Net-ethnography
• Interviews of community members
• Embedded participation
• Reading essays, policies, & written dialogues
• Analysing article history, user contributions
– Content analysis
• Departure point: grounded theory or existing categories. With multiple
annotators, iteratively refined annotation manual to achieve strong interannotator
agreement.
• Decision factors (WikiSym 2012)
• Walton’s argumentation schemes (CSCW 2013)
– Prototyping & iterative design
• Design (WikiSym 2012 demo)
• User study (reported in dissertation)

6. Corpus
• Article deletion dialogues
from English Wikipedia
started on a typical-volume day
• 72 dialogues (94 A4 pages)

7. Findings: pain points of
article deletion
• Article creators
• Novices visiting or newly joining Wikipedia
• No-consensus dialogues

8. Article creators
• Misunderstand policy
– “I do understand that articles on wikipedia need to be
sourced… it is due to have two [sources] once [our
website goes] live”
• Express high levels of emotion
– “To be honest it's been a real turn off adding articles
to WP and I don't think I will add articles again. So
smile and enjoy.”
• Learn from discussions
– “much as it would break my heart … it is perhaps
sensible that the piece is deleted.”
Net-ethnography
in 8th International Symposium on Wikis and Open Collaboration
(WikiSym 2012)

9. Novices’ arguments
• Structurally different to experts’ arguments
• More problematic arguments from novices
– Personal preference
– Requesting a favor
– Analogy to other cases
– No harm in keeping an article
– Large number of search engine hits
Argumentation schemes content analysis
in 16th ACM Conference on Computer-Supported
Cooperative Work and Social Computing (CSCW 2013)

10. No consensus discussions
“What works well is simply the community
agreeing on a verdict.”
Otherwise:
• Time-consuming & difficult to judge a case
• Same case may get raised repeatedly
• Emotional upset is more likely
– “messy”, “full of hate and pain” when overturned
Net-ethnography & interviews
in 8th International Symposium on Wikis and
Open Collaboration (WikiSym 2012)

11. Articulate criteria
Decision factors content analysis
in 8th International Symposium on Wikis and Open Collaboration
(WikiSym 2012)
4 Factors cover
– 91% of
comments
– 70% of
discussions
Factor Example (used to justify `keep')
Notability Anyone covered by another encyclopedic
reference is considered notable enough
for inclusion in Wikipedia.
Sources Basic information about this album at a
minimum is certainly verifiable, it's a
major label release, and a highly notable
band.
Maintenance …this article is savable but at its current
state, needs a lot of improvement.
Bias It is by no means spam (it does not
promote the products).
Other I'm advocating a blanket "hangon" for all
articles on newly- drafted players

12. Use criteria to augment interface
Prototype design (RDFa; custom ontology based on FOAF, SIOC)
in WikiSym 2012 Demos

13. 84% prefer our system
“Information is structured and I can quickly get an
overview of the key arguments.”
“The ability to navigate the comments made it a bit easier
to filter my mind set and to come to a conclusion.”
“It offers the structure needed to consider each factor
separately, thus making the decision easier. Also, the
number of comments per factor offers a quick indication
of the relevance and the deepness of the decision.”
Based on a formative evaluation user study with 20 novice users
in dissertation “Enabling reuse of arguments and opinions from online social disputes”