O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Metrics standardization. Wikimedia Research & Data Showcase, March 2014

Slides from the talk on metrics standardization from the March 2014 Wikimedia Research & Data Showcase.

  • Seja o primeiro a comentar

Metrics standardization. Wikimedia Research & Data Showcase, March 2014

  1. 1. Metrics standardization Dario Taraborelli • Aaron Halfaker Wikimedia Research and Data showcase March 2014
  2. 2. summer 2013
  3. 3. cohort-level metrics
  4. 4. cohort-level metrics project-level metrics
  5. 5. project-level metrics
  6. 6. project-level metrics
  7. 7. ENWIKI New Editors / day 1D: 21% 30D: 18% YTD: 20% Editor engagement vital signs key performance indicators for user engagement, community and content growth aggregated daily / weekly / monthly for every single Wikimedia project https://www.mediawiki.org/wiki/Analytics/Epics/Editor_Engagement_Vital_Signs 02/01: 1240 • summer 2014
  8. 8. Key metrics New users Community Content Curation Newly registered users New editors Productive new editors Surviving new editors ... Editors Active editors Very active editors IP editors Bots Page creators ... Edits Bot edits Uploads Pages ... Page deletions Reverts ... https://meta.wikimedia.org/wiki/Research:Metrics_standardization
  9. 9. Relevant Measure quantities that describe important phenomena Replicable Make research easily reproducible and verifiable Transparent Provide formal specifications, remove ambiguity Consistent Replace proprietary, ad-hoc metric definitions; compare apples to apples Robust Make metrics replicable via multiple data sources at any point in time Granular Computable at different time scales Rationale
  10. 10. Anatomy of a metric 1. specification
  11. 11. Anatomy of a metric 2. visualizations registration time Activation Trial Survival
  12. 12. Anatomy of a metric 2. visualizations New editor Productive new editor Surviving new editor
  13. 13. Anatomy of a metric 3. discussion
  14. 14. Anatomy of a metric 4. sensitivity analysis
  15. 15. Sensitivity analysis https://meta.wikimedia.org/wiki/Research:Productive_new_editor Does new editor productivity vary when we measure it over the first day or the first week?
  16. 16. Sensitivity analysis https://meta.wikimedia.org/wiki/Research:New_editor Does it really matter to limit new editor activation to main namespace edits only?
  17. 17. Sensitivity analysis https://meta.wikimedia.org/wiki/Research:Surviving_new_editor Does the length of the trial and survival period affect the measurement of new editor survival?
  18. 18. Why does this matter at all? 1. Data exploration “Newly registered users on German and Dutch Wikipedia have a higher activation rate than newbies on English Wikipedia” “Spanish Wikipedia adds every day twice as many new editors than German Wikipedia, despite having only half its new user activation rate” 2. Natural experiments “A change in abuse filter rules on the Italian Wikipedia significantly increased new editor survival”
  19. 19. Metric specification Sensitivity analysis Parameter recommendation Release Evaluation Evaluation
  20. 20. Metric specification Sensitivity analysis Parameter recommendation Release Evaluation Evaluation feedback
  21. 21. Questions? dario@wikimedia.org ahalfaker@wikimedia.org Read more https://meta.wikimedia.org/wiki/Research:Metrics_standardization Image credits W.Wood (1839) Index Entomologicus http://dx.doi.org/10.5962/bhl.title.12503

    Seja o primeiro a comentar

    Entre para ver os comentários

  • keita.bando

    Mar. 21, 2014

Slides from the talk on metrics standardization from the March 2014 Wikimedia Research & Data Showcase.

Vistos

Vistos totais

1.087

No Slideshare

0

De incorporações

0

Número de incorporações

22

Ações

Baixados

2

Compartilhados

0

Comentários

0

Curtir

1

×