Todd Carpenter's presentation at the opening of the Alternative Assessment discussion forum in San Francisco, CA on 10/09/13

1. Do We Need
Standards for
Alternative Metrics?
What are we hoping to
accomplish?
Todd Carpenter, Executive Director, NISO
NISO Alternative Assessment Initiative
Discussion Forum #1
San Francisco, CA | October 9, 2013
Friday, October 11, 13

2. • Non-profit industry trade association accredited
by ANSI with 150+ members
• Mission of developing and maintaining technical
standards related to information, documentation,
discovery and distribution of published materials
and media
• Volunteer driven organization: 400+ spread out
across the world
• Responsible in various wasy for standards like
ISSN, DOI, Dublin Core metadata, DAISY digital
talking books, OpenURL, MARC records, and ISBN
About
Friday, October 11, 13

3. We stand on the
shoulders of giants...
Friday, October 11, 13

4. The Mighty Citation
Friday, October 11, 13

5. Impact factor
Friday, October 11, 13

6. Thomson-Reuters
IMPACT FACTOR
Friday, October 11, 13

7. Two Stanford Students
Friday, October 11, 13

8. They wrote a paper
Friday, October 11, 13

9. They started a company in
someone’s garage
Photo:AP
Friday, October 11, 13

10. Friday, October 11, 13

11. Citations are awesome, aren’t they?
A perfect solution to all the world’s
problems, no?
A simple conclusion
can be drawn
Friday, October 11, 13

12. “What Des-Cartes did was a
good step.You have added
much several ways, &
especially in taking ye
colours of thin plates into
philosophical consideration. If
I have seen further it is by
standing on ye shoulders of
Giants."
Back to Sir Isaac Newton
Friday, October 11, 13

13. Citations
aren’t
always what
we think
they are
Caveat emptor
Friday, October 11, 13

14. Another problem with citations
Friday, October 11, 13

15. So where to from here?
Friday, October 11, 13

16. Define altmetrics,
please
Friday, October 11, 13

17. Usage Based Metrics
Friday, October 11, 13

18. Clickstream Analysis
Friday, October 11, 13

19. Social Media Metrics
Image: Danny Brown,The State of Social Media Marketing 2012
Friday, October 11, 13

20. Clickstream data yields high-
resolution maps of science
Johan Bollen, HerbertVan de Sompel, et al.
PLoS One, February 2009
Clickstream network analysis
Friday, October 11, 13

21. Clickstream data yields high-
resolution maps of science
Johan Bollen, HerbertVan de Sompel, et al.
PLoS One, February 2009
Clickstream network analysis
Friday, October 11, 13

22. Behavioral Metrics
Friday, October 11, 13

23. Multivariate analysis
Friday, October 11, 13

24. Sentiment Analysis
Friday, October 11, 13

25. Image: Domenico, Caron, Davis, et al.
Citation of non-traditional content
Citations to new
forms of
communication
aren’t as simple
as one might
think.
Define a use of
this?
Friday, October 11, 13

26. We’ve got
what, now how?
Friday, October 11, 13

27. Image: Flickr user karindalziel
What can be measured?
Friday, October 11, 13

28. How do we measure
things?
Friday, October 11, 13

29. At what granularity do
we measure things?
Friday, October 11, 13

30. Which measurements
are valuable?
Friday, October 11, 13

31. How do we ensure the
validity of those
measurements?
Friday, October 11, 13

32. How to
(or even if we should)
deal with other issues
such as gaming?
Friday, October 11, 13

33. Issues we face
with alternative
metrics
Friday, October 11, 13

34. Basic Definitions
(So we are all talking
about the same thing)
Friday, October 11, 13

35. Comparison across providers
Source:
Scott
Chamberlain,
Consuming
Article-­‐Level
Metrics:
Observations
And
Lessons
From
Comparing
Aggregator
Provider
Data,
Information
Standards
Quarterly,
Summer
2013,
Vol
25,
Issue
2.
Friday, October 11, 13

36. Consistency in what is counted
Source:
Scott
Chamberlain,
Consuming
Article-­‐Level
Metrics:
Observations
And
Lessons
From
Comparing
Aggregator
Provider
Data,
Information
Standards
Quarterly,
Summer
2013,
Vol
25,
Issue
2.
Friday, October 11, 13

37. Authorship
-----
Disambiguation
&
Contributor Roles
Friday, October 11, 13

38. Open exchange of
component data
Friday, October 11, 13

39. TRUST
Friday, October 11, 13

40. What is NISO trying to do?
Friday, October 11, 13

41. Friday, October 11, 13

42. Friday, October 11, 13

43. Alterna(ve
Assessment
Ini(a(ve
Phase
1
Mee(ngs
October
9,
2013
-­‐
San
Francisco,
CA
December
11,
2013
-­‐
Washington,
DC
January
23-­‐24
-­‐
Philadelphia,
PA
Phase
1
report
expected
in
May
2014
Friday, October 11, 13

44. Alterna(ve
Assessment
Ini(a(ve
Phase
2
Presenta(ons
of
report
(June
2014)
Priori(za(on
Effort
(June
-­‐
Aug,
2014)
Project
approval
(Sept
2014)
Working
group
forma(on
(Oct
2014)
Consensus
Development
(Nov
2014
-­‐
Dec
2015)
Trial
Use
Period
(Dec
15
-­‐
Mar
16)
Publica(on
of
final
recommenda(ons
(Jun
16)
Friday, October 11, 13

45. Criticism
Friday, October 11, 13

46. Why now?
Isn’t it too soon?
Can NISO to do this?
Friday, October 11, 13

47. How soon is now?
Friday, October 11, 13

48. These things take time
Friday, October 11, 13

49. It will be hard to change
Friday, October 11, 13

50. Friday, October 11, 13

51. “All I see now is
blonde, brunette,
redhead”
Friday, October 11, 13

52. One Very Last Thing…
Image from Will Lion: http://www.flickr.com/photos/will-lion/2646213692/
Friday, October 11, 13

53. Thank you!
Todd Carpenter, Executive Director
tcarpenter@niso.org
National Information Standards Organization (NISO)
3600 Clipper Mill Road, Suite 302
Baltimore, MD 21211 USA
+1 (301) 654-2512
www.niso.org
Friday, October 11, 13