Presentation at FLOSS community metrics workshop as part of the Community Leadership Summit co-located with O'Reilly OSCON.

1. Heartbeat: measuring installed base
by analyzing downloads
and
Scientific Software Network Map
James Howison
University of Texas at Austin

2. Downloads
• Great desire to measure something similar to
sales and/or market share
• Early focus on downloads but …
– A download is not a sale
– No direct reward
– Might be experimentation
– Strongly correlated with number of releases

3. Installed Base
• How many regular users does a piece of
software have?

5. Bibdesk daily downloads

6. Bibdesk installed base

7. What’s needed?
• High frequency data
• Some notification of new releases, or
• Some driver for frequent updates

8. Current work
• Focus on software work in science
– No convenient central repositories!
• Focusing on understanding what software is
used with what
– Complements, not dependencies
• Linking metrics from publications to runtimes,
and dependencies.

9. Paper
Software
Software
Software
Software Software
Software
Paper
Software
uses
depends on
uses
Indegree 2
Key
depends on
complements
complements
uses

10. Mentions in publications?

11. Types of mentions in publications
Mention Type Example
Cite to Publication … was calculated using biosys (Swofford & Selander 1981).
Cite to Project Name or
Website
… using the program Autodecay version 4.0.29 PPC
(Eriksson 1998).
Reference List has: ERIKSSON, T. 1998. Autodecay, vers.
4.0.29 Stockholm: Department of Botany.
Like Instrument … calculated by t-test using the Prism 3.0 software
(GraphPad Software, San Diego, CA, USA).
URL in text … freely available from http://www.cibiv.at/software/pda/
.
In-text name mention
only
… were analyzed using MapQTL (4.0) software.
Not even name
mentioned
… was carried out using software implemented in the Java
programming language.
@jameshowison DOI:
10.6084/m9.figshare.1146366

12. Types of Mentions
@jameshowison DOI:
10.6084/m9.figshare.1146366

13. Detecting complements
• http://scisoft-net-map.isri.cmu.edu/
• http://scisoft-net-map.isri.cmu.edu:7777/
• http://depsy.org/

14. Questions
• Ideas for discovering complements
– Software used with other software
• Anyone interested in mining publications (or
perhaps blogs etc) for software mentions
– Gold standard dataset at:
github.com/jameshowison/softcite