The document summarizes a pilot study that evaluated five measures for assessing the scholarly impact of data reuse: (1) number of citations of reused data, (2) quality of publications citing reused data, (3) breadth of disciplines and topics citing reused data, (4) size of the publication network stemming from reused data, and (5) number of downloads of reused data. The study analyzed a sample of 273 social science datasets and found that while some data was reused extensively, much more data was reused only a little or not at all according to the measures. The measures provided different perspectives on the impact of data reuse.

1. https://www.asis.org/rdap/
The impact of data reuse: a pilot
study of five measures
Kathleen Fear
April 5, 2013
LastName, Title

2. What is reuse impact?
• Scholarly contribution through producing data
– Recognized and rewarded through publications and
publication metrics
• Scholarly contribution through sharing data
– Recognized and rewarded through
…
?

3. What can we do to measure and communicate
the scholarly contribution a data producer
makes when their data is reused?

4. Pilot study of 5 measures
• Identify a set of social science datasets
• Find out how much and in what contexts they
have been reused
• Demonstrate a variety of measures
– Do they all come out the same?
– Or do different measures highlight different data?

5. Sample set: 273 studies
Release Date Processed vs. unprocessed
19%
30%
38%
2000 Processed
2001 Unprocessed
2002
81%
32%
Author Type
3% 3%
Single author
11%
Two or more authors
38%
Government
Non-governmental
institution
45%
Media organization

6. Reuse citations
• How many times has the data been reused?
• ICPSR Bibliography of Data-Related Literature
– Excluded: publications by study authors and
research team members, literature
reviews, commentary

7. Lots of data is
reused a little
Some data is
reused a lot
Even more is
reused not at all

8. Study ID Study Name Reuse count
National Comorbidity Survey: Baseline
6693 175
(NCS-1), 1990-1992
National Treatment Improvement
2884 32
Evaluation Study (NTIES), 1992-1997
Project on Policing Neighborhoods in
3160 Indianapolis, Indiana, and St. 34
Petersburg, Florida, 1996-1997
Hispanic Established Populations for the
Epidemiologic Studies of the Elderly,
2851 24
1993-1994: [Arizona, California,
Colorado, New Mexico, and Texas]
Drug Abuse Treatment Outcome Study
2258 19
(DATOS), 1991-1994: [United States]

10. How high-quality are
the data’s reuse
publications?

11. Secondary
Study ID Study Name
Impact
National Comorbidity Survey: Baseline
6693 83
(NCS-1), 1990-1992
Drug Abuse Treatment Outcome Study
2258 21
(DATOS), 1991-1994: [United States]
Hispanic Established Populations for the
Epidemiologic Studies of the Elderly,
2851 20
1993-1994: [Arizona, California,
Colorado, New Mexico, and Texas]
National Treatment Improvement
2884 19
Evaluation Study (NTIES), 1992-1997
Gambling Impact and Behavior Study,
2778 18
1997-1999: [United States]

12. How broadly or narrowly
is the data reused?

13. Diversity
• Variety, balance, disparity among reuse
publications + disparity between citing
disciplines and data discipline

14. DataID Study Title Diversity
National Organizations Survey (NOS),
3190 2.5000
1996-1997
Evaluation of the Gang Resistance
3337 Education and Training (GREAT) Program 2.2230
in the United States, 1995-1999
Police Stress and Domestic Violence in
2976 Police Families in Baltimore, Maryland, 2.1794
1997-1999
Aging, Status, and Sense of Control
3334 (ASOC), 1995, 1998, 2001 [United 2.0313
States]
Reintegrative Shaming Experiments
2993 2.0000
(RISE) in Australia, 1995-1999

15. How large is the
publication network
stemming from the data?

17. Downloaders
• How many individuals download the data?
• Unique users identified by email address or IP
address

18. Study ID Study Name Downloaders
National Comorbidity Survey: Baseline
6693 3787
(NCS-1), 1990-1992
World Values Surveys and European
2790 Values Surveys, 1981-1984, 1990-1993, 3393
and 1995-1997
Gambling Impact and Behavior Study,
2778 2637
1997-1999: [United States]
Alcohol and Drug Services Study (ADSS),
3088 2478
1996-1999: [United States]
3355 Recidivism of Prisoners Released in 1994 2209

19. Comparing metrics
Reuse Citations Sec. Impact Diversity Downloaders
6693 6693 3190 6693
3160 2258 3337 2778
2884 2851 2976 3088
2851 2884 3334 2833
2258 2778 3052 3334
2976 3385 2993 3337
2833 3160 3323 2258
2778 3337 2778 2976
3337 2833 3163 3212
3385 3023 3002 3002

20. Reuse count Downloaders
Secondary impact Diversity

21. Study Diver- Reuse Sec. Down-
Study Name
ID sity Count Impact loaders
Risk Factors for Violent
Victimization of
Women in a Major
3052 5 18 26 27
Northeastern City,
1990-1991 and 1996-
1997
Recidivism of
3355 Prisoners Released in 29 18 18 4
1994
Pennsylvania
3450 Sentencing Data, 30 14 6 29
1998

22. https://www.asis.org/rdap/
Thank you!
Questions?
Kathleen Fear
kfear@umich.edu
LastName, Title