It summit dataverse-bigdata-mercecrosas

1. A data repository system for sharing and
archiving research data
A Solution for Publishing FAIR research data:
Findable,Accessible, Interoperable, Reusable
@dataverseorg @mercecrosas

2. But it’s self-correcting
Science isn’t broken
“It’s just a hell a lot
harder than we give it
credit for”

3. Self-correction in science?
“trust, but verify”
Data (and code) should be
shared for other
researchers to inspect, test
and reuse

4. Findable Accessible
Interoperable and
Reusable (FAIR)

6. Research Data Lifecycle
(collect, process, analyze, compute)
Publish and
Archive Data
Publish Research
Results
Data
Citation
Metadata
Access
Controls

7. http://dataverse.org
Harvard Dataverse:
Generic data repository open
to researchers world wide
http://dataverse.harvard.edu
>1500 dataverses
(15% from Harvard)
> 65,000 datasets
> 1.7 Million downloads
Biomedical Dataverse:
Support for biomedical
large-scale data (pilot phase)

8. Dataverse Now Dataverse Big
Up to 100 files per dataset
Up to ~ Gb per file
> 1000s files per dataset
> Gb per file
Large file directory
structure or
Streaming data source

9. Social Science Big Data
Dataverse Projects supporting Big Data
Integration with Data
Management Systems
+
+
at HMS
at UNC
Biomedical Big Data
• 1000s image files per data set
• Non-http batch upload
• Data files in directory structure
Cloud Dataverse
+ Swift Storage
(scalable, access to
computing)
A Billion
streaming
Geotweets