I will describe the relationship between the Dataverse repository and the Astronomy Community. The relationship began when Dataverse sought to solve some of the problems faced by astronomers who wish to share or preserve their data products. I will detail the feedback loop necessary between tool developers and research scientists to make this relationship successful.
General Principles of Intellectual Property: Concepts of Intellectual Proper...
Astronomy Data in the Dataverse by August Muench
1. Astronomy Data in the Dataverse
● In Scope: taking advantage of standardized file type to
enhance discoverability and curation of Astronomy data;
● Out of scope:
a. Outreach to Astronomy Community;
b. Connections to Literature and Article search.
2. array data
header
key : valueFITS File array data
header
key : valueFITS File array data
header
key : value
FITS Files
● Flexible Image Transport System
a. 1981, developed
b. 2008, Version 3
● Our catch all data container
a. Tables, Sparse Matrices, Images, Cubes, Photon lists
b. once FITS, always FITS(*)
3. array data
header
key : valueFITS File array data
header
key : valueFITS File array data
header
key : value
Dataverse
Dataset
Citation Metadata
(manual authoring)
Astronomy & Astrophysics Metadata
(automated extraction + curation)
field0 = [unique(key0:value, key1:value, )]
field1 = [unique(key2:value, null, … )]
field2 = array.type(key3:value, key4:value,...)
field3 = array.columns
...
fieldN = there is no value validation
4. array data
header
key : valueFITS File array data
header
key : valueFITS File array data
header
key : value
Suggested directions:
1. Retain File Level metadata (no aggregation);
2. Validate File Level metadata by completeness & content;
3. Expose full unparsed header via API;
4. Provide transmutable data objects (CSV/TSV tables);
5. Compressed image previews.
File Metadata
key0 : value
key1 : value
…
keyN :
validate
completeness
5. array data
header
key : valueFITS File array data
header
key : valueFITS File array data
header
key : value
Suggested directions:
1. Retain File Level metadata (no aggregation);
2. Validate File Level metadata by completeness & content;
3. Expose full unparsed header via API;
4. Provide transmutable data objects (CSV/TSV tables);
5. Compressed image previews and other data widgets.
File Metadata
key0 : value
key1 : value
…
keyN :
validate
completeness
6. array data
header
key : valueFITS File array data
header
key : valueFITS File array data
header
key : value
Suggested directions:
1. Retain File Level metadata (no aggregation);
2. Validate File Level metadata by completeness & content;
3. Expose full unparsed header via API;
4. Provide transmutable data objects (CSV/TSV tables);
5. Compressed image previews and other data widgets.
/api/access/datafile/$i
d/metadata/header
7. array data
header
key : valueFITS File array data
header
key : valueFITS File array data
header
key : value
Suggested directions:
1. Retain File Level metadata (no aggregation);
2. Validate File Level metadata by completeness & content;
3. Expose full unparsed header via API;
4. Provide transmutable data objects (CSV/TSV tables);
5. Compressed image previews and other data widgets.
array.tsv
array.csv
array.json
8. array data
header
key : valueFITS File array data
header
key : valueFITS File array data
header
key : value
Suggested directions:
1. Retain File Level metadata (no aggregation);
2. Validate File Level metadata by completeness & content;
3. Expose full unparsed header via API;
4. Provide transmutable data objects (CSV/TSV tables);
5. Compressed image previews and other data widgets.