SlideShare a Scribd company logo
1 of 11
Download to read offline
Judicious use of custom
development in an open source
component architecture
Sky Bristol1
Natalie Latysh1
Dell Long2
Steve Tekell2
Jeff Allen2
U.S. Department of the Interior
U.S. Geological Survey
1. USGS Core Science Analytics,
Synthesis and Libraries
2. USGS Fort Collins Science
Center
AGU Fall Meeting 2014
Talking Points
• ScienceBase – inspiration and a brief history
• Development methodology
• Information architecture
• Cyberinfrastructure architecture
• API first
• Lessons learned and motivators for future
• Open invitation (and invite us to your party
too)
AGU Fall Meeting 2014
myUSGS Data
Explorer/Data
Uploader
Scientific Data
Catalog/Compr
ehensive
Science Catalog
ScienceBase 1.0
and then 2.0
ScienceBase-
Constellation
Collaborative
tools and simple
file upload
Metadata
Cataloging &
Research Item
Concept
Digital Repository
& Research Item
Faceting
Federation of
Disparate Data
Schemas
“In the research process, we need more than just a big catalog of data. We need all
of the other important information connected to our work – published papers,
manuscripts, software, and information about people, labs, projects, and others in
our field.”
Inspiration and History
AGU Fall Meeting 2014
Agile Development
AGU Fall Meeting 2014
Flexible Information Architecture
AGU Fall Meeting 2014
Cyberinfrastructure Overview
AGU Fall Meeting 2014
Manage and Contribute
Register
Find it at the USGS
Find it at the DOI
Find it at Data.gov
Find it in a Data.gov community
(climate.data.gov)
Find it in other communities
(GeothermalData.org)
Open Data and USGIN Methods (“abridged”)
AGU Fall Meeting 2014
API First
AGU Fall Meeting 2014
Lessons Learned
• We encounter some new type of data just about every
week and need to account for a lot more variability.
• It’s pretty expensive to run an institutional repository,
but we have it to do.
• Baking security in from the ground up adds quite a few
constraints to the ideal architecture.
• We can’t please everyone all the time, but if we keep
building great platforms then people come back.
• APIs (and everything else, for that matter) need to be
driven by real, specific use cases to be usable.
• R&D, engineering, and operations are all different and
need different management methods.
AGU Fall Meeting 2014
Community Framework and
Platform vs. Agency System
AGU Fall Meeting 2014
Questions
www.sciencebase.gov
sciencebase@usgs.gov
Natalie Latysh (nlatysh@usgs.gov)
Sky Bristol (sbristol@usgs.gov)

More Related Content

What's hot

20130525 toledo-odin-sergio ruiz
20130525 toledo-odin-sergio ruiz20130525 toledo-odin-sergio ruiz
20130525 toledo-odin-sergio ruizFESABID
 
Science base usage analysis - AGU2016 - in21d08
Science base usage analysis - AGU2016 - in21d08Science base usage analysis - AGU2016 - in21d08
Science base usage analysis - AGU2016 - in21d08Sky Bristol
 
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...MongoDB
 
Hack reduce introduction
Hack reduce introductionHack reduce introduction
Hack reduce introductionmontrealouvert
 
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speech
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speechBig Data Applied, Data Warehouse Institute St. Louis December 2013 speech
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speechDavid Strom
 
Top 5 AI and Deep Learning Stories - November 9, 2018
Top 5 AI and Deep Learning Stories - November 9, 2018Top 5 AI and Deep Learning Stories - November 9, 2018
Top 5 AI and Deep Learning Stories - November 9, 2018NVIDIA
 
Guest Lecture on Big Data in Business,
Guest Lecture on Big Data in Business, Guest Lecture on Big Data in Business,
Guest Lecture on Big Data in Business, saravana krishnamurthy
 
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Redis Labs
 
Big data – An Introduction, July 2013
Big data – An Introduction, July 2013Big data – An Introduction, July 2013
Big data – An Introduction, July 2013Peter Morgan
 
Infrastructure - a journey from datacentres to cloud
Infrastructure - a journey from datacentres to cloudInfrastructure - a journey from datacentres to cloud
Infrastructure - a journey from datacentres to cloudEqual Experts
 
Business Insight
Business InsightBusiness Insight
Business InsightMicrosoft
 
#19sverificado : How Data Analytics helped put order in chaos in Mexico's ...
#19sverificado :  How Data Analytics helped  put order in chaos in  Mexico's ...#19sverificado :  How Data Analytics helped  put order in chaos in  Mexico's ...
#19sverificado : How Data Analytics helped put order in chaos in Mexico's ...Gustavo Rene Antunez
 
Reproducibility and Dataverse
Reproducibility and DataverseReproducibility and Dataverse
Reproducibility and Dataversephilipdurbin
 
Cassandra for bay area big data developer
Cassandra for bay area big data developerCassandra for bay area big data developer
Cassandra for bay area big data developer19scottmiller
 
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...exponential-inc
 
Big data perspective solution & technology
Big data perspective solution & technologyBig data perspective solution & technology
Big data perspective solution & technologyPankaj Khattar
 
#4 FAIR - Keith Russell
#4 FAIR - Keith Russell #4 FAIR - Keith Russell
#4 FAIR - Keith Russell ARDC
 

What's hot (20)

20130525 toledo-odin-sergio ruiz
20130525 toledo-odin-sergio ruiz20130525 toledo-odin-sergio ruiz
20130525 toledo-odin-sergio ruiz
 
Science base usage analysis - AGU2016 - in21d08
Science base usage analysis - AGU2016 - in21d08Science base usage analysis - AGU2016 - in21d08
Science base usage analysis - AGU2016 - in21d08
 
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...
MongoDB World 2019: Hands-on with an open source, serverless Data Warehouse f...
 
Hack reduce introduction
Hack reduce introductionHack reduce introduction
Hack reduce introduction
 
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speech
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speechBig Data Applied, Data Warehouse Institute St. Louis December 2013 speech
Big Data Applied, Data Warehouse Institute St. Louis December 2013 speech
 
Top 5 AI and Deep Learning Stories - November 9, 2018
Top 5 AI and Deep Learning Stories - November 9, 2018Top 5 AI and Deep Learning Stories - November 9, 2018
Top 5 AI and Deep Learning Stories - November 9, 2018
 
Guest Lecture on Big Data in Business,
Guest Lecture on Big Data in Business, Guest Lecture on Big Data in Business,
Guest Lecture on Big Data in Business,
 
ieee cloud 2015 keynote talk
ieee cloud 2015 keynote talkieee cloud 2015 keynote talk
ieee cloud 2015 keynote talk
 
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
Building a Multi-dimensional Analytics Engine with RedisGraph by Matthew Goos...
 
Big data – An Introduction, July 2013
Big data – An Introduction, July 2013Big data – An Introduction, July 2013
Big data – An Introduction, July 2013
 
Infrastructure - a journey from datacentres to cloud
Infrastructure - a journey from datacentres to cloudInfrastructure - a journey from datacentres to cloud
Infrastructure - a journey from datacentres to cloud
 
Business Insight
Business InsightBusiness Insight
Business Insight
 
#19sverificado : How Data Analytics helped put order in chaos in Mexico's ...
#19sverificado :  How Data Analytics helped  put order in chaos in  Mexico's ...#19sverificado :  How Data Analytics helped  put order in chaos in  Mexico's ...
#19sverificado : How Data Analytics helped put order in chaos in Mexico's ...
 
Reproducibility and Dataverse
Reproducibility and DataverseReproducibility and Dataverse
Reproducibility and Dataverse
 
Cassandra for bay area big data developer
Cassandra for bay area big data developerCassandra for bay area big data developer
Cassandra for bay area big data developer
 
Tracking data lineage at Stitch Fix
Tracking data lineage at Stitch FixTracking data lineage at Stitch Fix
Tracking data lineage at Stitch Fix
 
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (In...
 
Big data perspective solution & technology
Big data perspective solution & technologyBig data perspective solution & technology
Big data perspective solution & technology
 
Books neended
Books neendedBooks neended
Books neended
 
#4 FAIR - Keith Russell
#4 FAIR - Keith Russell #4 FAIR - Keith Russell
#4 FAIR - Keith Russell
 

Viewers also liked

Self learning material
Self learning materialSelf learning material
Self learning materialSangam Singh
 
Pgt432 PBLproject
Pgt432 PBLprojectPgt432 PBLproject
Pgt432 PBLprojectourslides
 
2009年中国数学会学术年会与会者通讯录
2009年中国数学会学术年会与会者通讯录2009年中国数学会学术年会与会者通讯录
2009年中国数学会学术年会与会者通讯录Xu jiakon
 
Доски объявлений
Доски объявленийДоски объявлений
Доски объявленийAndreygold
 
Microsoft opensource
Microsoft opensourceMicrosoft opensource
Microsoft opensourceCDS
 
Microsoft opensource
Microsoft opensourceMicrosoft opensource
Microsoft opensourceCDS
 
Enfermedades respiratorias en niños de 0 a un
Enfermedades respiratorias en niños de 0 a unEnfermedades respiratorias en niños de 0 a un
Enfermedades respiratorias en niños de 0 a uneveryn
 
懷生國中與綠色教室
懷生國中與綠色教室懷生國中與綠色教室
懷生國中與綠色教室family
 
Airiti books user_guide
Airiti books user_guideAiriti books user_guide
Airiti books user_guideairitiBooks
 
ABC's of Talent Selection
ABC's of Talent Selection ABC's of Talent Selection
ABC's of Talent Selection Kyle Deweerdt
 

Viewers also liked (20)

Self learning material
Self learning materialSelf learning material
Self learning material
 
Pgt432 PBLproject
Pgt432 PBLprojectPgt432 PBLproject
Pgt432 PBLproject
 
2009年中国数学会学术年会与会者通讯录
2009年中国数学会学术年会与会者通讯录2009年中国数学会学术年会与会者通讯录
2009年中国数学会学术年会与会者通讯录
 
Monografia de cuapiaxtla
Monografia de cuapiaxtlaMonografia de cuapiaxtla
Monografia de cuapiaxtla
 
Communicating health and safety messages
Communicating health and safety messagesCommunicating health and safety messages
Communicating health and safety messages
 
Доски объявлений
Доски объявленийДоски объявлений
Доски объявлений
 
Microsoft opensource
Microsoft opensourceMicrosoft opensource
Microsoft opensource
 
Microsoft opensource
Microsoft opensourceMicrosoft opensource
Microsoft opensource
 
Tales2 go
Tales2 goTales2 go
Tales2 go
 
Enfermedades respiratorias en niños de 0 a un
Enfermedades respiratorias en niños de 0 a unEnfermedades respiratorias en niños de 0 a un
Enfermedades respiratorias en niños de 0 a un
 
懷生國中與綠色教室
懷生國中與綠色教室懷生國中與綠色教室
懷生國中與綠色教室
 
Airiti books user_guide
Airiti books user_guideAiriti books user_guide
Airiti books user_guide
 
Hartstichting 2009
Hartstichting 2009Hartstichting 2009
Hartstichting 2009
 
Agriprom Dairy
Agriprom DairyAgriprom Dairy
Agriprom Dairy
 
ABC's of Talent Selection
ABC's of Talent Selection ABC's of Talent Selection
ABC's of Talent Selection
 
Accessible Gardening ~ Surf Coast Shire Council
Accessible Gardening ~ Surf Coast Shire CouncilAccessible Gardening ~ Surf Coast Shire Council
Accessible Gardening ~ Surf Coast Shire Council
 
Alternativa #somalternativa
Alternativa #somalternativaAlternativa #somalternativa
Alternativa #somalternativa
 
Binder1
Binder1Binder1
Binder1
 
Pravdepodobnost
PravdepodobnostPravdepodobnost
Pravdepodobnost
 
Sekai design impact
Sekai design impactSekai design impact
Sekai design impact
 

Similar to Judicious use of custom development in an open source component architecture

Working towards Sustainable Software for Science (an NSF and community view)
Working towards Sustainable Software for Science (an NSF and community view)Working towards Sustainable Software for Science (an NSF and community view)
Working towards Sustainable Software for Science (an NSF and community view)Daniel S. Katz
 
Tools für das Management von Forschungsdaten
Tools für das Management von ForschungsdatenTools für das Management von Forschungsdaten
Tools für das Management von ForschungsdatenHeinz Pampel
 
Using splunk for_big_data
Using splunk for_big_dataUsing splunk for_big_data
Using splunk for_big_dataAccenture
 
Big Data Curricula at the UW eScience Institute, JSM 2013
Big Data Curricula at the UW eScience Institute, JSM 2013Big Data Curricula at the UW eScience Institute, JSM 2013
Big Data Curricula at the UW eScience Institute, JSM 2013University of Washington
 
Transforming Networking within ESIP using ResearchBit
Transforming Networking within ESIP using ResearchBitTransforming Networking within ESIP using ResearchBit
Transforming Networking within ESIP using ResearchBitErin Robinson
 
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...datacite
 
What is eScience, and where does it go from here?
What is eScience, and where does it go from here?What is eScience, and where does it go from here?
What is eScience, and where does it go from here?Daniel S. Katz
 
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...hsuleslie
 
Introduction to UC San Diego’s Integrated Digital Infrastructure
Introduction to UC San Diego’s Integrated Digital InfrastructureIntroduction to UC San Diego’s Integrated Digital Infrastructure
Introduction to UC San Diego’s Integrated Digital InfrastructureLarry Smarr
 
20160414 23 Research Data Things
20160414 23 Research Data Things20160414 23 Research Data Things
20160414 23 Research Data ThingsKatina Toufexis
 
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...Kerstin Lehnert
 
SGCI Science Gateways Landscape in North America
SGCI Science Gateways Landscape in North AmericaSGCI Science Gateways Landscape in North America
SGCI Science Gateways Landscape in North AmericaSandra Gesing
 
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...It is not just about open or closed, it is about control. - Mark Hahnel, Figs...
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...Repository Fringe
 
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...SURFevents
 
2016 Ocean Sciences Meeting tutorial
2016 Ocean Sciences Meeting tutorial2016 Ocean Sciences Meeting tutorial
2016 Ocean Sciences Meeting tutorialJosh Young
 
Global registries initiative frumkin omodei
Global registries initiative frumkin omodeiGlobal registries initiative frumkin omodei
Global registries initiative frumkin omodeiASIS&T
 
Guy avoiding-dat apocalypse
Guy avoiding-dat apocalypseGuy avoiding-dat apocalypse
Guy avoiding-dat apocalypseENUG
 
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science, a Digital Research...
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science,  a Digital Research...Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science,  a Digital Research...
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science, a Digital Research...Carole Goble
 

Similar to Judicious use of custom development in an open source component architecture (20)

Working towards Sustainable Software for Science (an NSF and community view)
Working towards Sustainable Software for Science (an NSF and community view)Working towards Sustainable Software for Science (an NSF and community view)
Working towards Sustainable Software for Science (an NSF and community view)
 
Tools für das Management von Forschungsdaten
Tools für das Management von ForschungsdatenTools für das Management von Forschungsdaten
Tools für das Management von Forschungsdaten
 
Using splunk for_big_data
Using splunk for_big_dataUsing splunk for_big_data
Using splunk for_big_data
 
Big Data Curricula at the UW eScience Institute, JSM 2013
Big Data Curricula at the UW eScience Institute, JSM 2013Big Data Curricula at the UW eScience Institute, JSM 2013
Big Data Curricula at the UW eScience Institute, JSM 2013
 
Transforming Networking within ESIP using ResearchBit
Transforming Networking within ESIP using ResearchBitTransforming Networking within ESIP using ResearchBit
Transforming Networking within ESIP using ResearchBit
 
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...
2013 DataCite Summer Meeting - DOIs and Supercomputing (Terry Jones - Oak Rid...
 
What is eScience, and where does it go from here?
What is eScience, and where does it go from here?What is eScience, and where does it go from here?
What is eScience, and where does it go from here?
 
BLC & Digital Science: Mark Hahnel, Figshare
BLC & Digital Science: Mark Hahnel, FigshareBLC & Digital Science: Mark Hahnel, Figshare
BLC & Digital Science: Mark Hahnel, Figshare
 
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...
Sediment Experimentalist Network (SEN): Sharing and reusing methods and data ...
 
Introduction to UC San Diego’s Integrated Digital Infrastructure
Introduction to UC San Diego’s Integrated Digital InfrastructureIntroduction to UC San Diego’s Integrated Digital Infrastructure
Introduction to UC San Diego’s Integrated Digital Infrastructure
 
20160414 23 Research Data Things
20160414 23 Research Data Things20160414 23 Research Data Things
20160414 23 Research Data Things
 
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...
IEDA: Making Small Data BIG Through Interdisciplinary Partnerships Among Long...
 
SGCI Science Gateways Landscape in North America
SGCI Science Gateways Landscape in North AmericaSGCI Science Gateways Landscape in North America
SGCI Science Gateways Landscape in North America
 
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...It is not just about open or closed, it is about control. - Mark Hahnel, Figs...
It is not just about open or closed, it is about control. - Mark Hahnel, Figs...
 
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...
Interactive and collaborative AI for biodiversity monitoring and beyond - JWK...
 
2016 Ocean Sciences Meeting tutorial
2016 Ocean Sciences Meeting tutorial2016 Ocean Sciences Meeting tutorial
2016 Ocean Sciences Meeting tutorial
 
Data Science and Urban Science @ UW
Data Science and Urban Science @ UWData Science and Urban Science @ UW
Data Science and Urban Science @ UW
 
Global registries initiative frumkin omodei
Global registries initiative frumkin omodeiGlobal registries initiative frumkin omodei
Global registries initiative frumkin omodei
 
Guy avoiding-dat apocalypse
Guy avoiding-dat apocalypseGuy avoiding-dat apocalypse
Guy avoiding-dat apocalypse
 
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science, a Digital Research...
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science,  a Digital Research...Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science,  a Digital Research...
Can’t Pay, Won’t Pay, Don’t Pay: Delivering open science, a Digital Research...
 

More from Sky Bristol

CanyonViews.pptx
CanyonViews.pptxCanyonViews.pptx
CanyonViews.pptxSky Bristol
 
USGS research infrastructure - AGU2016 - in13 e
USGS research infrastructure - AGU2016 - in13 eUSGS research infrastructure - AGU2016 - in13 e
USGS research infrastructure - AGU2016 - in13 eSky Bristol
 
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...Sky Bristol
 
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...Standards promote interoperability of USGS data on the U.S. Geoscience Inform...
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...Sky Bristol
 
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...Sky Bristol
 
ScienceBase and CINERGI - thoughts
ScienceBase and CINERGI - thoughtsScienceBase and CINERGI - thoughts
ScienceBase and CINERGI - thoughtsSky Bristol
 
ScienceBase Architecture - Access Methods
ScienceBase Architecture - Access MethodsScienceBase Architecture - Access Methods
ScienceBase Architecture - Access MethodsSky Bristol
 

More from Sky Bristol (7)

CanyonViews.pptx
CanyonViews.pptxCanyonViews.pptx
CanyonViews.pptx
 
USGS research infrastructure - AGU2016 - in13 e
USGS research infrastructure - AGU2016 - in13 eUSGS research infrastructure - AGU2016 - in13 e
USGS research infrastructure - AGU2016 - in13 e
 
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...
Ocean Biogeographic Information System - for NOPP Biodiversity Ad Hoc Working...
 
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...Standards promote interoperability of USGS data on the U.S. Geoscience Inform...
Standards promote interoperability of USGS data on the U.S. Geoscience Inform...
 
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...
Big Data R&D Strategy - Ensure the long term sustainability, access, and deve...
 
ScienceBase and CINERGI - thoughts
ScienceBase and CINERGI - thoughtsScienceBase and CINERGI - thoughts
ScienceBase and CINERGI - thoughts
 
ScienceBase Architecture - Access Methods
ScienceBase Architecture - Access MethodsScienceBase Architecture - Access Methods
ScienceBase Architecture - Access Methods
 

Recently uploaded

VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesKrzysztofKkol1
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecturerahul_net
 
Data modeling 101 - Basics - Software Domain
Data modeling 101 - Basics - Software DomainData modeling 101 - Basics - Software Domain
Data modeling 101 - Basics - Software DomainAbdul Ahad
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jNeo4j
 
Zer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfZer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfmaor17
 
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...OnePlan Solutions
 
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...OnePlan Solutions
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfRTS corp
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?Alexandre Beguel
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slidesvaideheekore1
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxRTS corp
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogueitservices996
 
Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsJean Silva
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...Bert Jan Schrijver
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolsosttopstonverter
 
Best Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITBest Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITmanoharjgpsolutions
 
Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...Rob Geurden
 
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxUI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxAndreas Kunz
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonApplitools
 

Recently uploaded (20)

VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecture
 
Data modeling 101 - Basics - Software Domain
Data modeling 101 - Basics - Software DomainData modeling 101 - Basics - Software Domain
Data modeling 101 - Basics - Software Domain
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
 
Zer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfZer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdf
 
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
 
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slides
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogue
 
Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero results
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration tools
 
Best Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITBest Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh IT
 
Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...
 
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxUI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
 

Judicious use of custom development in an open source component architecture

  • 1. Judicious use of custom development in an open source component architecture Sky Bristol1 Natalie Latysh1 Dell Long2 Steve Tekell2 Jeff Allen2 U.S. Department of the Interior U.S. Geological Survey 1. USGS Core Science Analytics, Synthesis and Libraries 2. USGS Fort Collins Science Center
  • 2. AGU Fall Meeting 2014 Talking Points • ScienceBase – inspiration and a brief history • Development methodology • Information architecture • Cyberinfrastructure architecture • API first • Lessons learned and motivators for future • Open invitation (and invite us to your party too)
  • 3. AGU Fall Meeting 2014 myUSGS Data Explorer/Data Uploader Scientific Data Catalog/Compr ehensive Science Catalog ScienceBase 1.0 and then 2.0 ScienceBase- Constellation Collaborative tools and simple file upload Metadata Cataloging & Research Item Concept Digital Repository & Research Item Faceting Federation of Disparate Data Schemas “In the research process, we need more than just a big catalog of data. We need all of the other important information connected to our work – published papers, manuscripts, software, and information about people, labs, projects, and others in our field.” Inspiration and History
  • 4. AGU Fall Meeting 2014 Agile Development
  • 5. AGU Fall Meeting 2014 Flexible Information Architecture
  • 6. AGU Fall Meeting 2014 Cyberinfrastructure Overview
  • 7. AGU Fall Meeting 2014 Manage and Contribute Register Find it at the USGS Find it at the DOI Find it at Data.gov Find it in a Data.gov community (climate.data.gov) Find it in other communities (GeothermalData.org) Open Data and USGIN Methods (“abridged”)
  • 8. AGU Fall Meeting 2014 API First
  • 9. AGU Fall Meeting 2014 Lessons Learned • We encounter some new type of data just about every week and need to account for a lot more variability. • It’s pretty expensive to run an institutional repository, but we have it to do. • Baking security in from the ground up adds quite a few constraints to the ideal architecture. • We can’t please everyone all the time, but if we keep building great platforms then people come back. • APIs (and everything else, for that matter) need to be driven by real, specific use cases to be usable. • R&D, engineering, and operations are all different and need different management methods.
  • 10. AGU Fall Meeting 2014 Community Framework and Platform vs. Agency System
  • 11. AGU Fall Meeting 2014 Questions www.sciencebase.gov sciencebase@usgs.gov Natalie Latysh (nlatysh@usgs.gov) Sky Bristol (sbristol@usgs.gov)