Open Data on AWS

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Jed Sundwall, Global Open Data Lead
Open Data on AWS
https://opendata.aws

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Agenda
Overview of Open Data on AWS
How shared data on the cloud can accelerate research
Finding data shared on AWS
Sharing data on AWS
Chat with Ann Molin from Hack for Sweden

No Up Front Expense
Pay for what you Use
Improve Time to
Market & Agility
Scale Up and
Down
Self-Service
Infrastructure
AWS Cloud
Equipment
Resources and
Administration
Contracts Cost
Traditional
Infrastructure

Why does AWS care about open data?
Many AWS customers supply data
to the public to accelerate research
and product development.
Many AWS customers use data
shared on AWS to create new
products and services.

Companies want more value from their data
Complications:
Siloed approaches don’t work anymore
It’s too expensive and limiting to store data
on-premises
Data is:
Implication:
A new approach is needed to extract insights
and value
Growing
exponentially
From new
sources
Increasingly
diverse
Used by
many people
Analyzed by
many applications

“…data must be organized, well-documented,
consistently formatted, and error free. Cleaning the data
is often the most taxing part of data science, and is
frequently 80% of the work.”
— Data Driven by DJ Patil and Hilary Mason
Undifferentiated heavy lifting

Flipped data flow in the cloud
Traditional approach:
Move data to computing resources.
Cloud approach:
Move computing resources to data.
Amazon S3
Amazon
EC2
Amazon
EMR
Amazon
Athena

Cloud data lakes are the future
Customers want:
To eliminate data silos
To move to a single store, i.e. a data lake in the cloud
To store data securely in standard formats
To grow to any scale, with low costs
To analyze their data in a variety of ways
To have real-time analytics
To predict future outcomes
Data Lake

Sharing data in the cloud lets data
users spend more time on data
analysis rather than data acquisition.
https://opendata.aws

Advantages of sharing data in the cloud
Global community of users
Faster pace of research Lower cost of research
New services and tools

AWS Public Datasets
https://registry.opendata.aws

AWS Public Datasets
https://registry.opendata.aws/tag/satellite-imagery

Data at work

Graph by Drew Bollinger (@drewbo19) at Development Seed
Landsat on AWS

Monitoring at-risk bodies of water from space
The Blue Dot Observatory uses
Sentinel-2 satellite data on AWS to
monitor water bodies around the world.
“The cost to process one month of data
for about 7,000 bodies of water
currently in the system is 6 EUR. It is
possible to set up world-scale systems
with a shoestring budget.”
opendata.aws/bluedot

Opening high resolution elevation data
USGS shares over 10 trillion lidar point
cloud records from across the US on AWS.
“The democratization of elevation data
[…] promises to revolutionize approaches
to applications from flood forecasting and
geologic assessments to precision
agriculture and infrastructure
development.”
— Kevin Gallagher, USGS
opendata.aws/usgs-lidar

Facilitating over 31 million journeys made in London every day
When Transport for London opened up
access to its data, application developers
and researchers used it to create more
than 600 applications that provide
services to 42 percent of Londoners,
saving an up to estimated £130 million
per year.
opendata.aws/tfl

Finnish Meteorological Institute data on AWS
The Finnish Meteorological Institute
makes large scale weather and air quality
models available through the AWS Public
Dataset program.
registry.opendata.aws/hirlam
registry.opendata.aws/silam

Finnish Meteorological Institute data on AWS
0
10000
20000
30000
40000
50000
60000
70000
06/2017
08/2017
09/2017
11/2017
12/2017
02/2018
04/2018
05/2018
07/2018
09/2018
HIRLAM DOWNLOADS FROM DIFFERENT CHANNELS
FMI AWS Normalised Trendline of FMI Trendline of AWS

NauticAI – transforming the maritime industry
“Our system automatically receives
motion data, combines it with ship
location and external weather, and
securely feeds the data over an API to the
customer’s analytics system. This real-
time data stream gives the customer
information on how wind and waves
affect on-board navigation. Combined
with weather forecast data, it enables
prediction and optimization of
navigation.”
opendata.aws/nauticai

Finding data on AWS
Using the Registry of Open Data on AWS (RODA)

Registry of Open Data on AWS
https://registry.opendata.aws/

Registry of Open Data on AWS – Tags
https://registry.opendata.aws/tag/sustainability

Registry of Open Data on AWS – Usage examples
https://registry.opendata.aws/usage-examples

Registry of Open Data on AWS – How to contribute
https://github.com/awslabs/open-data-registry

Guide to sharing data on AWS
Sharing
Data on
AWS
Over 40 pages of insights on data
sharing and case studies from
customers including:
- Transport for London
- Canada’s Communications
Security Establishment
- The US Geological Survey
- The Allen Institute for Brain
Science
https://opendata.aws/guide-pdf

Thank you!
jed@amazon.com

Chat with:
Ann Molin
Head Project Manager at Hack for Sweden
hackforsweden.se

Open Data on AWS

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Open Data on AWS

Semelhante a Open Data on AWS (20)

Mais de Amazon Web Services

Mais de Amazon Web Services (20)

Open Data on AWS