The Finnish Meteorological Institute (FMI) has begun distributing some of its open data via Amazon Web Services (AWS) cloud platform. This includes weather data from the HIRLAM model covering Europe, as well as air quality data from the global SILAM model. FMI data on AWS is freely accessible through Amazon S3 buckets and includes near-real time and forecast data. Making FMI data available on AWS expands its potential audience and allows certain users like those needing full model datasets to more conveniently access and analyze the data.

1. Open Data Distributed on
Amazon’s Cloud Service
RoopeTervo, Rostislav Kouznetsov, Mikhail Sofiev, Mikko
Visa
Finnish Meteorological Institute

2. 2
Finnish Meteorological
Institute opened its data
FMI starts to open the
software
2013 2016
FMI join AWS Public
Dataset Program
2017

3. 4
• Finnish Meteorological Institute
opened its data in 2013.
• Basically everything that FMI has
property rights was opened.
• Both (near) real-time and historical and
climatological data.
• Data is provided in freely in machine
readable format.
https://en.ilmatieteenlaitos.fi/open-data

4. 5
…and
a little over
900 000 data downloads
per day
(10.4 req/s)
At the moment
about 11 700 users

5. 6
Data is valuable only when it reach
relevant audience
Maximal coverage requires several
different channels and services
 FMI joins Amazon’s Public Datasets

6. FMI OpenData on AWS
7
FMI OpenData is also distributed on Amazon Web Services (AWS)
Cloud platform
• Started in May 2017 as a pilot
• Hirlam surface and pressure levels in the first stage
• Especially convenient for users who need the whole model data
• i.e for post-processing or generating map visualizations
• Licence: CC BY 4.0

7. FMI Hirlam on AWS
8
• Content
• Hirlam Surface
• Hirlam Pressure Levels
• Coverage: Europe
• Grid resolution: 7,5 km
• Updates: 4 times a day
• Time range: 54 hours (from model run start)
• Time step: 1 hour
• Archive kept during the pilot
• Parameters: http://en.ilmatieteenlaitos.fi/open-data-on-aws-
s3

8. FMI Hirlam on AWS
9
• Access through buckets:
• Surface data: fmi-opendata-rcrhirlam-surface-grib
• Pressure level data: fmi-opendata-rcrhirlam-pressure-grib
• Browse bucket content:
• http://fmi-opendata-rcrhirlam-surface-grib.s3-website-eu-west-1.amazonaws.com/
• http://fmi-opendata-rcrhirlam-pressure-grib.s3-website-eu-west-1.amazonaws.com/
• Public Amazon SNS topics are available for every new object added to the Amazon
S3:
• arn:aws:sns:eu-west-1:916174725480:new-fmi-opendata-rcrhirlam-surface-grib
• arn:aws:sns:eu-west-1:916174725480:new-fmi-opendata-rcrhirlam-pressure-grib

9. AWS gets more popular
10
0
10000
20000
30000
40000
50000
60000
70000
HIRLAM DOWNLOADS FROM DIFFERENT CHANNELS
FMI AWS Normalised Trendline of FMI Trendline of AWS

10. 11
Means roughly 300 users who
fetch whole data
operatively
At the moment 54 000
(whole model) downloads
per month fromAWS

11. Global air quality model added to AWS S3
• Global coverage, surface data
• 5 days of hourly forecast
• Resolution: 0.2x0.2 degree
global regular grid
• Parameters:
NO2, NO, SO2, O3, CO, PM10
and PM2.5, and air density
• Formats:
NetCDF and ZARR
• Testing stage
https://waqi.info/forecast/#/

12. • Full atmosphere, 29 layers up to 10 Pa
• Full CBM4 + Strato +SOA: 113 tracers
• ECMWF IFS meteorology
• Free-flying model (assimilated meteo)
• Applications:
• Global AQ/AC forecast
• Boundaries for regional AQ anywhere
Data fromSILAM model

13. Satellite vs Silam evaluation MODIS
MODIS AQUA/TERRA L2 AOD
vs.
collocated model
Daily composite for Jan 6 2020
Notable:
• Saharan dust plume
• Anthropogenic pollution from India
• China in clouds
• Australian fire plumes down to southern
Pacific
• Understate background AOD (e.g. south
Atlantic)

14. Model development and
applications, user support:
Mikhail Sofiev, Rostislav Kouznetsov
Head of Modelling group of Atmospheric
Composition Department:
Ari Karppinen
Mailing:
first_name.last_name (at) fmi.fi
Open Source
http://silam.fmi.fi
Wild fires
Desert dust
CBM4 + strato.
CBM4 + SOA +
Stratosphere
CBM4
CBM4 + SOA
SOx
Pollen
Radioactive
Passive
self-decay
Long-lived
multi-media
Transformations
Area
Point
Nuclear bomb
Source types
Map of
species
masses
Emission
Transformation
Transport
Advection
diffusion
Aerosol
dynamics
Bio-VOC
Pollen
Sea salt
Simple
Basic
Transformation
Dry
Wet
Deposition
Initialization,
3D-Var
Simulation control
forward adjoint 4D-Var EnKF
PSC
SOA VBS
DMS
Acid-basic
Data fromSILAM model

15. FMI SILAM on AWS
16
• Content: SILAM Global surface
• Coverage: Global
• Grid resolution: 0.2 x 0.2 degree
• Updates: 1 times a day
• Time range: 5 days ahead
• Time step: 1 hour
• Archive 30 days
• Parameters:
http://en.ilmatieteenlaitos.fi/open-data-on-aws-s3

16. FMI OpenData on AWS
17
• Access through buckets:
• NetCDF: fmi-opendata-silam-surface-netcdf
• ZARR: fmi-opendata-silam-surface-zarr
• Browse bucket content:
• http://fmi-opendata-silam-surface-netcdf.s3-website-eu-west-1.amazonaws.com/
• http://fmi-opendata-silam-surface-zarr.s3-website-eu-west-1.amazonaws.com/
• Public Amazon SNS topics are available for every new object added to the Amazon S3:
• arn:aws:sns:eu-west-1:916174725480:new-fmi-opendata-rcrhirlam-surface-grib
• arn:aws:sns:eu-west-1:916174725480:new-fmi-opendata-rcrhirlam-pressure-grib

17. 18
See documentation:
https://en.ilmatieteenlaitos.fi/open-data-on-aws-s3
Questions?
roope.tervo@fmi.fi
https://en.ilmatieteenlaitos.fi/open-data