Deux services de données géospatiales offerts par le Service météorologique Canada (Environnement Canada) seront présentés. Le Datamart du SMC permet de télécharger des données brutes produites par le Service météorologique du Canada (SMC) et rend possible la cueillette automatisée des produits et données météorologiques du SMC. Une grande diversité de données s’y retrouvent, tels les avertissements météo, les prévisions publiques en XML, les données brutes de prévision, les observations météorologiques et bien davantage. Le Datamart du SMC jouit d’une grande popularité avec plus de 12 millions d’accès quotidiennement. Les services web géospatiaux GeoMet permettent au public d’intégrer dans leurs outils SIG, cartes interactives ou appareil mobile, les données brutes de modèles de prévision numérique du temps et de la mosaïque nord-américaine des radars météo par le biais de deux standards de l’Open Geospatial Consortium : le WMS et le KML. Qu’il s’agisse par exemple de la pluie observée par les radar météo ou les prévisions de précipitation de neige ou une animation des prévisions de températures , les utilisations de ces données sont nombreuses et d’une grande valeur pour de nombreux décideurs.

1. Datamart du SMC : Données météo ouvertes
&
GeoMet : Accès à la météo en WMS et KML
Alexandre Leroux, M.Sc., Ing.
Service météorologique du Canada
Environnement Canada
Vision Géomatique 2014
12 octobre 2014

2. Environment Canada’s
Meteorological Service of Canada
Canadian Meteorological Centre, Dorval
Authoritative data
That’s the only place in Canada where numerical
weather prediction (NWP) models are run in a
fully 24/7 operational context

3. Meteorological Service of Canada’s
Open Data Server: The MSC Datamart
Page – 13-05-09

4. MSC Datamart: http://dd.meteo.gc.ca/
4

5. MSC Datamart — What it is
• Free and Open access for specialized users
• Open data: freely available to everyone to use and republish as they wish,
the on the sole condition of attribution of the original data to Environment
Canada
• Open formats: published specifications that can be used and implemented
by anyone
• Target audience has some Weather and/or IT knowledge
• Designed for direct, automated (batch) downloading of data via HTTP
protocol
• Offer is complementary to EC’s weather.gc.ca website
• Service is anonymous, supported on best-effort basis
5
• License
• Give credit to Environment Canada
• Commercial usage permitted
• No username/password required

6. 6
History
• 2004
• Initial rollout with MSC's GRIB data
• 2006
• AMIS (satellite service for the delivery of
meteorological alphanumeric data) ends;
MSC Datamart used as alternative
• 2007-2013:
• Addition of many other data types for
specialized users
• 2014:
• Real-time notification service
• Continuous improvement of offered
products
…

7. MSC Datamart — Statistics highlights
• Visitors and download volume: increased by factor of 7 since 2009
• 500,000 visitors daily
• 14,000,000 downloads daily (includes directory listing requests)
7
• Popular products:
• Citypages weather XML: 350M files/month
• NWP model data in GRIB:
• NAEFS: 30M files/month
• GDPS: 18M files/month
• RDPS: 4M files/month
• Observations in SWOB-XML: 3M files/month
• Most significant increases in past year: NAEFS Ensemble NWP
forecasts, Citypages weather, Weather warnings

8. Increasing use of the MSC Datamart as a
dissemination channel:
Average number of files downloaded daily
8

9. Increasing use of the MSC Datamart as a
dissemination channel:
Average data size downloaded daily
9

10. Who are the users?
10
• Public sector
• Provincial agencies and departments, including Emergency
Management Organizations
• State-owned companies
• Municipalities
• Private sector
• Weather companies
• Mobile applications developers
• Agriculture, transportation, energy, insurance, universities, etc.
• Hundreds of thousands of ordinary people… through third-party
mobile and web applications

11. 11
Popular datasets
• Citypage weather XML
• Used directly by apps on mobile devices and other third-party products
• Numerical Weather Prediction forecasts model data in GRIB:
• Global
• Regional
• Ensemble
• HRDPS
• Precipitation Analysis
• Wave model
• Weather warnings in CAP-XML
• Weather observations in SWOB-XML (OGC O&M)

12. And there’s lots more available…
• Nowcasting data
• Weather bulletins
• Air Quality Health Index
• Marine weather forecasts
• Meteocodes
• Hurricane tracks forecast
• Weather forecasting regions
• Vertical profiles, forecasted and observed
• Diamondback Moth trajectories
12

13. Product example: Citypage XML
weather.gc.ca website MSC Datamart
13

14. 14
Applications

15. Managing increasing NWP data volumes:
NWP datasets provided daily on the MSC Datamart
90 GB of NWP data per day
(1.8% of total daily NWP production)
9 GB
per day 9 GB
Regional
Forecast
System
52 GB
per day
12 GB
per day
Global
Ensemble
Forecast
System
Global
Forecast
System
per day
High Resolution
2.5km windows
Forecast
System
… and other NWP models such as
the Wave model, RDPA, etc
Based on the 5TB of operational NWP input (including additional
post-processing) we make available on MSC Datamart

16. Open data formats currently offered
16
• Main data formats offered:
• GRIB2 format — a World Meteorological Organization standard
• Comma-separated values (CSV)
• Extensible Markup Language (XML)
• Raw text
• Other formats currently offered:
• Esri Shapefiles
• GIF images

17. 17
User Interaction
• « Contact us » on the website
• Word of mouth
• dd_info mailing list: ~600 subscribers
• http://lists.ec.gc.ca/cgi-bin/mailman/listinfo/dd_info
• Online documentation
• dps-client@ec.gc.ca: bug reports, questions, requests

18. The AMQP Experiment
• Fact: the MSC Datamart is a “pull” service: to retrieve time
sensitive data, users need to make repeat requests, which
fail continually until new data appears
• Solution: notify users when the product they want is
available, and give them the option to download
• The technological tool: the ‘Advanced Message Queuing
Protocol’, a finance and banking industry high-rate
message routing system (open standard, software libre)
• Win-win: less bandwidth used by both the user and MSC
• Users are invited to use it since January 2014
18

19. Future Plans for the MSC Datamart
• Upcoming geospatial web services for MSC Datamart
content => ‘GeoMet 2’
• Increased presence on Government of Canada’s Open Data
Portal, open.canada.ca
• Improved metadata ISO19115-NAP for discovery and access
• Assess AMQP real-time notifications experiment and make
recommendations
• New potential datasets:
• HRDPS-Continental NWP model
• Climate and hydrological datasets
• Upper air analyses
• Caldas, Synthetic surface data
• “Store front” on ec.gc.ca / canada.ca
19

20. Meteorological Service of Canada’s
GeoMet Geospatial Web Services
Page – 13-05-09

21. MSC has been providing access to NWP model
outputs data and static maps for a long time
but only specialized users
could manipulate the data and use it...
until we began offering new interactive geospatial
data access methods: WMS and KML
this is GeoMet!
launched to the public in July 2013

22. Canadian weather layers on your smartphone,
in your tools and your web maps?
you can now, with GeoMet

23. Web Map Service (WMS) and Network-KML?
- these are geospatial web services interoperable standards
- interactive data visualization
- can be accessed from various devices
web pages, client software, smartphones

24. What can WMS can do for you?
Capabilities
- visualize data products and raw data
- WMS is a web service, there’s no file download involved
- layers are served on-the-fly, there’s no pre-generated images
- data is sent only for a client-specified geographic extent
- data is always the latest available
- enable users to integrate the layers directly into their tools

25. how many html lines to build a website with an
interactive map of GeoMet's weather layers?
lowering the barrier to consume MSC data

26. fully interactive
web mapping website
- zoom, pan, query data
- control over multiple data layers
- overlay external data sources
- you can make your own map
24-lines html website showing the
GDPS Rain (QPF) layer

27. GeoMet KML in Google Earth:
easy access for everyone
HRDPS East - Air temperature
zoom in and GeoMet sends
you higher resolution data
these layers can be animated

28. What’s in GeoMet for you?
Weather radar mosaic:
- American and Canadian radar mosaic at 4km resolution,
updated every 10 minutes, both 8 and 14-colors styles available
NWP models:
- Global Deterministic Prediction System (GDPS) at 25 km resolution
- Regional Deterministic Prediction System (RDPS) at 10 km resolution
- High Resolution Deterministic Systems (HRDPS) at 2.5 km resolution
- GEM - Modelling Air quality and Chemistry (GEM-MACH) model
- Regional Deterministic Precipitation Analysis (RDPA)
Layers include:
- Temperature, precipitation, wind speed & direction, cloud cover, humidity, pressure
and much more
- Derived products such as: dominant precipitation type, wildfire forecasts, etc
GeoMet also serves its metadata in ISO19115
There’s even a new WMS server for NWP-Research experimental products, ‘RPN-WMS’:
http://collaboration.cmc.ec.gc.ca/rpn-wms/?service=WMS&request=GetCapabilities

29. MSC data users are requesting this type of service
EMOs, industry, DND, the public, etc
«Overall we are very impressed with the capability
and support the direction that you are taking with this
[GeoMet] initiative as it will be beneficial to many
consumers of Canadian meteorological data.»
- Iain Russell, Director, Meteorological R&D,
The Weather Network / Pelmorex Media Inc.
participants to the GeoMet Public Trial

30. GeoMet WMS allows sophisticated users
to interact with MSC data into their tools
no file download
the data is always current
ArcGIS, one of plenty of software that
supports the WMS standard
data transferred only for the context-specific geographic extent

31. GeoMet WMS layers into
Sécurité civile du Québec’s
G.O.LOC tool
This tool is used for emergency response (911,
emergency measures, etc)
Only ~100 kilobytes of png tiles is
transferred on-the-fly,
no multi-megabytes file involved

32. GeoMet WMS layers into MASAS
Weather radar composite served in WMS
MASAS: location-based situational awareness information and
alerts for first response and emergency management agencies

33. Another MASAS example

34. CHC’s hurricane and tropical storm forecast tracks
from the GeoMet KML

35. Weather app for Windows tablets developed by Apption,
relying on GeoMet’s WMS layers

36. In ArcGIS Online too

37. It’s mostly about enabling MSC clients to
easily create products for their clients
GeoMet WMS
GeoMet KML
The Public
Accessing MSC
weather layers
via third-party web
maps, apps on
mobile devices,
on Google Earth,
and more!
Specialized users
and Developers
Integration in their own tools,
interactive web maps,
mobile apps, and more!

38. UberWeather.com
leveraging GeoMet to enhance
their offer to the public

39. OpenWeatherMap.org

40. anyone from the public can build
such tools now by using GeoMet
interactive web-based interface to
analyse weather model outputs
colder air from the mountains
goes around Bowen Island
UrbanGEM Viewer for MSC’s CUDM system
example show temperature layer, wind barbs and Google relief

41. Build custom dashboards for your needs,
which contain live GeoMet layers
Data Quality
Dashboard
MetShow

42. Ontario Fire Program

43. GeoMet’s not only for maps!
leveraging the WMS GetFeatureInfo command…
BigWaveDave.ca
punctual data from GeoMet
to feed tables
nXstream
Technologies
feeding graphs

44. Weather Vigilance

45. using readily available, widely used and mature technology and
interoperable standards
It’s all about geospatial data dissemination
Geospatial data is any data located in space:
most of MSC’s data is geospatial data!
Opening the door to new possibilities
- enabling our clients and partners to generate a new set of added-value
products and services
- easing location-based services, e.g. to provide integration with smartphones
and mobile devices
- streamlining MSC internal processes

46. if there’s only one line to remember
GeoMet
versatile and efficient data dissemination
method for displaying weather layers on
interactive websites, mobile devices and
client software using the WMS standard

47. GeoMet’s official page:
http://www.ec.gc.ca/meteo-weather/default.asp?lang=En&n=C0D9B3D8-1
(or just google ‘GeoMet Environment Canada’)
- Join the mailing list
- Let us know about what you do with it

48. GeoMet Usage Statistics
• Serving about 100,000 maps daily during weekdays
• No detailed usage statistics specifically tuned for
geospatial web services at the moment, but trend clearly
show increasing interest and usage
48

49. Future Plans — GeoMet 2
• Lots more data, such as serving the MSC Datamart
content via geospatial web services
• Offering the data in more formats and additional
geospatial web services (e.g. GeoJSON, WFS, WCS, etc)
• Increasing MSC data’s discoverability and access on
Canada’s Open Data Portal
49

50. Pour questions et commentaires :
alexandre.leroux@ec.gc.ca