1. ESA Thematic Exploitation
Platforms - Geohazards
Pedro Gonçalves
Terradue
2nd Expert Advisory Group Meeting
Geneva, 5 / 6 February 2019

2. ● R&D activities to create an ecosystem of
interconnected Thematic Exploitation
Platforms
● Users access a work environment
containing the data and resources
required, as opposed to downloading
and replicating the data “at home”
Thematic Exploitation Platforms
The fundamental principle is to move
the User to the data and tools

3. Specific fields in the domain
● Geohazards domain
○ Earthquakes, Volcanoes, Landslides …
● Hydrology domain
○ Hydrological modelling, Flood mapping, Water quality
monitoring …
● …
Data-intensive research
● Targeted data
● Targeted software
THEMATIC Exploitation Platforms

4. ● Very fast access to large volume of data
○ EO data
○ In-situ data
● Distributed processing
○ Hybrid Cloud infrastructure
○ Commercial, Public and Private Clouds
Thematic EXPLOITATION Platforms

5. ● Collaborative tools
○ Referencing Publications
○ Sharing processing results
● Different roles
○ Community scientists
○ Data providers
○ Resource providers
Thematic Exploitation PLATFORMS

6. GEP designed in the context of:
▪ Geohazards Supersite initiative (GSNL)
▪ CEOS Disasters Working Group
User-driven model for partnership and
community building
▪ Started from Int. Forum on Satellite
EO and Geohazards organised by ESA
and GEO in Santorini in 2012 (140+
participants)
Geohazards Exploitation Platform | GEP

7. Platform based on virtualization & federation of
satellite EO data
▪ Provide services & support to the geohazards community
On-demand & systematic processing services
▪ Cloud Compute power, managing multi-tenant resources
Access to Copernicus Sentinels repositories
▪ Plus access to hundred TBs of EO data archives (ERS and
ENVISAT), and other EO missions (ALOS-2, Cosmo-Skymed
and TerraSAR-X) under CEOS WG Disaster and the GSNL
agreements
Geohazards Exploitation Platform | GEP

8. GEP | Initial Usage Scenarios
Users from geoscience centres hosting geohazard experts
▪ with skills in EO (to process, analyze, validate, integrate data)
▪ generating DRM products for decision-makers (End Users)
▪ End Users aren’t intended to be direct users of the platform
Initially targeting three main scenarios:
▪ EO Data Exploitation - A user runs on-demand processing services
▪ New EO Service Development - A user integrates a new processing chain or
modifies/enhances an on-line processing chain
▪ New EO Product Development - A user runs a chain global/systematic to generate
a new derived information layer

9. GEP | Roadmap
GEP Early Adopters Programme
Want to apply as early adopter of the GEP (limited slots)? contact@geohazards-tep.eu
Open under invitations/approval
Consolidation and evolution of the service
Access to EPOS IP users (within available slots)
GEP v1 validation with early adopters
Limited to early adopters
(under invitation), evolution
of the service on-going
Open,
consolidated and
stable service
GEP v2 pre-operations GEP v2 pre-ops ext.
170+ platform early adopters
100+ new users in the period
Today
GEP v2 engineering
50 platform early adopters
28 new users in the period
22 platform early adopters
integrating application or
exploiting on demand processing
Nov 2015 Jul 2018
KO AR
Dec 2016
CDR
Dec 2018Feb 2015
As of Dec 2018:
1100+ registered users
from 86 countries

10. ▪ 20+ on-demand services using Optical & SAR
data grouped in Thematic Apps, according to
the defined goals of Community Managers
▪ New basic services providing full resolution and
change detection imagery for rapid online
visualization
▪ 8 systematic services delivering continuously
updated information layers on GEP, including
the large scale production of Sentinel-1 InSAR
browse images at both 100m and 50m
resolution over tectonic regions and volcanoes
GEP | Enriched Services Portfolio

11. GEP | Early Adopters (examples)
User organisation Areas
Ecole Normale Supérieure de Paris (France) Etna, Italy and Corinth Rift, Greece
DLR IMF (Germany) European tectonic mask
Altamira Information (Spain) Test sites on landslides and earthquakes
ISTerre / Institut de Physique du Globe de Paris (France) Subduction zones of Latin America, the NAFZ and Tibet.
INGV Roma (Italy) Alto Tiberina Fault and Fogo Cape Verde
INGV Roma (Italy) Marmara, East sector of NAFS
INGV Roma (Italy) Haiti and West Java
ETH (Switzerland) Large surface deformations caused by landslides in Bhutan Himalaya
NOA (Greece) Geohazard sites in Greece incl. Corinth Rift
SATIM (Poland) Silesia & Warsaw (Poland)
Obs. Physique du Globe de Clermont-Ferrand (France) Piton de la Fournaise in La Réunion, Cordon del Azufre / Lastarria in Chile–Argentina
INGV Catania (Italy) Etna & Campi Flegrei / Vesuvius
British Geological Survey (UK) Urban areas of Great Britain
University of Leeds (UK) Active deformation in the Alpine-Himalayan belt
ESA Over calibration sites: Rain forest, Germany (DLR targets), Australia Milan, Chicago, Sao Paulo
ESA (Progressive Systems SLR) Greater Cairo, South Rayan dune field, Middle Egypt province and Aswan province
CNR IREA (Italy) Tests on Italian volcanoes and Hawaiian and Japanese volcanic and seismic areas
Universita De L’ Aquila (Italy) Abruzzo region: L’ Aquila and Teramo for post-seismic ground displacements
Volcanoes
Earthquakes
Landslides
Subsidence

12. GEP | Early Adopters (examples)
User organisation Areas
CNR ISSIA (Italy) Indonesia
IPGP (France) Asia, N& S America, Indian Ocean
Universidad de Concepcion (Chile) Southern Andean zone
Laboratoire de Dynamique Terrestre et Planétaire (France) South America active volcanoes and tectonics
BRGM (France) French coast subsidence
AIM CEA (France) La Reunion
National Cartographic Center (Iran) Iran
Instituto Geologico y Minero de Espana (Spain) SouthEast Spain
USGS (USA) Latin America volcanoes
CVGHM (Indonesia) Indonesian and Mexican volcanoes
CNES (France) Validation of tools for interferometric coherence over Syria and France
Istanbul Technical University (Turkey) Deformation time series (volcanoes and faults )and mean velocity maps
Institute of Geodesy Cartography and Remote Sensing
(Hungary)
Validated multi-technique of dynamics of anthropogenic ground deformations
Universitat Autònoma de Barcelona (Spain)
Identification of areas affected by wildfires, erosion and debris deposition/ ground motion due
to debris deposition in Mallorca island
NASA Goddard Space Flight Center (USA) Ground deformation time series - landslides, volcanoes and flood mapping
Royal museum for Central Africa (Belgium) Image correlation for the investigation of surface displacement field of landslides
University College of London (UK) UK landslides
University of Rabat(Morocco) Morocco seismic activity
Volcanoes
Earthquakes
Landslides
Subsidence

13. GEP | Early Adopters (July 2018)
▪ 172 users (from 77 organisations and 35 countries) are carrying out projects on the platform via the Early Adopters Programme
▪ Mainly European users, but also from the rest of the world: Asia (Turkey, China, Thailand, South Korea, Indonesia, India, Malaysia & Iran),
Africa (Morocco, Nigeria, Ethiopia), Latin America (Mexico, Chile, Ecuador, Brazil and Argentina) and North America (USA and Canada).

14. User Scheduled Processing
The data are automatically selected on a
regular basis, according to different policies:
■ Time driven
■ Data driven
Scheduled User Processing
https://geohazards-tep.eu
Supported by

15. It’s a data driven systematic processing. The
service has followed a ramp-up process
starting from Dec 2016 until Aug 2017:
■ EU Tectonic area
■ World tectonic area (25%)
■ World tectonic area (40%)
It currently processes 150+ Sentinel-1 SLC
pairs per day.
DLR InSAR Browse Medium
Resolution Service
Supported by
BELNET-BEGRID (Belgium)
Data Driven Scheduled Processing
https://geohazards-tep.eu

16. Data Driven Scheduled Processing
It’s a data driven systematic processing. This
service provides an amplitude and coherence
product at 50m systematically from
Sentinel-1 data for a number of volcanoes
(20+) globally.
DLR InSAR High Resolution
Change Monitoring Service
https://geohazards-tep.eu
Supported by
ReCaS Bari (Italy)

17. The User triggers the processing services from
the Geobrowser Web interface.
The User searches and filters the EO data
from an Opensearch Catalogue, then passes
the selected inputs to a processing service.
User-Driven processing
https://geohazards-tep.eu
User Driven Processing
Supported by

18. This service provides the coherence and
intensity changes for Sentinel-1 TOPSAR IW
data pairs performed through SNAP. SNAP is
a common architecture for all Sentinel
Toolboxes, which is ideal for Earth
Observation processing and analysis.
ESA SNAP Sentinel-1 COherence
and INtensity (COIN)
Supported by
User Driven Processing
https://geohazards-tep.eu

19. P-SBAS stands for Parallel Small BAseline
Subset and it is a DInSAR processing chain
for the generation of Earth deformation time
series and mean velocity maps. Input: SLC
(Level-1) Sentinel-1 data.
CNR-IREA P-SBAS Sentinel-1
processing on-demand
https://geohazards-tep.eu
Will be
supported by
BELNET-BEGRID (Belgium)
User Driven Processing

20. GEP | Monthly Statistics (Dec 2018)
EO Data collections accessible
(i.e. the user can run a processing against these
data on the GEP)
Global Coverage with open access:
Sentinel-1, Sentinel-2, Sentinel-3, Landsat-8, ERS (SAR IM Level-0), ENVISAT (ASAR
IM Level-0)
Limited coverage collections with restricted access:
ALOS-2, TerraSAR-X, COSMO SkyMed, RADARSAT-2, Pleiades
Several datasets from various EO missions for the Charter Platform Prototype:
ALOS-2, TerraSAR-X, SPOT6/7, Pleiades, UK-DMC-2, Kompsat-2/3/5, Resurs-P,
Kanopus-V, RAPIDEYE, GF2, PROBA-V, SUOMI
Number of processing requests launched
[by data source, month]
Sentinel-1 (4.534), Sentinel-2 (1.039), ENVISAT (30), Landsat-8 (138), ERS (30),
KOMPSAT-5 (61), Pleiades/SPOT (88), UK-DMC-2 (20), ALOS-2 (4), TerraSAR-X (2),
SUOMI (8)
Quantity of processing hours [month] 7.664 CPU/hours
Number of products generated [month] 18.072
Number of products published [month] 8.375
Number of products downloaded [month] 6.029

21. GEP | Users Uptake (Dec 2018)
The GEP has a total of 1131 registered users from 86 countries world wide

22. GEP | Users geo-distribution (last 30 days)
1-Year

23. GEP | Cloud Platform - Monitoring

24. Enhanced Data Gateway
▪ Automatic multi-sourcing to optimise data access requests
▪ Programmable and systematic data caching
▪ Data usage accounting
▪ Personal cloud storage (repository)
GEP | Cloud Platform - Data
Supported Data Providers
Daily figures

25. Application Integration
● Applications developed in any programming language supported
○ C/C++, Java, Python, Matlab and IDL
● Continuous Integration and Deployment Environment with
automatic packaging and deployment in production environments

26. ● OGC Testbeds activity to identify a strategy for
packaging applications able to run in different
Cloud computing production centres
● OWS Context documents allow applications to
be specified and deployed using:
○ Information about the Docker container
providing a consistent environment for
execution
○ WPS as the execution interface information
about the input data and respective
○ OpenSearch catalogue end-points.
GEP | Standardization

27. ● Improved Production Center, with (auto)scalability allowing
cost-effective data processing on Cloud Computing clusters
● Deployment in multiple Cloud-based
processing environments
○ EGI.eu, Amazon AWS, PSNC,
CloudFerro (EO IPT Poland)
● On going/in negotiation with
○ Serco ONDA DIAS: integration of the GEP as Reference Front Office
○ CloudFerro CREODIAS: integration of systematic chains of the GEP
○ EODC in the context of the EOSC-Hub project
GEP | Cloud Platform - Computing

28. ▪ User Communities remain the owners of their
created assets, and decide how to share these
on the platform, for other users to find and
reuse these assets
▪ They are also empowered to decide on the
target production environment of their
packaged applications
▪ no lock-in with Cloud provider
▪ Open APIs needed for emerging marketplaces
(e.g. EOSC-HUB)
GEP | Cloud APIs - SaaS

29. Uptake & Sustainability
CEOS Pilots
▪ Support for CEOS Pilots for complex workflows
▪ Multiple-pairwise image correlation for the
monitoring of surface deformation from
optical image time-series)
▪ Processing service for measuring horizontal
displacements from time-series of optical
satellite images
▪ ESA and CNES to setting up a mechanism
to bring Pleiades data on GEP for on-line
processing
Sentinel-2
surface velocities
over the
Harmaliere
landslide
in French Alps
MPIC-OPT service developed by
CNRS / EOST (Strasbourg)
S-2 image tile
Harmalière landslide
(French Alps)

30. Uptake & Sustainability
International Disasters Charter
▪ International Charter Major and Space
Disasters Board selected GEP as Prototype
processing platform
▪ Testing the support of Project Manager and
Value Adding partners for Charter activations.
▪ Help rapidly screen data with data
processing (co-registering multi-sensor
collections), and image analysis (e.g.
change detection)
▪ Automatic notifications about Charter
activations
▪ Supporting 18 missions

31. Uptake & Sustainability
Scientific networks - EPOS
▪ EPOS, European Plate Observing System, is a
long-term plan to facilitate integrated use of
data, data products, and facilities from
distributed research infrastructures for solid
Earth science in Europe.
▪ GEP was selected as the gateway for the
Satellite Data Thematic Core Service in EPOS
▪ EPOS Implementation: the GEP will provide the
interface for the Satellite Data
▪ EPOS Operations: GEP users will have access to any
other data available within EPOS IP and
▪ EPOS ERIC will purchase GEP services (estimative
~100k/year) with ESA as an associated
partner within the Satellite Data TCS
governance scheme

32. GEP | Knowledge Hub Resources
GitHub, Zenodo,
Twitter, Forums,
Notebooks ...

33. ● Dedicated Geohazards TEP Zenodo community
○ Users upload a products file (+ metadata)
○ Reviewed by the Scientific Communicator
○ After approval, automated process to create
Digital Object Identifiers (DOI) for each product
● The author citations are managed and
sharable from Zenodo
● The publication of approved products on the
GEP Community area is fully transparent for
users (GEP uses the Zenodo API)
Publishing GEP Results via Zenodo

34. ● Exploring the Earth
Observation
Catalogues from a
Notebook
● Accessing
OpenSearch API
directly from Python
● Analyse results as
data structures
(Panda/Geopanda)
Data Exploitation: Target Reproducibility

35. ● Notebooks
documenting how
to plot and analyse
Sentinel data
Data Exploitation: Access to EO Products

36. ● Notebook analysis
of a stack of
Sentinel-1 data
● Backscatter profiles
for reference image
used in flood
analysis
Data Exploitation: Time Series Analyses

37. ● Invoking GEP
services from a
notebook
● DIAPASON
Stripmap service
over Fogo volcano
through a WPS
Service
Data Exploitation: WPS Services

38. GEP | Take-home Messages
▪ Benefit from fast access to EO data, storage
capacity and processing resources offered by
platform-based solutions
▪ A hosted processing platform is a partnership
that needs the support of scientist/developer
to offer operational services providing value
added information to the community
▪ Platform e-collaboration and reproducible
knowledge promote innovation and response
capacity

39. Platforms: Take-home Messages
Cloud Platform Solutions built for Earth sciences
communities:
▪ to share their findings, streamline their
creation of new data products and make
these accessible, interoperable
▪ to document their developments as
reproducible experiments
▪ with FAIR guiding principles to make data
findable, accessible, interoperable and reusable

40. ▪ Agriculture monitoring
▪ Air quality
▪ Biodiversity
▪ Climate Change
▪ Cold Regions Monitoring
▪ Crop Monitoring
▪ Disaster Risk Reduction
▪ Earthquakes
▪ Energy
▪ Food Security
▪ Land management
▪ Oceanography
▪ Volcanology
▪ Water quality
▪ Wetlands
User communities

41. Open Cloud Strategy
■ Freedom of movement
Users are free to move data as needed.
By supporting distributed instances of its
EO Data management layer, deliver the
required level of data locality to ensure high
performance processing with optimized costs
■ Developers community
Support and nurture Cloud communities that
collaborate on evolving open source
technologies, including at the level of the
Platform engineering team, when it comes to
deliver modular extensions.
■ Self-service provisioning
The Platform’s end-users are able to
self-provision and manage their required
ICT resources transparently and perform their
tasks autonomously
■ Open APIs
Embrace Cloud bursting APIs that can be
easily plugged into the Platform’s codebase
to expand the Platform with Providers offering
complementary strategic advantages for
different user communities

42. Looking forward
hearing from you!
https://www.terradue.com
Pedro Gonçalves. Director
pedro@terradue.com