Digital Infrastructures for Research 2018
https://www.digitalinfrastructures.eu/
Topic "Business Models, Sustainability and Policies"
Session "Open Science: Skills and Credits"
ISCTE, Lisbon, Tuesday 9th October 2018
4. Why Terradue ?
▪ Terradue is an ESA spin-off (2006)
▪ From day one providing support to application
builders in earth sciences, to use satellite EO
data as information source
▪ Early days Grid Computing. Nowadays, Cloud
PaaS, complemented with APIs for Cloud
bursting, aimed at the transfer in production of
released applications
▪ Business model: Platform-centered, a
collaborative workplace for value adders to
interact & co-createNow opening new Ellip solutions
to beta testers, who will help us
improve our Platform services
5. Why value co-creation ?
▪ There is still need for significant capacity
building within the earth sciences community
in exploiting EO data
▪ We believe the value co-creation model is the
way forward to unleash innovation
▪ Since 2014 we’re evolving Terradue Cloud
Platform, user-driven, via ESA & EU funding
▪ A steady and careful process, as well as a
learning process
Now opening new Ellip solutions
to beta testers, who will help us
improve our Platform services
8. ▪ EO data is a unique source of global
measurements over decades
▪ When calibrated and combined with other sources, it
empowers validation & interpolation models
▪ Collaboration process: a feedback loop of
providers & consumers
▪ More providers of pre-processed data layers
▪ New demands for structured information
The growth model is pulled by the
communities
Earth Observation Platform
Producers
Consumers
9. Partnerships
We are always seeking for new Partnerships to
expand our Platform services to users :
▪ Data providers
▪ e-Infra / IaaS providers
▪ Technology providers
▪ Scientific Toolbox/Algorithm/Model providers
A particular focus from community requirements
is to boost the joint use of EO & in-situ data
10. EGI.eu & Terradue Partnership
M2M communications realised initially via OCCI and then via native APIs for
cloud computing, ensuring integration, portability, and interoperability.
Use of standard interfaces
We can rely on well-defined operational processes (e.g. FitSM), having the
EGI Foundation acting as single point of contact of the EGI Federated Cloud.
Reliable operations
We use Cloud resources within the European boundaries. In fact, it is
mandatory for certain processing services due to their organisations’ policies.
European cloud resources
The partnership with EGI Foundation enhances the sustainability of the
involved projects, via SLAs / OLAs helping to sustain operational lifetimes.
Project(s) sustainability
Oct. 2018
https://www.egi.eu/about/newsletters/egi-terradue-a-long-lasting-partnership/
Nov. 2015
12. Other partnership examples
Science providers:
● research and commercial
(GEP, HEP, NextGEOSS)
Data providers:
● EPOS RI on Geohazards TEP
for data uploads and
processing
● Open Data policies in the EO
data world
● (cf. RDA interview)
Technology providers:
● OpenNebula (since before ESA
TEPs early stage, circa 2012)
● OGC standards (Terradue as
standard editor)
Cloud IaaS providers:
● EGI.eu (NextGEOSS)
● EOSC.eu (EOSC-hub)
● DIAS (AppLab, GEP)
https://www.rd-alliance.org/interview-terradue
14. ● Since Terradue’s inception 12 years ago, our
company DNA pushes us to innovate
● We believe technology serves new ways to
design products, that people can use to better
work collaboratively
● Platform Operations goals
○ Manage well defined usage scenarii as ‘products’
○ Pilot a unified foundation for software solutions
○ Innovate smoother, end-to-end business processes matching
customer needs
○ Guarantee the ability to share data among applications
○ ...
Why do Platform Operations exist ?
15. ● Platform Operations require a thoughtful mix of
stability & innovation
● We’re embarking on a new journey to learn how
to tune this mix, based on:
○ The market needs
○ Our own skills and capacities to deliver
software of high quality
○ The cost / benefit of the solutions that we
decide to market
○ The market recognition of what we do
How to run Platform Operations ?
Stability
Timely
Innovation
22. ● Ellip Notebooks
○ “Create interactive laboratory notebooks”
● Ellip Workflows
○ “Design scalable processing chains”
● Ellip Launchpads
○ “Select a data processor and deploy it”
● Ellip Infohubs
○ “Connect a WPS to a custom Geobrowser”
Ellip Solutions
Vestibulum
congue
Vestibulum
congue
Vestibulum
congue
Vestibulum
congue
Ellip
Infohubs
Ellip
N
otebooks
Ellip
Launchpads
Ellip
W
orkflow
s
23. Ellip Solutions in a nutshell
I want to build and maintain a processing service
Interactively
prototype
Integrate for
scalability
Run a scale
Monitor and
collaborate
26. So, what does a platform mean nowadays?
The Ellip Solutions are built for earth sciences
practitioners:
▪ to share about their findings, streamline
their creation of new data products and make
these accessible, interoperable,
▪ and document their developments on the PaaS
as reproducible experiments
In line with the FAIR guiding principles to make
data findable, accessible, interoperable and
reusable
28. Hybrid Cloud
Openstack API -
powered by jclouds
Poznań Supercomputing
and Networking Center
EC2 - powered by jclouds
Amazon Web Services
OCCI,
native cloud APIs
EGI Federated Cloud
Opennebula
registered partner
Terradue
Openstack API -
powered by libcloud
IPT.PL
AWS
EGI
IPT
PSNC
T2
29. www.terradue.com
Scheduled Data Processing
29
24 processing nodes provided by
EGI.eu (NextGEOSS partner)
Growing season
2016-2017
Biopar
products
Terradue Cloud Platform
Biopar Application Burst to EGI.eu
30. www.terradue.com
Scheduled Data Processing
30
It’s a data driven systematic processing. The
service will follow a ramp-up period starting
from September 2016 until end 2017:
■ EU Tectonic area
■ World tectonic area (40%)
■ World tectonic area (70%)
It currently processes 150+ Sentinel-1 SLC
pairs per day.
DLR InSAR Browse Medium
Resolution Service
Supported by
BELNET-BEGRID (Belgium)
31. www.terradue.com
User-Driven Data Processing
31
This service provides the coherence and
intensity changes for Sentinel-1 TOPSAR IW
data pairs performed through SNAP. SNAP is
a common architecture for all Sentinel
Toolboxes, which is ideal for Earth
Observation processing and analysis.
ESA SNAP Sentinel-1 COherence
and INtensity (COIN)
Supported by
GOEGRID-GWGD (Germany)
34. All of it: an Open Cloud strategy
■ Open APIs
Embrace Cloud bursting APIs that can be
easily plugged into the Platform’s codebase,
so to expand the Platform offering with
Providers offering complementary strategic
advantages for different user communities.
■ Developers community
Support and nurture Cloud communities that
collaborate on evolving open source
technologies, including at the level of the
Platform engineering team, when it comes to
deliver modular extensions.
■ Self-service provisioning and
management of resources
The Platform’s end-users are able to
self-provision their required ICT resources
and to work autonomously.
■ Users rights to move data as
needed
By supporting distributed instances of its EO
Data management layer, the Platform
delivers the required level of data locality to
ensure high performance processing with
optimized costs.
35. All of it: an Open Cloud strategy
▪ The Ellip subscribers remain the owners of
their created assets, and decide how to
share these on the platform, for other users
to find and reuse these assets.
▪ They are also empowered to decide on the
target production environment of their
packaged applications (no lock in with
Cloud provider)
Producers
Consumers
36. That’s it!
▪ Last message: we’re always looking for
new collaborators
▪ Join our team in Rome La Città Eterna
15 members, 5 EU nationalities
▪ We are welcoming experts in EO
applications and cloud technologies
▪ Exciting new challenges rolling out Ellip
for Earth Science