GEOFRAME: a system for doing hydrology by computer

CUAHSI BIANNUAL MEETING - BOULDER (CO) - JULY 14-16 2008

GEOFRAME
a system for doing hydrology by computer
R. Rigon, A. Antonello, S. Franceschi, D. Giacomelli, E. Cordano,
S. Endrizzi, S. Simoni, M., Dall’Amico, C. Tiso, F. Zanotti

University of Trento (IT) - HYDROLOGIS


GEOtop

Figure modiﬁed after Liang et al, 1994
Rigon et al., JHM, 2006, Bertoldi et al., JHM, 2006, Simoni, 2007, Endrizzi, 2007
www.geotop.org



JGrass/uDig

http://www.jgrass.org, http://jgrasstechtips.blogspot.com/



MODELS

Mademoiselle Rose, ca. 1820. Eugène Delacroix.
IS MODELING SCIENCE ?


Models we are talking about are computer applications

Equations Parameters Data

Mass, momentum and Equation’s constant. In
energy conservation. Forcings and
time!
Chemical observables
In space they are
transformations usually heterogeneous

In the past they were built as monolithic programs



I - Once a model, design and implemented as a monolithic
software entity, has been deployed, its evolution is totally in
the hands of the original developers. While this is a
good thing for intellectual property rights and in a commercial
environment, this is absolutely a bad thing for science and
the way it is supposed to progress.

Robbed from a CCA presentation


II - Independent revisions and third-party
contributions are nearly impossible and especially when
the code is not available.
Models falsiﬁcation (in Popper sense) is usually impossible by
other scientists than the original authors.

III- Thus, model inter-comparison projects give usually
unsatisfying results. Once complex models do not
reproduce data it is usually very difﬁcult to
determine which process or parameterization was
incorrectly implemented.



Q: HOW CAN WE BE MORE “GALILEIAN” ?

A: YES, PRODUCING AND PROMOTING
OPEN SOURCE MODELS. THIS HOWEVER
IS NOT ENOUGH SINCE MODELS SHOULD
B E S T R U C T U R A L LY E A S Y T O
UNDERSTAND, DOCUMENT, MODIFY,
M A I N TA I N , A N D FAVO R P RO C E S S E S
ANALYSIS.


MODELLING, FOR WHO ?
Which end user do you have in mind ?

Baboon, Papius Anubis
SCIENTIST ARE NOT THE ONLY MODELS USERS


• Prime users: take or prepare decisions at a
political level

• Technical users: prepare projects or maps
Users/Actors

for the primary users

Modiﬁed from Rizzoli et al., 2005
• Other end-users: national agencies,
representative groups, etc. They may take or
prepare decisions at national or regional level, or
represent stakeholder groups.

• Model and application developers/
modellers: build models and targeted
applications



• Coders: implement models, applications and
tools.

• Linkers: link existing models and applications.

• Runners: execute existing models, but they
create and deﬁne scenarios.

• Players: play simulations and experiments
Roles

comparing scenarios and making analyses.

• Viewers: view the players’ results, have a low
level of interaction with the framework.

• Providers: provide inputs and data to all other
user roles.


Hard Soft Linkers Runners Player Viewers Providers
Roles
Coders Coders
Users

Prime

Other End
Users

Technical

Researchers

Modiﬁed from Rizzoli et al., ,2005



Thus there exists a “zoo” of users, each one with different abilities.

Bundling data, algorithms and the graphical user interface of a
model in an application with the old techniques makes the
models very hard to re-use out of its original context.

AN ECOLOGY OF MODELS


NEW (well relatively) MODELING PARADIGMS
Object-oriented software development. O-O
programming is nothing new, but it has proven to be a successful
key to the design and implementation of modelling frameworks.

Models and data can be seen as objects and therefore they can
exploit properties such as encapsulation, polymorphism, data
abstraction and inheritance.
Component-oriented software development. Objects
(models and data) should be packaged in components, exposing for
re-use only their most important functions. Libraries of
components can then be re-used and efﬁciently integrated across
modelling frameworks.Yet, a certain degree of dependency of the
model component from the framework can actually hinder reuse.

MODELLING BY COMPONENTS


BENEFITS
Discrete units of software which are re-usable
even outside the framework, both for model components
and for tools components.
Seamless and transparent access to data, which
are made independent of the database layer.

A number of tools (simulation, calibration, etc.) that the
modeller will be free to use (including a visual modelling
environment).

A model repository to store your model (and
simulations) and to share it with others.



BENEFITS FOR SCIENTISTS

Tools for studying feedbacks among different processes.

Encapsulation of single processes or submodels

New educational tools and a “storage” of hydrological
knowledge using appropriate onthologies

MUCH MORE in the ﬁeld of possibilities



THERE EXIST SUCH MODELING
INFRASTRUCTURE ?
Economic modelling frameworks^. GAMS (general
algebraic modelling system, http://www.gams.com) and GTAP
(global trade analysis program, http://
www.gtap.agecon.purdue.edu ) are some of the most used
modelling systems in the agro-economic domain. They can also
account for social variables, such as unemployment.

^from Rizzoli et al., (Modeling Framework (SeamFrame)
Requirements 2005



INFRASTRUCTURE ?
Environmental modelling frameworks. If we limit to the
agricultural domain, the list is quite limited. There is no ‘real’
framework according to the deﬁnition, but APSIM, STICS
and CropSyst provide some of the functionalities. In this area
SEAMFRAME is an emerging technology. When we consider
the water management sector, we ﬁnd many examples,
such as TIME (the invisible modelling environment), IMT,
OpenMI, and OMS, and, to a certain respect, JUPITER-API.

^ extended from Rizzoli et al., (Modeling Framework
(SeamFrame) Requirements 2005



INFRASTRUCTURE ?

Other modelling software environments of notable
interest are SME, MMS, ICMS, Tarsier, Modcom,
Simile, but they are integrated modelling environments, not
frameworks. This means that they can be used to perform
assessments, analyses, decision support, but they do not provide
programming structures such as classes, components, objects,
design patterns to be used to create end-user applications.

^from Rizzoli et al., Modeling Framework (SeamFrame)
Requirements, 2005



INFRASTRUCTURE ?

Atmospheric Sciences: Earth Sciences Modeling Framework
(ESMF) (including Earth System Curator)

High Performance Computing: Common Component
Architecture (CCA)



AN EXAMPLE OF FURTHER NEEDS

Figure modiﬁed after Liang et al, 1994
Rigon et al., JHM, 2006, Bertoldi et al., JHM, 2006, Simoni, 2007, Endrizzi, 2007
www.geotop.org

AWARE OF HYDROLOGICAL PROCESSES


GEOtop Equations Surface Runoff

Rigon et al., JHM, 2006, Bertoldi et al., JHM, 2006, Simoni, 2007, Endrizzi,
2007



GEOtop Equations
Turbulent Sensible and Latent Heat Exchanges

2007



GEOtop Equations Subsurface Flow

2007



GEOtop Equations Soil Temperature

2007



Required Data
- Digital Elevation Model (and
derived geomorphic information - as
lakes, river networks, hillslopes - and
geomorphometric information - like
contributing areas, slopes, curvatures)

- Sky View Factor

- Vegetation and Land Use (for
interception, ET, surface roughness)

- Soil Depth, Texture and
Type (for hydraulic and thermal
properties)

2007



Requires Boundary Conditions



REQUIRES ALSO:

BOUNDARY
CONDITIONS

METEO FORCINGS

PARAMETERS

INITIAL CONDITIONS

METHODS OF
INITIALIZATION


Forcings
- Precipitation (quantity
and type, spatially
distributed)

- Relative humidity
(spatially distributed)

- Wind Speed and
direction (spatially
distributed)

- Solar Radiation
(spatially distributed)



Parameters
- Soil Parameters (e.g.
van Genuchten)

- Surface roughness
(for water celerity)

- Bulk aerodynamic
properties

- Ice and snow
properties

- Atmosphere
radiative properties



Required Initial Conditions
- Soil moisture (proﬁle,
in terms of matric
potential, spatially
distributed)

- Soil temperature
(proﬁle, spatially
distributed)

- Surface water (if
present)

- Snow cover (if present)


To Sum up
Equations Parameters Data

Mass, momentum Equation’s
and energy constant. In time! Forcings and
conservation. observables
In space they are
Chemical heteorgeneous
transformations

Data Assimilation.
Calibration,
Numerics,
Data Models.
derivation from
boundary and
Tools for Analysis.
proxies
initial conditions



Besides
input tools output

CLIMATE
forecast models
scenarios

expert
human
systems
expertise

forecasting
StAtistics
ﬁeld
observations

MODELLING BY COMPONENTS REPRISE


Models as components
Monitored Evapotranspiration
data

Weather Surfaces Surfaces
Interception Runoff
Generator

Weather
Subsurface Flow
Forecast

http://www.openmi.org, http://www.openmi-life.org/, http://public.wldelft.nl/display/OPENMI/Home



Models as components

Channel Routing
Evapotranspiration
1

Surfaces Channel Routing
Runoff II

Channel Routing
Subsurface Flow
III

http://www.openmi.org, http://www.openmi-life.org/, http://public.wldelft.nl/display/OPENMI/Home



Decision making

EVALUATION OF STRATEGIES
THROUGH
THROUGH
MODELS
THROUGH
MODELS
MODELS

DATA INTERPRETATION STRATEGIES FOR POLICY MAKERS



GEOFRAME
external
users web database

Interfaces
(Java/udig/JGrass/BeeGIS)

Analysis Models
Tools (UNITN/R) (UNITN/OpenMI)

Database
(PostgresSQL/PostGIS/HSQL/H2/CUAHSI)

DEPLOYEMENT


PREREQUISITES
OPEN SOURCE

Programming LANGUAGE NEUTRAL

PLATFORM NEUTRAL: Windows, Linus and Mac

BUSINNES NEUTRAL: GPL would be ﬁne, LGPL better

TARGETED AT PERSONAL PRODUCTIVITY OF
DIFFERENT USERS
People come before program efﬁciency.

DEPLOYEMENT


PREREQUISITES
BUILT BY OPEN SOURCE TOOLS

DEPLOYABLE THROUGH THE WEB

ALLOWS WRAPPING OF EXISTING CODES BUT
PROMOTES BETTER PROGRAMMING STRATEGIES

DATA BASE PROVIDED

CUAHSI SPECIFICATIONS AWARE
OGC COMPLIANT

CAN BE ENDOWED WITH ONTOLOGIES
DEPLOYEMENT


The complete framework
Eclipse RCP
uDig
JGrass
PostGIS
Postgres
GIS engine
J-Console Engine
OpenMI
Web
The Horton
UIBuilder
services
Machine
WMS Models
WFS-T H2 spatial
WPS

GRASS
BeeGIS

DEPLOYEMENT


Java
Eclipse RCP
uDig
JGrass
BeeGIS

SOLIDITY: The framework bases on the solid fundaments of the Eclipse RCP
framework first created by IBM.

CONNECTIVITY and USERFRIENDLYNESS: The GIS framework is based on the
uDig GIS framework, specialized in accessibility and remote connections

ANALYSIS: The JGrass extentions define a layer of powerful GIS analysis tools
and a straight connection to the GRASS GIS

MOBILITY: The BeeGIS extentions supply tools for digital field surveying

DEPLOYEMENT


Connectivity and web standards
DATABASE: The GIS framework is ready to connect to
Database: external relational databases as postgres, mysql or
PostGIS-Postgres oracle. To spatial data servers like postgis, Oracle
H2 spatial spatial and Arcsde. It also comes with an internal
spatial database based on H2 (no indexing yet)‫.‏‏‬

It would be fairly easy to create connections to
RESTful?
RESTful services to acquire data.

Web services W E B S E R V I C E S A N D S TA N D A R D W E B
PROTOCOLS: The framework supports OGC
WMS web standards like the web mapping service
WFS-T (WMS), the web feature service, also in
transactional format (WFS-T). An efforth for
soon WPS the web processing service is ongoing.

DEPLOYEMENT


The analysis engine THE CONSOLE ENGINE: the console engine
supplies a framework for modeling
J-Console Engine development and scripting environment
for fast methodology testing.The engine
OpenMI contains already masses of modules called
The Horton Horton Machine for various terrain
Machine analyses as well as a stability model and
hydrologic models.
Also the engine gives access to the GRASS
Models
analysis modules.

THE STANDALONE MODE: The need for
GRASS usage of the modelling environment on
supercomputer deﬁned a heavily decoupled
design for the console engine. The framework
deﬁnes a strict interface between GUI and
analysis engine, which makes it easy to exploit
the console engine in standalone mode on
server-side.
DEPLOYEMENT


The relationship to OpenMI
THE OPENMI ENGINE: the console engine exposes


a compiler for an OpenMI based modeling
OpenMI language.This gives a way to write scripts to execute
openmi chained models.
The Horton THE OGC STANDARDS EXTENTION: The need

Machine for big vector and raster data forced the team to
extend the OpenMI standard interfaces with two GIS
Models OGC standards:
the OGC feature model

the OGC grid coverage service (in prototype mode)‫‏‬

OGC IN JGRASS: the OGC feature and grid coverage models are served by the
geotools libraries. The coverage model is based on the Java Advanced
Imaging library and supports tilecaching for processing of large dataset. Coverage
data are passed to native languages as C, C++ and Fortran through the easy
adoptable JNA libraries.

DEPLOYEMENT


Strategic calls for cooperation
Statet: it is a small RCP plug-in coordinated by Stephan Wahlbrink for
working with R scripts and documentations. R is quot;a language and
environment for statistical computing and graphicsquot;. The autor has
already been contacted.

Talend – Spatial data integrator: CampToCamp develops a psatial data
integrator based on Talend. A ﬁrst meeting was made to discuss
architectural issues. By end of the year the developers of JGrass and
SDI will meet to prototype connectivities between the the softwares.

AND TO EVERYBODY OF GOOD WILL OBVIOUSLY!

DEPLOYEMENT


SNAPSHOTS OF THE SYSTEM ...
It really exists (yes in a pre-alpha version)!

DEPLOYEMENT


GEOcuencas (modified from Mantilla and Gupta,
2005)

Meteo
ET
Forcings

Reservoirs
Snow and intakes

Runoff and Routing modeling

DEPLOYEMENT


Snapshot of a DEM of river Adige with subbasin partition

DEPLOYEMENT


Snapshot of a DEM of river Adige with subbasin partition

active nodes

DEPLOYEMENT

AdB Adige – CUDAM - Bilancio idrico di superﬁcie di primo livello

Active means that they are connected to db information

hydrometers

reservoirs

returns

meteo stations

JHYDRO



• Monitoring stations:
hydrometers

reservoirs

returns

meteo stations

JHYDRO



• Monitoring stations:
hydrometers

reservoirs

returns

withdraws
meteo stations

JHYDRO

CUAHSI BIANNUAL– MEETING - BOULDER (CO)nel JULYdell’Adige 2008
AdB Adige CUDAM - Difesa idrogeologica e bilancio idrico - bacino 14-16

The database is based upon ArcHydro (Maidment, 2002)

DEPLOYEMENT


Using a modiﬁed Pfafstetter enumeration (Verdin and
Verdin, 1999)

First Level

third level

second level

JHYDRO


J-HYDRO: STRUTTURA DELLA RETE
Livello rete 5
Livello rete 4
Livello rete 3
Livello rete 2
Livello rete 1

JHYDRO


J-HYDRO: The network structure

Livello rete 4
Livello rete 3
Livello rete 2
Livello rete 1

JHYDRO



Livello rete 3
Livello rete 2
Livello rete 1

JHYDRO



Livello rete 2
Livello rete 1

JHYDRO



Livello rete 1

JHYDRO


JGRASS: visualization of the DB

LAYERS VIEW

DATABASE CONNECTION

JGRASS


JGRASS: visualization of other data

JGRASS


Models’ runs

OPENMI MODELS


Models’ runs: monitoring stations

OPENMI MODELS


Rainfall Forcings

OPENMI MODELS


Models’ runs: parameters

OPENMI MODELS


Summary of results

OPENMI MODELS


JGRASS: visualization of results

OPENMI MODELS

http://jgrasstechtips.blogspot.com/

WEB DEPLOYEMENT OF THE DEVELOPERS SETUP


PERSPECTIVES AND CONVERGENCES
SMF
E

rth System CURATOR
Ea
OM
S



These above systems are not mutually exclusive they
have many features that can be integrated in each of the
systems, for sure in GEOFRAME.

It is to suppor t cooperation and reinforce
interoperability among these characteristics.

And we, GEOframers offer our unselﬁsh collaboration.


For instance future components could adopt the ESMF
as modeling infrastructure.

Strength: This would allow interoperability with the
Atmospheric Science work.

Weaknesses: At the moment using the infrastructure is
not as easy as it could be. Documentation is very
FORTRANISH and more examples should be added for
C/C++ programmers. An effort should me made to
compile the whole thing using Eclipse, besides than using
makeﬁles.


OMS already provides:

- Components descriptions
-Tools for adding a component
- Model Wizard
- Descriptions of Model Attribute Connectivity
- Connection to COLAB

which could be added to GEOFRAME probably almost
“as is”



At the same time GEOFRAME can provide to OMS:

- a Java interconnected GIS system
- a Java database for storing data information
- The J-console for scripting and dealing also with
OpenMI compliant models



Unfortunately OMS provides mechanism for component
linking different from OpenMI. However OpenMI is a set
of interfaces which is pretty much evolving and this is
the right time to make efforts for interoperability.



Not OMS (which uses JNative) nor GEOFRAME*
(which uses JNA) have a system for binding different
languages codes. Thus there is room to adopt CCA (or
other) solutions, if they result feasible.

*Nor OpenMI on which GEOFRAME is based


EPILOGUE

OUR AIM IS NOT TO MODEL
E V E RY T H I N G * O R D O A M O D E L O F
EVERYTHING BUT GIVE A SPACE WERE
DIFFERENT, EVEN CONTRADICTORY, IDEAS,
AND DATA CAN BE EXPLOITED IN A WAY
WHICH PROPELS COLLABORATIVE EFFORTS
BY SCIENTISTS AND USERS.
*“Correctly interpreted, you know, pi contains the entire history of the human race.”
-Dr. Irving Joshua Matrix, from M. Gardner, “The magic numbers of dr. Matrix”


Direct Contributors:
Andrea Antonello JGRASS, JHydro, JConsole, core developer and architect
Giacomo Bertoldi GEOtop developer (energy budgets, vegetation)
Emanuele Cordano GEOtop developer (Richards equation, I/O)
Matteo Dall’Amico GEOtop developer (permafrost, GEOtop-mono)
Stefano Endrizzi GEOtop developer (energy budgets, snow, permafrost)
Silvia Franceschi JGRASS models developer and architect
Davide Giacomelli JHydro developer
Andreas Hamm JConsole developer
Marco Pegoretti GEOtop developer (surface runoff and routing)
John Preston JGrass ver. 1 and 2 core developer
Riccardo Rigon GEOFRAME architect
Silvia Simoni GEOtop developer (Richards equation), GEOtop-FS developer
Christian Tiso GEOtop power user, apostle and Horton Machine developer
Paolo Verardo GEOtop developer (Radiation budgets)
Fabrizio Zanotti GEOtop-FS developer, GEOtop power user and apostle

Erica Ghesla, Andrea Cozzini, Silvano Pisoni and others contributed to the Horton Machine


THANKS FOR YOUR ATTENTION

Numerics
Statistics
Dance, Henry Matisse, Hotel Biron early 1909

Hydrology

Geography
Analytics


J-Console is the core to open OpenMI to modellers



Where does the console live ?



Issuing JGRASS commands



Issuing GRASS commands



Issuing BEANSHELL commands



BTW YOU CAN USE THE CONSOLE

WITHOUT

JGRASS



IT IS EASY TO CREATE YOUR

COMMAND


Write your OpenMI code like the one you can ﬁnd on:
http://jgrasstechtips.blogspot.com/search/label/console


CREATING A ModelAlgorithmCommandDosomething (MACAD): The above is the list of the available MACADs ... add your own


CREATING THE XML FOR THE GUI BUILDER


CREATING AN RCP ACTION TO LAUNCH THE PROGRAM



TRYING IT

GEOFRAME: a system for doing hydrology by computer

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a GEOFRAME: a system for doing hydrology by computer

Semelhante a GEOFRAME: a system for doing hydrology by computer (20)

Mais de Riccardo Rigon

Mais de Riccardo Rigon (20)

Último

Último (20)

GEOFRAME: a system for doing hydrology by computer