F. De Fausti, 5 maggio 2021 - WPH Earth Observation:Deep Learning Segmentation for Improved Land Cover Maps and Estimates

1. WPH Earth Observation:
Deep Learning Segmentation for
Improved Land Cover Maps and Estimates
Fabrizio De Fausti, Erika Cerasti, Angela Pappagallo, Francesco Pugliese, Diego Zardetto
Mauro Bruno, Monica Scannapieco
Istat | DCME
Istat, 5 Maggio 2021
ESSnet Big Data Pilots II: Risultati e prospettive di sviluppo verso le Trusted Smart Statistics
FABRIZIO DE FAUSTI

2. WPH Earth Observation
2 LAND COVER MAP & ESTIMATES | FABRIZIO DE FAUSTI
Task 1 – Agriculture
2.1.1 Case study 1 - Crop recognition, mapping and monitoring
2.1.2 Case study 2 – Monitoring of the off-season vegetation cover
2.1.3 Case study 3 (BE) – Crop recognition with very high resolution aerial data
Task 2 – Built-up area
2.2.1 Case study 4 - Implementing SDG indicator 11.7.1
2.2.2 Case study 5 – Urban sprawl across urban areas in Europe
2.2.3 Case study 6 - Combination of administrative and Earth Observation data to
determine the quality of housing
Task 3 – Land Cover
2.3.1 Case study 7 - Comparing «in-situ» and «remote-sensing» collection mode for land
cover data
2.3.2 Case study 8 – Land cover maps at very detailed scale
Task 4 - Settlements, Enumeration Areas and Forestry
2.4.1 Case study 9 - Update the INSPIRE Theme Statistical Units dataset and preventing
forest fire

3. WPH Earth Observation
3 LAND COVER MAP & ESTIMATES | FABRIZIO DE FAUSTI
Task 1 – Agriculture
2.1.1 Case study 1 - Crop recognition, mapping and monitoring
2.1.2 Case study 2 – Monitoring of the off-season vegetation cover
2.1.3 Case study 3 (BE) – Crop recognition with very high resolution aerial data
Task 2 – Built-up area
2.2.1 Case study 4 - Implementing SDG indicator 11.7.1
2.2.2 Case study 5 – Urban sprawl across urban areas in Europe
2.2.3 Case study 6 - Combination of administrative and Earth Observation data to
determine the quality of housing
Task 3 – Land Cover
2.3.1 Case study 7 - Comparing «in-situ» and «remote-sensing» collection mode for land
cover data
2.3.2 Case study 8 – Land cover maps at very detailed scale
Task 4 - Settlements, Enumeration Areas and Forestry
2.4.1 Case study 9 - Update the INSPIRE Theme Statistical Units dataset and preventing
forest fire

4. WPH Earth Observation
4 LAND COVER MAP & ESTIMATES | FABRIZIO DE FAUSTI
Poland Case Study 1: Crop recognition, mapping and monitoring
- Sentinel 1 and Sentinel 2
- R,G,B,NIR
- Cadastral particels
- 3 satellite captures
- ML: Random Forest and Artificial Neural Networks

5. WPH Earth Observation
5 LAND COVER MAP & ESTIMATES | FABRIZIO DE FAUSTI
Finland Case Study 2: Monitoring of the off-season vegetation cover
- Sentinel 1 and Sentinel 2
- Integrated Administration and Control System (IACS)
- Remote Classifier

6. WPH Earth Observation
6 LAND COVER MAP & ESTIMATES | FABRIZIO DE FAUSTI
France Case Study 4: Implementing SDG indicator 11.7.1
SDG Indicator 11.7.1
“average share of the build-up area of cities that is
open space for public use for all, by sex, age and
persons with disabilities “
• Sentinel-2
• Open Street Map
• National Topographic Database (BDTOPO)
• Piano CA-DASTRAL

7. Istat Case Study
7
GOALS
Land Cover (LC) statistics and maps are a very important statistical product. As they require a big
effort to be created, the idea is to build an automatic system that processes satellite images in
order to generate:
• Automatic Land Cover Estimates
• Automatic Land Cover Maps
HOW
• Standard approach: Spectral Signature
• New approach: Using Deep Learning (CNN for classify-and-count and U-Net for segmentation)
RESULTS
A Deep Learning-based (CNN + U-Net) integrated architecture that gives accurate results for all LC
classes
LAND COVER MAP & ESTIMATES | FABRIZIO DE FAUSTI

8. ML Approaches to LC from Images
8
Different LC classes have different reflectance
spectra
Variation of reflectance with EM frequency can be used
to predict LC class
Trained ML algorithm predicts the LC class of image
pixels independently
Decision on each pixel does not depend on neighboring
pixels
Different LC classes have different visual/spatial
patterns
Variation of visual/spatial patterns can be used to predict
LC class
Trained ML algorithm (CNN/U-net) predicts LC class of
image pixels based on information from neighboring pixels
Decision on each pixel depends on the whole sub-image
(tile) the pixel belongs to
ANNUAL CROP FOREST
HIGHWAY
INDUSTRIAL
RESIDENTIAL
RIVER
LAND COVER MAP & ESTIMATES | FABRIZIO DE FAUSTI

9. CNN: Satellite Images Dataset
9
ANNUAL CROP FOREST
HIGHWAY
INDUSTRIAL
RESIDENTIAL
RIVER
HERBACEOUS VEGETATYION
PASTURE PERMANENT CROP
EuroSAT dataset
(https://github.com/phelber/eurosat):
• Based on Sentinel-2 satellite images
• 27000 geo-referenced and labeled image patches
(each one of 64x64 pixels)
• 10 different Land Use and Land Cover classes,
with 2000-3000 images per class
• RGB (8-bit) and Multi-Spectral (13 spectral bands,
16-bit) versions available
SEA LAKE
LAND COVER MAP & ESTIMATES | FABRIZIO DE FAUSTI

10. CNN: Inception-V3 Architecture
10
Input Satellite Image
RESIDENTIAL
Classify-and-Count
Approach
LAND COVER MAP & ESTIMATES | FABRIZIO DE FAUSTI

11. CNN: Example of Automated LC Map
11
[A]
The ‘Lecce image’
(751 km2)
[B]
Automated LC map
derived from the ‘Lecce
image’
[C]
Edge line of the
‘Residential’ class derived
from [B] overlaid on [A]
LAND COVER MAP & ESTIMATES | FABRIZIO DE FAUSTI

12. CNN: The Overestimation Issue of ‘River’ and ‘Highway’
12
~600
m
~500
m
[D]
A detailed view of the course of the Arno River (cropped from the
‘Pisa image’, 443 km2) overlaid with a semitransparent version of
the corresponding automated LC map
[E]
A highway fragment from
the ‘Lecce image’ overlaid
with the edge line of the
‘Highway’ class
LAND COVER MAP & ESTIMATES | FABRIZIO DE FAUSTI

13. CNN: Example of Automated LC Map
LAND COVER STATISTICS | FABRIZIO DE FAUSTI
13
• In order to train the U-Net we need a label image
for each EuroSAT dataset image.
• We created a segmentation mask for each EuroSAT
dataset image of ‘River’ class using the High
Resolution Layer data provided by Copernicus.
• Our dataset: 1500 validated segmentation masks
• We build similar labels for the ’Highway” class
using Open Street Maps data.
EuroSAT
‘River’ images
Segmentation
masks

14. U-Net: Architecture
LAND COVER STATISTICS | FABRIZIO DE FAUSTI
14

15. U-Net:Segmentation Solves the ‘River’ Overestimation Issue
LAND COVER STATISTICS | FABRIZIO DE FAUSTI
15
CNN
Classification
U-Net
Segmentation

16. Conclusions
LAND COVER STATISTICS | FABRIZIO DE FAUSTI
16
The new integrated architecture (CNN + U-Net) works very well for all LC classes:
o The U-Net takes care of LC classes “River” and “Highway”
o The CNN copes with all the other LC classes
o Partial LC maps produced by 1) and 2) are merged to yield a final complete LC map
• to better test the accuracy of our automated LC estimates
• to compare them to those produced by the LUCAS survey
We are currently scaling-up our experiments to much larger territories:
italian NUTS-regions in order:
NEXT STEPS
RESULTS
Output LC maps are detailed and accurate
Output LC statistics are sound

17. Grazie
FABRIZIO DE FAUSTI | defausti@istat.it