Presentation for FOSS4G ASIA 2018

1. FOSS4G ASIA 2018 2018/12/03
Developing a Deep Learning
tool for Map Tiles
N. Iwasaki, D. S. Sprague, N. Ishitsuka
and T. Sakamoto
Institute for Agro-Environmental Sciences, NARO
1

2. /32
Geospatial information as a Big Data
• Huge amount of Data
– Satellite image, GPS, OSM, Drone, IoT Sensor, etc…
• Open Data policy
– We can obtain huge amount of data free
2
Introduction
Brovellia 2015, https://www.slideshare.net/mariabrovelli/the-role-of-geospatial-information-in-a-hyper-connected-society

3. /32
Deep Learning and Geospatial Big Data
• A combination of geospatial data and
deep learning have great potential
3
Introduction
https://pacificspatial.com/wp-content/uploads/2016/12/20181024_DigitalGlobe_%E9%85%8D%E5%B8%83%E7%94%A8.pdf

4. /32
Agenda
• Introduction
– Geospatial information as a Big Data
– Map tile for geospatial analysis
– Deep Learning for geospatial data
• Development of Deep Learning tools for map
tile
– pix2pix for map tile
– qpix2pix
• Case study
– Multimodal data to semantic map
– Topographic map to semantic map
• Conclusion
4
Introduction

5. /32
Deep Learning and Geospatial Big Data
• A combination of geospatial data and
deep learning have great potential
5
Introduction
https://github.com/mapbox/robosat
https://2017.stateofthemap.Zorg/2017/ai-assisted-road-tracing-for-openstreetmap/

6. /32
Barrier for utilization of Geospatial Big Data
• Data preparation
– Deep Learning needs huge amount of data
• We have to spend huge time to data preparation
– NOT ONLY Deep Learning, BUT ALSO GIS analysis
– Download, unzip, transform, convert, merge, etc…
• Just like prepare material for cooking
6
Introduction

7. /32
Map tile (or Slippy map)
• Standardized geospatial information
– typically 256×256 pixel images
• Approximate hole earth as square
– Z0: hole earth, Z1:Divied 2 by 2, Z02:…
• Access to map tile as below
• http://hoge.hoge/{z}/{x}/{y}.ext
– X Direction:West-East, Y direction:North-Sount, Z : Zoom
7
Introduction
Sematic diagram of Map tile
GSI, Japan

8. /32
Week point of traditional Geospatial data
• Non-visualized data. Unclear whether useful
data or not.
• It is required to do many preparation, such
as convert file type, change coordinate
system, extract and merge data, etc…
8
Introduction

9. /32
Advantage of Map tile
• Non-visualized data. Unclear whether useful
data or not.
Available as Web Map
Easy to confirm useful or not
• It is required to do many preparation, such
as convert file type, change coordinate
system, extract and merge data, etc…
Unified coordinate system and easy
handling format
JPG, PNG, GeoJSON
Only “{z}/{x}/{y}” is required
9
Introduction

10. /32
Advantage of map tile for Deep Learning
• Huge amount and various kind of data
is already available
– For training and target data
• For ex. Terrapattern, Using OpenStreetMap
10
Introduction
https://techcrunch.com/2016/05/25/terrapattern-is-a-neural-net-powered-reverse-image-search-for-maps/

11. FOSS4G ASIA 2018 2018/12/03
Map tile
Background image

12. FOSS4G ASIA 2018 2018/12/03
Map tile
Background image
Visualization
Analysis target
General purpose geospatial data format

13. /32
Purpose
• Development of deep learning tools
for Map tile
• Applying Deep learning to Map tile
– Multimodal data to semantic map
– Topographic map to semantic map
• Evaluate efficacy of Map tile for Deep
Learning material
13
Introduction

14. /32
Generative Adversarial Network (GAN)
• Generative model (not classification model)
– Training Generator and Discriminator
• Generator generate fake and Discriminator classify it as a
"Real" sample.
• The goal is that Generator creates images that is difficult
to determine authenticity by Discriminator
• pix2pix is a famous program
– https://github.com/phillipi/pix2pix
14
Development of deep learning tools for Map tile
Model of Generative Adversarial Network
https://stats.stackexchange.com/questions/277756/some-general-questions-on-generative-
adversarial-networks

15. /32
pix2pix for Map tile
15
Development of deep learning tools for Map tile
Sample of images generated by pix2pix.
https://qiita.com/octpath/items/acaf5b4dbcb4e105a8d3
• Improve pix2pix to be possible to use
map tile directory
– Possible to multi layers as input data
• Published in GitHub
• https://github.com/NARO-41605/pix2pix_map_tiles

16. /32
Workflow of pix2pix for map tile
• DataSetMake_tfwiter.py
– Download map tile and prepare training data
• pix2pix_multi.py
– Performing Deep Learning and confirming result
• qpix2pix
– Apply pix2pix model to certain area
16
Development of deep learning tools for Map tile
DataSetMake_tfwiter.py
pix2pix_multi.py
Download data
Prepare training data
Create model
Model
confirming result
qpix2pix
Apply model

17. /32
Workflow of pix2pix for map tile
• Working environment
– Ubuntu 16.04, Python 2.14, tensorflow,
pix2pix-tensorflow
• Recommend: GPU Device and CUDA.
– QGIS 2.18
17
Development of deep learning tools for Map tile
DataSetMake_tfwiter.py
pix2pix_multi.py
qpix2pix
Download data
Prepare training data
Create model
Model
confirming result
Apply model

18. /32
Workflow of pix2pix for map tile
• DataSetMake_tfwiter.py
• Input JSON
18
Development of deep learning tools for Map tile
python DataSetMake_tfwiter.py "images_x_start" "images_x_end"
"images_y_start" "images_y_end" "zoom_level"
--inputJson "INPUTJSON"
--outputPath "OUTPUTPATH"
{
"targetURL": {URL of target map tile, type, format} ,
"inputURL": [
{URL of training map tile 1 , type, format} ,
{URL of training map tile 2 , type, format}
]
}

19. /32
Workflow of pix2pix for map tile
• pix2pix_multi.py
19
Development of deep learning tools for Map tile
pix2pix_multi.py --input_dir "INPUT_DIR"
--mode {train,test} #train: training, test:confirm model
--output_dir "OUTPUT_DIR"
--checkpoint "CHECKPOINT"
--max_steps "MAX_STEPS"
--max_epochs "MAX_EPOCHS“
--ngf "NGF“ #
--ndf "NDF“ #
--input_ch "INPUT_CH"
--target_ch "TARGET_CH"
--GPUdevice "GPUDEVICE"

20. /32
QGIS Plugin for using pix2pix model
• Qpix2pix
– Classify displayed image
20
pix2pix for Map tilesの開発

21. /32
Case study
1. From Landsat and DEM to Paddy rice
planted area.
– Test for simple data conversion
2. From Landsat and DEM to Vegetation
map.
– Test for semantic map
3. Old Topographic map to Land use
map
– BW map to color map
21
Development of deep learning tools for Map tile

22. /32
Data for Deep Learning
• Training data
– Landsat: May and November 2000
• Pan sharpened and Thermal image, from AIST
– DEM
• Target Data
– Paddy planted area (Sakamot et al., 2017), from Github
• https://github.com/NARO-41605/JPaddyMap_test
22
From Landsat and DEM image to Paddy planted area
Training data
Target Data

23. /32
Usage of DataSetMake.py and pix2pix_multi.py
23
From Landsat and DEM image to Paddy planted area
$ python DataSetMake_tfwiter.py 7268 7272 3213 3216 13
--inputJson jsonLan2Jpaddy.txt --outputPath train_data
$ python pix2pix_multi.py --mode train --input_dir train_data --input_ch 20 --target_ch 4
--max_steps 100 --max_epochs 20 --output_dir ./trained_model --GPUdevice 0
$ python pix2pix_multi.py --mode test --input_dir train_data --input_ch 20 --target_ch 4
--output_dir ./target_html --checkpoint ./trained_model --GPUdevice 0
jsonLan2Jpaddy.txt

24. /32
Result
• To develop original map, required
complex data
– Vegetation map, statistical data
• We can developed similar map with only
few command and public available data
– But, quality is not verified
24
From Landsat and DEM image to Paddy planted area

25. /32
Data for Deep Learning
• Training data
– Landsat: May and November 2000
• Pan sharpened and Thermal image, from AIST
– DEM, from GSI, Japan.
• Target Data
– Vegetation map, produce my MOE and published by
ECORIS
25
From Landsat and DEM image to Vegetation map
Training data Target Data

26. /32
Result
• It is confirmed that generate another
map from same data.
26
From Landsat and DEM image to Vegetation map

27. /32
Result
• Over all quality is seems good
– But details are not enough
• Still just similar image
27
From Landsat and DEM image to Vegetation map
Result Target

28. /32
Result
• Over all quality is seems good
– But details are not enough
• More contrasted color map may be needed
– Similar color is not good for GAN
28
From Landsat and DEM image to Vegetation map
Result Target

29. /32
BW map to color
• Training data
– Old Black white topographic map
• Target data
– Land use map derived from the topographic
map
29
From Landsat and DEM image to Vegetation map
Training Target

30. /32
Result
• Accuracy is depend map legend and
density.
30
From Landsat and DEM image to Vegetation map
Ｔａｒｇｅｔ
Result

31. /32
Result
• Accuracy is depend map legend and
density.
31
From Landsat and DEM image to Vegetation map
Ｔａｒｇｅｔ
Result

32. /32
Conclusion
• Map tile as General geospatial data
– Huge amount of data has published
– Valuable for deep learning
• Development of pix2pix for Map Tile
– Use multiple map tile as training data
• Some case study
– It is confirmed that it is possible to develop
several maps easily.
– But, improvement and evaluation of
classification accuracy is father work.
• Map tile is useful for Deep Learning data.
– Further work
• Possible to semantic segmentation and evaluation
Developing a Deep Learning tool for Map Tiles 32