Geopaparazzi Workshop given at Foss4g Asia 2018

1. GEOPAPARAZZI
NEVER OUT OF DATA IN THE FIELD
Andrea Antonello, Silvia Franceschi
HydroloGIS s.r.l.
Foss4G-Asia 2018
December 2th 2018

2. HYDROLOGIS
small (very small) engineering company
Bolzano (north Italy)
2 cofounders: Andrea Antonello, Silvia Franceschi

3. HYDROLOGIS
environmental engineers specialized in hydrology,
hydraulics, geomorphology and forestry
PhD in Environmental Sciences and Mountain
Environment and Agriculture
developers of
the HortonMachine library in the fields of: hydrology,
hydraulic, forestry
Geopaparazzi app for digital field mapping
gvSIG/uDig plugins and customization
OSGeo Charter Members

4. WHAT IS THIS WORKSHOP ABOUT?
geopaparazzi intro
data centralization with geopaparazzi survey server
not done but present in the material: data preparation and
evaluation with GIS
not done at all: vector editing

5. GEOPAPARAZZI

6. INTRODUCTION
Geopaparazzi is a tool for digital field mapping. Originally it
has been developed for fast qualitative engineering /
geologic surveys but it can be used also as a tool for
emergency management, general field activities and
tourism.
Geopaparazzi is available on the . Search
for Geopaparazzi on your phone or get it from the online
android market.
Geopaparazzi doesn’t work on iPhone and Windows
Phone.
Android Play Store

7. MAIN OBJECTIVE
The main aim of Geopaparazzi is to have a tool that:
can be always at hand when needed
gives the possibility to take georeferenced pictures during
the survey
gives further possibility to import pictures and data into a
main GIS application
is extremely easy to use and intuitive, providing just few
important functionalities

8. MAIN FEATURES
The main features available in Geopaparazzi are:
georeferenced notes: text notes, pictures, sketches, forms
gps tracks logging
a map view for browsing and navigation of the data
easy export of collected data
simple generation of any type of background map

9. INSTALLATION
1. connect your Android device to
internet
2. open Play Store
3. type geopaparazzi within the
Search textbox

10. DASHBOARD ICONS

11. ACTIONBAR ICONS

12. TAKE NOTES
1. simple notes: text, picture, sketch
2. possibility to insert GPS or last map
center coordinate
3. complex notes based on structured
forms

13. PROJECT INFO
1. remove/add new data
2. edit and save the project’s
metadata

14. GPS LOGGING
1. enter a new log
name
2. possibility to
continue from
the last log
3. logging button
shows
enablement

15. MAP VIEW
long tap the
upper right
floating button
to clear the
map from tools
tap again to
get them back

16. MAP VIEW TOOLS
1. notes
tap: add new note
long tap: open notes list
2. bookmarks
tap: add new bookmark
long tap: open bookmarks list
3. activate log analysis tool
4. activate measure tool

17. EDITING TOOLS
long tap:
select
editable
layer
tap: enter
editing
mode

18. IMPORT
1. GPX
2. csv files bookmarks_xyz.csv (label,
lat, lon, zoom)
3. import WMS as mapurl definition
4. create a simple spatialite database
with a point, line and polygon table
5. download a project from a server
6. download a profile from a server

19. EXPORT
1. KMZ: only export that supports
proper complex notes representation
2. GPX
3. bookmarks
4. extract all images into a folder
5. export notes as PDF
6. upload the project to a server

20. SUPPORTED BACKGROUND DATA
Available background data are accessed from the select tile
sources menu:

21. MAPSFORGE
1. open source project under LGPL
license
2. .map extension
3. binary vector format
4. maps available for download at:
5. lot of data in little space
http://download.mapsforge.org/maps/

22. MBTILES
1. open specifications:
2. .mbtiles extension
3. image tiles format
4. can be created in many different
ways (ex. hortonmachine, gvSIG,
gdal)
5. small regions need huge space
Mapbox
presentation page

23. MAPURL
1. simple textfile describing an
online/offline TMS or online WMS
service
2. .mapurl extension

24. EXAMPLES
online
mapnik
offline
mapsforge
online TMS offline
mbtiles

25. SUPPORTED OVERLAY DATA
Overlay data are accessible from the map menu. Basically
Spatialite databases are supported:

26. PROPERTIES: COLOR AND SIZE
set shape
set stroke width
set fill and stroke
color

27. LABELLING
set dimension
set field

28. THE GEOPAPARAZZI SURVEY SERVER

29. THE SERVER
The Geopaparazzi Survey Server (GSS) is a web application
that allows geopaparazzi users to synchronize their projects
data with a central server.
Its companion is an Android app named Geopaparazzi
Survey Server Sync (GSSS) .
The app can connect to geopaparazzi projects and
synchronize the data contained using the unique device ID
to upload the data to the server.
The server can be installed through docker easily on linux
pcs.
available on google play

30. INSTALLATION
Installation instructions are available on
.
 The installation on an ubuntu operating system is
extremely easy! So use linux!
the project
documentation site

31. INSTALLATION OF DOCKER ON
UBUNTU LINUX
Long story short installation is all about having docker
installed:
Test the installation with the hello world
and once you are sure it works you can run the server
already.
sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -a -G docker $USER
docker run hello-world

32. INSTALLATION OF GSS ON UBUNTU
LINUX
To start the server, just run:
This will download and install the docker image on your
system (if it is not there already) and then start a server that
uses /home/hydrologis/GSS as data folder and port 8080 for
its webservice.
Open your favorite browser and enter the url:
docker run --rm -v /home/hydrologis/GSS:/home/basefolder
-e JAVA_OPTS="-Xms1256m -Xmx4g -Dstage.globalfolder=/home/basefolder/"
-p 8080:8080 -p 9092:9092 moovida/gss:v2.15
http://localhost:8080

33. You should get the following login screen:
You can login with user=god and password=god, which as
you can guess is an admin.

34. THE DASHBOARD
The dashboard view shows a simple chart listing the amount
of information for each device.
If no data are available, the following will be shown:

35. In case of existing data:

36. THE MAPVIEW
The mapview features the Surveyor list and a map panel.
Data can be browsed and filtered by timestamp.

37. The surveyor can be actived by selecting the checkbox in the
table. Once the data of the surveyor are loaded, through the
zoom button on top of the table it is possible to zoom to the
surveyor’s data extent.

38. THE SETTINGS VIEW
In the settings view it is possible to:
add, modify and remove surveyors.

39. activate an automatic registration mode for 5 minutes.
New devices that connect during the 5 minutes are
automatically added, if the surveyor does not exist. The
name of the surveyor by default is the id of the device.
This can be changed later.

40. create web users and groups. There are two levels of
users: admins and normal users.

41. configure background maps. Several map services can be
added to the background maps.
 Make sure that you have the rights to use the maps
you select.

42. change the locale of the application

43. THE EXPORT VIEW
From the export view it is possible to:
export the form data of one or more surveyors to PDF

44. export the all the data of one or more surveyors to KMZ

45. download the complete spatial database of the server.
The database is in H2GIS format and can then be
connected to with gvSIG, orbisGIS, OpenJump (maybe
others also)

46. THE PROJECT DATA VIEW
In the project data view it is possible to upload basemaps,
overlay databases and geopaparazzi projects to make them
available to the devices through the sync app:

47. THE FORM BUILDER VIEW
The form builder can be used to visually create forms for
geopaparazzi and make them available through the sync
app:

48. THE LOG VIEW
The log view can help to understand issues when they
happen. It lists various types of messages. The two most
usefull types are:
ACCESS: to see if and why a user could not log in or
upload data
ERROR: to understand why a particular problem is
happening

49. THE GEOPAPARAZZI SURVEY SERVER
SYNC APP (GSSS)

50. INSTALLATION
The GSSS can be installed from the .play store

51. Once installed and launched it will just show an empty
welcome view.

52. THE SIDE MENU
In the side menu it is possible to access several features.

53. CHECK YOUR DEVICE ID!
The first thing to do is to check if the device has an own
unique device id. If it has one, the following screen will be
shown

54. SET THE SERVER URL
To be able to connect to the GSS server, the url of the server
needs to be inserted. For the workshop we will use an
existing demo server http://www.test.hydrologis.com

55. PREPARE THE SERVER FOR THE
SURVEY

56. CREATE WEB USERS (LET’S CALL
THEM COORDINATORS)
To prepare the survey we first create web users for the
coordinators. They will be able to create the forms and if
they have admin rights also prepare data for the survey.
Select the Web Users Settings entry and a user for every
coordinator.

57. CREATE SURVEYORS
A coordinator can be different from the actual surveyor. We
therefore need to enable the surveyors. A surveyor can be
added manually:
or with magic ;-)

58. CREATE SURVEYORS WITH MAGIC
Enable Automagic Registration and just connect with the
devices.
 Do this one at the time to be able to connect IDs
properly.

59. FROM GSSS DO ANY CONNECTING
OPERATION
For example do a data download refresh:

60. CHECK AND MODIFY ON THE SERVER
A new surveyor will be available on the server:
Double-click and rename the device id to something
meaningful:

61. GET THE DATA FROM THE SERVER
Datasets available that we will use are of basemap, overlay
and project type:

62. sri-lanka.map: mapsforge map of Sri Lanka. Less than
100Mb for all the Openstreetmap data of Sri Lanka on your
device

63. moratuwa_university.mbtiles: mbtiles map of an ortophoto
of the region of the university. The image is from google
maps and has pure educational purposes.

64. foss4gasia_venue.mbtiles: mbtiles map of the georectified
venue image.

65. osm_sri_lanka.sqlite: spatialite vector database with some
data of the region of Colombo-Moratuwa

66. DOWNLOAD THE DATA WITH GSSS
If you open the GSSS, enter the Data Download view and
push refresh, you should see the available data. The
different icons are for basemaps (grid), overlays (polyline)
and project.

67. LOAD THE DATA INTO
GEOPAPARAZZI
To load basemaps enter the tiles icon from the main
actionbar, then use the FAB button to browse the filesystem
for the data:

68. To load overlays enter the map context menu, then use the
FAB button to browse the filesystem for the data:

69. To load an existing geopaparazzi project choose Load
existing project from the main menu to browse the
filesystem for the project file:

70. THE FORM BUILDER

71. WHAT WE WANT TO CREATE
Let us assume we need to survey the buildings on the
campus. For this we will have a a nice form with 3 parts:
General

72. Structural

73. Images

74. STEPS TO CREATE THE FORM
First go to the From Builder view:

75. STEP 1: CREATE TAG FILE
Push add to create a new tag definition:
Let’s call it foss4gasia_yourname.

76. STEP 2: ADD A SECTION
Sections are what will be a single button in the geopaparazzi
add note view. Let’s call this buildings.

77. STEP 3: ADD A FORM
Forms are seen in geopaparazzi as different tabs (General,
Structural, Images).

78. STEP 4: ADD DIFFERENT WIDGETS
Widgets are textfields, checkboxes, comboboxes etc.

79. The widget has a few important notions:
key: that will be used also as column in shapefiles export
isLabel: is set to true, in the geopaparazzi map, the text
will be shown as the point label
isMandatory: is set to true, geopaparazzi will require that
value to be set.

81. You should get on the server:

82. NOW PLEASE CREATE THE OTHER
TWO TABS (STRUCTURAL & IMAGES)

83. MAKE YOUR FORM VISIBLE WHEN
YOU ARE READY
Until you are not done, the form doesn’t have to be visible to
the sync app.

84. DOWNLOAD THE FORM (GSSS)
To get the forms on your device just open the GSSS and
enter the Tags Download section. Refresh to get the list and
download.

85. LOAD THE FORM (GEOPAPARAZZI)
Nothing needs to be done in geopaparazzi to load the new
forms, since the forms are saved directly where
geopaparazzi will look for them. Just open the app and see if
the new Buildings button is added to the existing notes
buttons:

86. NOW YOU ARE READY! GO FOR YOUR
NOW YOU ARE READY! GO FOR YOUR
NOW YOU ARE READY! GO FOR YOUR
NOW YOU ARE READY! GO FOR YOUR
NOW YOU ARE READY! GO FOR YOUR
NOW YOU ARE READY! GO FOR YOUR
NOW YOU ARE READY! GO FOR YOURNOW YOU ARE READY! GO FOR YOUR
SURVEY!
SURVEY!
SURVEY!
SURVEY!
SURVEY!
SURVEY!
SURVEY!SURVEY!

87. GET THE SURVEY DATA IN THE
SYSTEM
Once back from the survey, GSSS can be used to load the
incremental (only new notes and logs are synchronized)
survey to the server.

88. THE SYNCHRONIZATION VIEW
If you never loaded a geopaparazzi project before, it is
simple to load one (*.gpap):

89. SYNCHRONIZE THE DATA
Once you choose a project, the notes, logs and images will
be visible in 3 distinct tabs.

90. CLEAN UP AND MAKE DIRTY
From the side menu it is also possible to access some tricky
settings:
the possibility to reset the connected database to be in a
complete dirty state. After that the database will upload
everything as if it never had done before.
the possibility to reset the database to a clean state. After
that no data are synchronized. Only new data surveyed in
geopaparazzi will be uploaded again.
 these settings are dangerous tools, be careful.

91. SEND A LOG
If you experience issues or problems of any kind, you can
use the send debug log button to send some debug
information about the error to the developers. This will help
them to solve the issue and create a new release.

92. GEOPAPARAZZI: NEVER OUT OF
DATA IN THE FIELD
Find out the latest Geopaparazzi news:
Join the Geopaparazzi user’s discussion:
And there’s a group for Geopaparazzi developers too:
If you prefer Facebook: Geopaparazzi Users Around The
World!
http://jgrasstechtips.blogspot.com/search/label/geopaparazzi
https://groups.google.com/forum/#!forum/geopaparazzi-
users
https://groups.google.com/forum/#!forum/geopaparazzi-
devel
https://www.facebook.com/groups/350587351732480/

93. THE END…​ OR MAYBE NOT?
Process your survey data with gvSIG
Data preparation in gvSIG
Data preparation in Spatialite-Gui
Create forms in gvSIG

94. APPENDIX 0: INSTALL THE
NECESSARY

95. GVSIG INSTALLATION
gvSIG is available for different platforms
download the most recent release for your favorite
platform from the
you can choose to use the portable version which is a zip
file containing all the data you need to run the program
otherwise you can use (workshop folder):
for windows machines gvSIG-desktop-2.4.0-2850-final-
win-x86_64-standard.exe
for linux machines gvSIG-desktop-2.4.0-2850-final-lin-
x86_64-standard.run
gvSIG website

96. GVSIG PLUGIN INSTALLATION
the plugins are supplied as a gvSIG package using the
gvSIG addons manager
enter the Addons manager from the Tools menu
select the third option Installation from URL and then
select the url of the Testing version push on next

97. In the Fast Filter search for the plugins that start with
org.hortonmachine.gvsig and install them

98. THE MOBILE FORMS DESIGNER
PLUGIN
Then do the same with the Mobile forms designer plugin.
Once everything is installed, restart gvSIG.
If everything worked properly, you should have the
following menu:

99. APPENDIX 1: PROCESS YOUR
SURVEY DATA WITH A DESKTOP GIS

100. VIEW GEOPAPARAZZI PROJECTS
Open the geopaparazzi projects viewer

101. The viewer allows to choose a folder of existing projects.
Select the folder and hit the refresh button.

102. To load the data of a project, right click on it and push load
project, double click to expand the content.

103. OPEN AN IMAGE
For the projects select to zoom to right click and Load
Project in Viewer, for every item (notes, tracks, images) right
click for options (ex open image).

104. OPEN A GPS LOG

105. EDIT METADATA
Right click on the project to edit metadata.

106. CONVERT A PROJECT TO GIS DATA
In gvSIG do an add layer, it will present you Geopaparazzi as
an option

107. When loading a project, the metadata of the project are
checked and a list of available data is presented.
1. in the upper part we have the
project metadata
2. the list of available layers that
will be created on import is
shown
3. it is possible to create memory
layers instead of shapefiles, but
it is not recomended

108. The new layers of gps logs, notes and media are created
containing most possible information in the attributes table

109. APPENDIX 2: DATA PREPARATION IN
GVSIG

110. PREPARE AN MBTILES
Launch gvSIG and create a new view. Then enter the
properties and set the right projection. It needs to be the
one of the data (EPSG:4326).

111. Then drag the shapefiles and the raster in the view and put
the layers in the proper order:

112. Now open the integrated tiles creator:
set name and folder and (very important) the max zoomlevel
(15 for testing here)…​ and push the OK button.

113. APPENDIX 3: DATA PREPARATION IN
SPATIALITE

114. SPATIALITE GUI
Open the spatialite gui and create a new database

115. Once the db is created, select the Load Shapefile button

116. Import the shapefiles: example of buildings

117. Important, the Update Layer Statistics command needs to
be executed manually Make a refresh of the database
before this operation.

118. Check your data with the map preview

119. LOAD THE DATA ON THE DEVICE
When connecting via USB-cable, we experiences many times
database corruptions, so we started using the app: Airdroid

120. APPENDIX 4: CREATE A SIMPLE
FORM IN GVSIG

121. WHAT ARE FORMS
Geopaparazzi supports complex notes called form based
notes. To use them, a tags.json file needs to be created and
placed inside the geopaparazzi folder inside the sdcard. In
that file a json format description of the wanted tags and
forms needs to be placed.
By default, to help the user to start, a sample tags.json is
created in the Geopaparazzi installation. It contains form
samples and examples to show all supported form widgets.

122. CREATE A FORM WITH GVSIG
gvSIG has a simple yet powerful form designer, created by
the gvSIG Association, which is accessible by the HMachine
menu.

123. CREATE A SECTION
Before doing anything, the section needs to be created. The
section is what in geopaparazzi will be a button.

124. CREATE A FORM
A Form is a tab in the actual geopaparazzi form. Let’s add a
General Info tab from the form toolbar.

125. CREATE FIELDS
Once a form (tab) has been created, fields can be added to
it. Fields can be of various types. Let’s add a string and a
boolean.

126. PREVIEW THE FORM
At any time one can preview the form in the preview panel.
The first time you need to save the form before the preview.

127. SAVE TO TAGS.JSON
the form can be saved to
file at any moment.
it is best to name the file
tags.json, since that is
the name geopaparazzi
will look for.
Upload the form to your device into the folder
geopaparazzi.

128. CHECK THE FORM ON THE DEVICE
The form will appear in the add notes view.