1. Using Open Source Software
For Mobile Data Collection
Andy Faust, GISP
North Central Wisconsin Regional
Planning Commission
WLIA Annual Conference
Lake Geneva, Wisconsin
February 13, 2013

2. Agenda
• Overview of the ODK Project
• Backend Storage Options
• Install ODK Collect
• Design Custom Forms
• Data Collection
• Analyze / View / Export Data
• Questions / Review

3. Open Data Kit
• Open Data Kit (ODK) is a suite of tools that allows data
collection using mobile devices and data submission to an
online server, even without an Internet connection or mobile
carrier service at the time of data collection.
• Created by developers at the University of Washington's
Computer Science and Engineering department , Members
of Change, and Google.
Open Data Kit is an open-source project available to all.
1

4. Open Data Kit
2
•http://www.youtube.com/watch?v=IRYzf0Xg3qw&feature=player_embedded

5. Opendatakit.org
• Open Data Kit (ODK) is a free and open-source set of
tools which help organizations author, field, and
manage mobile data collection solutions. ODK
provides an out-of-the-box solution for users to:
1. Build a data collection form or survey online web
application (XLSForm is recommended for larger
forms);
2. Collect the data on a mobile device to send it to a
server. The application on the device;
3. Aggregate the collected data on a server and
extract it useful formats
3

6. Aggregate
• provide blank forms to ODK Collect,
• accept finalized forms (submissions) from ODK Collect and
manage collected data,
• visualize the collected data using maps and simple graphs,
• export data (e.g., as CSV files for spreadsheets, or as KML
files for Google Earth), and
• publish data to external systems (e.g., Google Spreadsheets
or Google Fusion Tables).
4

7. Aggregate
• ODK Aggregate can be deployed on Google's App
Engine, enabling users to quickly get running
without facing the complexities of setting up their
own scalable web service.
• ODK Aggregate can also be deployed locally on a
Tomcat server (or any servlet 2.5-compatible web
container) backed with a MySQL or PostgreSQL
database server.
5

8. Install Aggregate
• appengine.google.com
• Run web application on Google’s server
• http://code.google.com/p/opendatakit/
Once you are signed up you need to create an
application, so just click on the button “Create an
Application” and give your application a name (called
“application identifier”). This name needs to be unique
among all users applications, so it might take a while to
find a unique one. Save your new application. After you
have created your first application you need to specify
your cell phone number. Google will then send you a
SMS with a code that you enter into their site. This
confirms that you are the owner of this Google App
Engine account.
6

9. Overview of ODK System
Server
Forms
Builder
Collect
Data
Store
Mobile
Collection
XML
Forms
•App Engine
•Local Server
•Form Hub
•ODK Build
•XLS Forms
•XML
•Text
•Numbers
•Date
•Pictures
•Video
•Sound
•Barcodes
7

11. https://appengine.google.com/
https://developers.google.com/appengine/docs/billing
8

12. 9
Verify your Google App Engine Account

13. 10
Enter Verification code

14. 11
Sign in to your Google App Engine Account using your Gmail Account

15. 12

16. 13
Remember this for install
wlia9999

17. 14

18. Download ODK Aggregate v1.2.0 windows-installer.exe
15
http://code.google.com/p/opendatakit/downloads/list

19. 16

20. 17
1 2
3 4

21. 18
1 2
3 4

22. 19
1 2
3
4

23. 20
1 2
3
4

24. http://build.opendatakit.org/
21
Online web application to create forms. You must register for a free account to use Build
Can be used to create basic survey forms

25. 22
Create a basic text question. To add new fields to the survey select type from bottom.

26. 23
Add a numeric question.

27. 24
Add a media question to take a picture

28. 25
Add a question with select multiple

29. 26
Add question with choose one

30. 27
Save form and export to App Engine Enter App ID from page 13

31. 28
Sample Form on App Engine

32. 29
Backend Options
Google App Engine Local Server Formhub.org
PROS PROS PROS
No Hardware Required More Control Free
Low Cost More Secure Easy to setup
Easy to setup No Hardware Required
CONS CONS CONS
Cost Cost Loss of Control
Loss of Control Hardware Required Less Secure
Less Secure Harder to setup Form Limitations

34. http://www.apachefriends.org/en/xampp-windows.html
30
XAMPP is a free and open source cross-platform web server solution stack package
consisting mainly of Apache and MySQL database.
XAMPP can be helpful to get your own local server up and running.

35. 31

36. 32

37. 33

38. 34

39. http://localhost
35

41. 36

42. 37

43. http://www.mysql.com/downloads/connector/j/5.1.html
C:xampptomcatlib
38

44. 39

45. 40

46. Type = cmd
CD C:xamppmysqlbin
41

47. 42

48. 43

49. copy the ODKAggregate.war to the webapps directory of the Apache Tomcat installation
44

50. 45

52. 46
Formhub uses the ODK source code to create an online application that can be used for free

53. 47
Create formhub sign in

54. 48
Fill in registration and wait for email confirmation

55. For help: https://formhub.org/getting_started/
49
Sign in and give formhub a try.

57. text Text input.
integer Integer (ie, whole number) input.
decimal Decimal input.
select_one [options] Multiple choice question; only one answer can be selected.
select_multiple [options] Multiple choice question; multiple answers can be selected.
note Display a note on the screen, takes no input.
geopoint Collect GPS coordinates.
image Take a photograph.
barcode
Scan a barcode, requires the barcode scanner app is
installed.
date Date input.
datetime Accepts a date and a time input.
audio Take an audio recording.
video Take a video recording.
calculate Perform a calculation.
Question Types
50

58. start Start date and time of the survey.
end End date and time of the survey.
today Day of the survey.
deviceid IMEI (International Mobile Equipment Identity)
subscriberid IMSI (International Mobile Subscriber Identity)
sim_serial SIM serial number.
phone_number Phone number (if available).
Metadata
51

59. 52
Survey tab – Were database fields and questions are added to your survey.

60. Column Description
name name of database field (no spaces in field name)
label What is displayed on the screen
hint Hint that is displayed on the screen
constraint constraint used to validate input….example: . > 0 and . < 120
constraint_message constraint message displayed if constaint is not met
required "yes" if the input is required
appearance numbers, maps, year, annotate, draw, signature, compact
default enter a default value
calculation displays a value calculated from another field
choice_filter Used for Cascading Selects
Survey Tab – Column Definitions
53

61. 54
Choices tab – Were selection list are located

62. 55
Settings tab – Were name and version of form are set

63. Choices Tab
Column Description
list_name option from the select
name value that is added to database
label what is displayed on survey screen
Settings Tab
Column Description
form_title Form Title
form_id Form ID
public_key Public ID Key used for encryption
submission_url Submission URL
version Form Version
1.the form ID should be short (ideally < 10 characters).
2.the form ID should be unique within your organization.
3.the form ID must not contain any spaces or punctuation characters.
4.the form ID should contain only alphanumeric characters and the characters '_' and '-'.
5.the form ID should start with a letter.
6.it may be useful to add a version designation (e.g, "medinfo-01") to aid in version tracking as you revise your
forms.
56
Form ID yyyymmddnn
yyyy = year ie. 2013
mm = month ie. 02
nn = form iteration within the given day

64. 57
Sample Survey ODK_WorkshopformWLIA1.xls - survey tab

65. 58
Sample Survey ODK_WorkshopformWLIA1.xls - choices tab

66. 59
Sample Survey ODK_WorkshopformWLIA1.xls - settings tab

68. 60
Samples shown using local server used for Wisconsin Hazard & Assessment & Mapping (WHAM)

69. 61
Pie Chart Display Example

70. 62
Map Display Example

71. 63
Publish to Google Fusion Tables or Google Spreadsheets

72. 64
Publish to Google stay active and live. As new data is collected it is pushed to Fusion
tables or Spreadsheets

73. 65
Google Fusion Table

74. 66
Google Fusion Table Map Display

75. 67
Change Info Window – Pick what fields are displayed

76. <div class='googft-info-window' style='font-family: sans-serif'>
<img src="{Photo1}" style="width: 150px; vertical-align: top; margin-center: .5em" />
<h2>{Address}</h2>
<b>Description:</b> {Description}<br>
<b>Amount:</b> {Amount}<br>
<b>Insurance:</b> {Insurance}<br>
<b>Classification:</b> {Classification}<br>
<b>Municipality: {mcd}<br>
<b> <a href="{Photo1}" target="_blank">Photo1</a>
<a href="{Photo2}" target="_blank">Photo2</a>
<a href="{Photo3}" target="_blank">Photo3</a> <br>
</div>
68
Change Info Window – Custom using HTML

77. 69
Custom Info window displayed

78. 70
Change Info Window – Pick what fields are displayed

79. 71
From Aggregate to CSV

80. 72
From Aggregate to CSV

81. 73
KML to Google Earth

82. 74
KML to Google Earth

83. 75
Data displayed in Google Earth