SlideShare a Scribd company logo
1 of 12
Download to read offline
University of Leicester 2010: Databases and Web Mapping the
Open Source way

Introduction
There is considerably more to web mapping than Google Mash-ups. While the approach outlined in this
section is a bit more complicated, it's a lot more powerful, flexible and interesting, and you have
complete control over your maps.
There are three parts to this section: databases, map servers and web servers. It's not possible to go into
full detail about any of these areas in the time available, but hopefully you will go away with an
overview of how things work and an idea about where to go for more information.
Hopefully, this section will allow you to load data into a spatial database and learn how to work with
and interrogate it. You will then use a map server and web server to display it in a web-based map over
suitable base data. There will be brief digressions into displaying your data in a desktop GIS, and into
using web mapping services. Finally we'll look at some of the more advanced and interesting new
developments in web mapping.

Practicalities
Please download www.archaeogeek.com/downloads/leics2010.zip and extract it to /home/user.
Open Quantum GIS (geospatial/desktop GIS/Quantum GIS) and enable the following plugins:
Mapserver Export, SPIT, and PostGIS Manager (Plugins/Manage Plugins then tick the boxes).

Databases
The database we are going to use is PostgreSQL (http://www.postgresql.org), which is a server-based
multi-user database with support for storing spatial vector data such as points, lines, and polygons. The
difference between server-based databases such as PostgreSQL and file-based databases such as
Microsoft Access is that there is complete separation between the data, stored on the server, or host,
and the software used to access the data on the remote pc, or client. The software used to access the
data is usually one of the following:
1.PSQL- a command line package bundled with the server installation
2.PgAdmin3- with a graphical user interface allowing access to tables, management tasks and queries
3.PhpPgAdmin- a web-based user interface similar to PgAdmin3
4.Desktop GIS packages such as Quantum GIS and GvSIG
5.Map Servers
PostGIS is an extension for PostgreSQL that provides additional functionality such as spatial queries
(within, distance from, inside and so on) and coordinate systems. These allow you to do complex
spatial queries and management of your data without needing any additional software.

Basics
Data is stored on the HOST server in a DATABASE, and accessed using a USERNAME, PASSWORD,
and PORT. The HOST parameter is likely to be an IP address (numbers of the form 192.168.3.23 which
form the unique address of the server on the network), a name, or “localhost” if you are connecting
directly from the server. The port parameter is usually the default 5432.
Each program needs these 5 parameters in order to connect to the database. The Detailed configuration
of user security (who is allowed to access which database, and using which computer) is possible using
text-based config files on the host.
We are using the following parameters:
       host: localhost
       port: 5432
       username: user
       password: user
The way of communicating with the data in a PostgreSQL database is by using Structured Query
Language, or SQL. This is text-based, and human-readable, with a very precise syntax and grammar.
We will cover a small subset of SQL when we learn how to query the data in a later section.
PostGIS has to be installed after PostgreSQL. It creates additional tables in the database to store
information on coordinate systems. PostGIS can either be installed on an individual database, or in the
template from which all new databases are created.
Installing PostgreSQL and PostGIS are beyond the scope of this workshop, but the easiest way to tell if
PostGIS is installed is to check for the presence of the geometry_columns and spatial_ref_sys tables in
the PostgreSQL template database.
Task: Open PgAdmin3 (applications/Development/pgAdmin3) and create a new empty database.
Note the name that you give it, and choose user as the owner, and template_postgis as the
template. Check it is created with the geometry_columns and spatial_ref_sys tables (expand
schemas, public, tables).

Loading data into PostgreSQL
There are several ways of loading spatial data into PostgreSQL. The most common are two command
line packages called OGR2OGR and SHP2PGSQL. We will, however, use the SPIT plugin, which is
part of the Quantum GIS package. The plugin will not be enabled when you first open Quantum GIS.
SPIT requires you to make a connection to the database that you created above, in order to access it.
For this, it needs the following parameters: hostname, database name (whatever you called your
database above), username and password.
Task: Make a new connection to the database you created above and connect to it. Remember to
check the “save username and password” boxes!
To load data into the database using SPIT you need the following additional parameters:
1.The location and name of the shapefile
2.The coordinate system of the data
3.The name of the table you wish to load it into (it's easier if you make this all lower-case)
The sample data supplied is in EPSG 4326 format.
Task: Load the united_kingdom_counties shapefile into your database with an SRID of 4326
(untick the “use default srid” box).
You can check that the data has loaded using PostGIS Manager- this will show you the text-based
attributes of the data. The geometry of the data will be stored in the geometry column in the table,
called the_geom. Note that other importers call it wkb_geometry. The different names do not matter, as
the postgis-enabled database contains a table called geometry_columns that lists the geometry column
used by each table, and the coordinate system in use. This is automatically filled in during the import.
TASK: Use PostGIS Manager to check your shapefile has loaded, and also to check that the
geometry_columns table has been filled in.
Note that it is good database behaviour to ensure your data has a primary key (in this case called gid),
and in the case of spatial data that it has a spatial index too. This has to be done in a query window
(more below) and has the syntax:
CREATE INDEX indexname ON tablename USING GIST(geometryfield);


TASK: Use the SQL window in PostGIS Manager to add a spatial index to the
united_kingdom_counties table, using the geometry field the_geom.

Displaying Data
Displaying data stored in PostgreSQL tables is easy in Quantum GIS, using the Layer/Add PostGIS
Layer option. The same parameters are required to make a connection to the database, as with using
SPIT.
TASK: Display the united_kingdom_counties layer in Quantum GIS.

Querying data
The data in PostgreSQL can be queried using Structured Query Language as described above. The
basic syntax is as follows:
SELECT columns FROM table WHERE some condition is met;
A complete guide to using SQL is beyond the scope of this workshop but good cheatsheets can be
found at (for example) http://www.cheat-sheets.org/. In brief:
• SELECT * FROM table;
   This will select all the data in all the fields from a given table.
• Logical and Arithmetic operators can be used in the WHERE condition to choose data that meets a
  given parameter (or parameters).
• In the WHERE condition, text is enclosed in commas 'like this' but numeric data is not.
• The wildcard character % can be used in the WHERE condition LIKE, for example to find all names
  that end in 's':
   SELECT * FROM table_with_names_in WHERE “firstname” LIKE ('%s');
To display your query on the map (if appropriate), you need to create a view:
   CREATE VIEW your_view AS SELECT …;
You can then add that to QGIS as if it's any other PostgreSQL layer.
TASK: Use the PostGIS Manager to select all united_kingdom_counties that begin with L and
display them on the map.
Spatial Querying
In a PostGIS-enabled database, it is possible to construct complex spatial queries, like buffers,
intersections, unions, as well as management functions like re-projecting your data into a new
coordinate system. The syntax is an extension to SQL, and many examples can be found in the PostGIS
cheat-sheet and manual (http://postgis.refractions.net/download/postgis-1.4.1.pdf).
Some examples:
1.Find the total extent of a layer:
SELECT st_extent(the_geom) FROM united_kingdom_counties;
This returns a box containing a pair of coordinates representing the upper left and lower right extents of
the layer.
2.Find out which county a particular point is in:
SELECT “PROV3NAME” FROM united_kingdom_counties WHERE
st_within(geomfromtext('POINT(-1.4 52.6)', 4326), the_geom);
TASK: Find out which county the above point is in.

Map Servers
A map server is a program that lives on a host server and accepts requests from a client computer (web
or desk-based) and returns geographic data from a variety of defined data sources. These can be raster
(base-mapping) or vector (points, lines and polygons).
In its most simple form that map server packages up all the defined data sources and returns a simple
flat image to the client. Given additional parameters, the map server can return an image of a limited
subset of the data sources, and for a given bounding box or coordinate system.

Types of Map Server
There are two main open source map servers available. These are:
1.Minnesota Mapserver (usually abbreviated to Mapserver: http://mapserver.org/). This is a cgi
executable, ie a program that runs when required by a web server. Configuration is done using a text
file called a map file.
2.Geoserver (http://geoserver.org). This is a java-based package with a web interface for configuration.
There are strengths and weaknesses to both packages. It is possible to run both packages on the one
server, returning data from the same data sources. In this course we will just be using Minnesota
Mapserver.

Installation
Detailed installation instructions are beyond the scope of this tutorial, but information can be found for
all major operating systems at the mapserver website. In brief, the program comprises the main
executable, mapserv, which must be placed in a location accessible to the web server, and a series of
utility tools for working with maps and data.
       We can test mapserver at the command line and check it's version number, and the formats it
       can read by entering:
/usr/lib/cgi-bin/mapserv -v


We can also test mapserver using a web browser by going to:
       http://localhost/cgi-bin/mapserv
The response “no query information to decode. QUERY_STRING is empty” actually means it's
working.
TASK: Check your mapserver installation using both these methods.

Map File Syntax
The map file is a text file with a .map extension. It must live in a location accessible to the web server.
It comprises three basic sections, of which the second two are subsets of the first.
   1. Map
       This defines global configuration options such as the name, the maximum extent, where to find
       the data, and supporting files, and the level of error reporting.
   2. Web
       This contains the locations of the folders where mapserver stores temporary files and images.
   3. Layer
       Each data source in the map is defined as a layer, along with it's type, connection information
       and styling.
Below is a very simple map, with one single layer, showing the hierarchy of the sections. Note that
each section ends with the keyword END. Note also that, although mapserver is not case-sensitive, it is
standard practice to put keywords in capitals and text strings in quotation marks (numeric data should
not be in quotation marks).
If a map is to be used in a dynamic fashion, such as on a web site, it normally has a number of
additional sections such PROJECTION, which defines it's coordinate system, and CLASS, which
contains the styling information for a layer.
MAP
      NAME "sample"
      STATUS ON
      SIZE 600 400
      EXTENT -180 -90 180 90
      SHAPEPATH "../data"
      IMAGECOLOR 255 255 255
      #
      # Start of web interface definition
      #
      WEB
          IMAGEPATH "/ms4w/tmp/ms_tmp/"
          IMAGEURL "/ms_tmp/"
      END

      #
      # Start of layer definitions
      #
      LAYER
          NAME 'global-raster'
TYPE RASTER
            STATUS DEFAULT
            DATA bluemarble.gif
      END
END


Visualising a map file
There are a number of ways to quickly view a mapfile:
1.Shp2img
This is a utility bundled with most mapserver installations that generates an image from a map file. The
syntax is as follows (all on one line in a command window):
       shp2img -m /home/user/leics2010/demo.map -o /home/user/leics2010/demo.png

2.Checking using a browser
This method is more complex but will generate an error message rather than a blank image in the event
of an error, so can be more informative. The basic syntax is as follows, but note that this differs slightly
from operating system to operating system (all on one line in your browser address window):
       http://localhost/cgi-bin/mapserv?map=/home/user/leics2010/demo.map&mode=map


TASK: Test demo.map with the two methods above.

Adding new layers to a map
Adding additional layers is simply a case of adding additional layer declarations to the map file. Layers
are drawn in the order they appear in the map file, so the ones you want to be drawn on top should be
placed after your background layers.
The STATUS keyword dictates whether the layer is drawn automatically (DEFAULT) or on request
(ON/OFF). Use DEFAULT to test your map.
The TYPE keyword indicates the geometry of the layer. Values include RASTER,LINE, and
POLYGON.
Note that the symbology in demo.map is very simple, and will only work for polygon layers. Normally
it is necessary to define a symbols text file that includes your definition for the symbols you wish to
use, and then to refer to that in your map file.
TASK: Explore the shapefiles available in /home/user/shape_files using Quantum GIS. Find a
suitable polygon layer to display on top of the united_kingdom_counties layer and add it to the
map file. Test it using the browser or using Shp2Img. Remember to switch the STATUS to
DEFAULT.

Displaying single layers
The STATUS keyword dictates whether a layer is always visible (DEFAULT), or should be requested
(ON/OFF). ON/OFF are used in other applications where layers are toggled on or off- OFF dictates that
the layer is initially switched off when the map is loaded.
Set the Layer status to “ON” rather than “DEFAULT”, and then specify the layers as part of the URL:
http://localhost/cgi-bin/mapserv?map=/home/user/leics2010/demo.map&layer=layer1

Connecting to a PostgreSQL data source using Mapserver
Mapserver uses the 5 parameters defined earlier to connect to a PostGIS-enabled PostgreSQL data
source. The CONNECTIONTYPE parameter must also be used, and set to PostGIS. Additionally the
DATA parameter must include an SQL query that selects some or all of the geometry from the specified
table. Note that the SELECT keyword is omitted from the query, and all the query terms must be in
lower-case.
       LAYER
         NAME "UK Counties"
         STATUS ON
         TYPE POLYGON
         CONNECTIONTYPE POSTGIS
         CONNECTION "host=localhost port=5432 dbname=leics2010 user=user
       password=user"
         DATA "the_geom from united_kingdom_counties"
       END

TASK: Change the united_kingdom_counties layer from a shapefile to PostgreSQL layer, and test
it.

Web Servers and Web Pages

Introduction
A web server is a program that sits on a host and responds to remote requests from a client pc using
http (hypertext transfer protocol). This is normally in the form of a web page that is then displayed in a
browser. The web page simply needs to be stored in a location accessible to the web server.
The simplest form of web page generally has the suffix html (it is written in hypertext markup
language). The client requests a page using its URL (Uniform Resource Locator- usually the name of
the web site, plus the directory the page is stored in, plus the name of the page) and the web server
sends that page to the client. The client browser renders the page, and sends off further requests for the
additional resources referred to in the page, such as images, or styling information.
Web servers look for web pages in a particular location. This varies (unfortunately) from installation to
installation. In this case it is /var/www. The pages will be accessible via a web browser at the URL
http://localhost.
TASK: Allow write access to the web folder by typing the following at a command prompt:
       sudo chown -R user /var/www
       sudo chmod -R 755 /var/www
and when it asks for a password type 'user ' (no quotes).

Basic structure of a web page
Web pages have a very simple pre-defined structure. Like map files, they contain a hierarchy of
elements that perform different functions, in this case defined by opening and closing tags. There can
be only one set of each tags. The basic structure is as follows:
       <html>
             <head>
<title> My Home Page </title>
             </head>
             <body>
                   HELLO WORLD!
             </body>
       </html>


The <head> section is used to tell the browser information about the page and how it should be run. In
addition to the title (which will appear in the title bar of the browser, it may contain metadata about the
language of the page, and links to external files for styling or scripting.
In the <body> section of the page, everything outside of a pair of angle brackets will be displayed on
the web page. As well as plain text, such as in the example above, other attributes (keywords in angle
brackets) can be used for styling, formatting text as a link to another web page, or showing an image.
By using “view source” in a web browser, you can view the structure of any web page.
Task: Use Mousepad to create a sample web page based on the syntax above and save as
demo1.html in /var/www. View it in a web browser by going to http://localhost/demo1.html

Scripting
Scripting is a way of getting a web page to do more than display static text or images. For example, it is
used to create forms to allow you to submit data to a web site, and to display data from a database or
map.
There are many scripting languages available, but there are only two types:
   1. Server-side
       In this case the web page contains instructions that are interpreted using additional software on
       the web server. The result is then formatted as a standard web page and rendered as such by the
       client (so “view source” shows you the rendered structure rather than the original script).
       Examples of this are pages that display data from a database. The usual language (and software)
       used for this is PHP (PHP Hypertext Processor- this is not a mistake). These pages usually have
       the suffix php. This has the disadvantage that the page must be submitted back to the server for
       information (such as a filled-in form) to be processed.
       For example, these three lines of code in a page will give you a wealth of information about
       your web server environment.
       <?php
               phpinfo();
       ?>


   2. Client-side
       In this case the browser interprets the instructions after it has received the web page from the
       server. The most common language used is javascript. The script is either placed in the <head>
       element of a standard html web page, in it's own <script> tags, or in an additional file requested
       by the browser (these will usually have the suffix js). This is the approach that we are going to
       be using for our web mapping.
Note that JavaScript is case-sensitive. Many functions use “Camel Case”, whichLooksLikeThis.
Task: Use Mousepad to create a file in /var/www called phpinfo.php with the code above in it,
and run http://localhost/phpinfo.php in your web browser.

Debugging/Solving Problems
Firefox comes with a number of very useful plugins to help debug openlayers problems. The error
console (Tools/Error Console) will highlight syntax errors in your code (such as missing brackets). The
extensions Firebug (http://getfirebug.com/) and HttpFox (https://addons.mozilla.org/en-
US/firefox/addon/6647/) provide more comprehensive help with JavaScript and HTTP calls
respectively.
TASK: Install the add-ons above (you will need to restart your browser)

And finally... Web Mapping

Introduction
There are a number of different open source web mapping options, generally using either client or
server-side scripting to work. We are going to concentrate on openlayers (http://openlayers.org), which
is written in javascript. The information below is largely taken from the excellent opengeo openlayers
workshop (http://workshops.opengeo.org/openlayers-intro/), which also goes into more advanced
mapping.
Task: Go to http://localhost/openlayers/examples in your web browser and take a look at some of
the options. The simplest one is lite.html. Right-click on the web page and select “view source” to
see how the page is put together.

Making a map
You don't really need to know much/any javascript to use openlayers in its basic form, but you do need
to respect the syntax and the order in which the components are declared and used.
This works in the following way:
   1. The OpenLayers library is loaded using the script tag.
   2. There is a positioning element (<div>) in the body of the page, with the id “map” that contains
       the map.
   3. A map is created using JavaScript.
   4. A layer is created using JavaScript, and then added to the map.
   5. The map zooms out as far as possible.
   6. The JavaScript function gets called only after the body tag’s onLoad event.

For example:
<html>
      <head>
            <script src="http://www.openlayers.org/api/OpenLayers.js"></script>
            <script type="text/javascript">
                  var map;
                   function init() {
                        map = new OpenLayers.Map('map');
                        mylayer = new OpenLayers.Layer.MapServer( "World Map",
                              "http://localhost/cgi-bin/mapserv.exe",
                              {map: 'C:/world_mapfile.map'});
                        map.addLayer(mylayer);
map.zoomToMaxExtent();
                  }
            </script>
      </head>
      <body onload="init()">
            <div id="map" style="width: 600px; height: 300px"></div>
      </body>
</html>


Step 1 (red): Getting the OpenLayers Library
<script src="http://www.openlayers.org/api/OpenLayers.js"></script>


The URL points to the location of the JavaScript file that loads OpenLayers. It can be a local file, or the
version hosted on the openlayers web site.

Step 2 (orange): Positioning the map in the web page
<div id="map" style="width: 600px; height: 300px"></div>


The <div> tag is a container with the id “map” for the map that we are creating in our page. Later,
when we initialize the JavaScript map object, we will give it this id.
The size of the map is determined by the size of the element that contains it. Here, we have set the size
of the map to be 600 pixels by 300 pixels.

Step 3 (brown): The Map Object
map = new OpenLayers.Map('map');


There are lots of "maps" in this statement! This line creates a new OpenLayers.Map object with the to
go in the div 'map' and makes it a variable called map so it can be referred to elsewhere in the code.
The OpenLayers.Map object, tells the openlayers javascript that we are using the Map class. This
allows us to access a whole library of pre-built code without needing to write the code ourselves.

Step 4 (blue): Creating a Layer
mylayer = new OpenLayers.Layer.MapServer( "World Map",
      "http://localhost/cgi-bin/mapserv.exe", {map: 'C:/world_mapfile.map'} );
      map.addLayer(mylayer);


Like a Mapserver map file, OpenLayers organizes a single map into several layers or data sources. In
this case, we are adding a new object of the Mapserver layer class, telling it the location of the
mapserver executable, the location of the map file, and the layer we want from it (“World Map”). We
are then assigning it the variable name mylayer, which is then used to load the layer in the map.
Different layer classes need different parameters to be given. Examples of many different layer types
can be seen on the openlayers web site (http://openlayers.org/dev/examples/)
Step 5 (purple): Positioning the Map
map.zoomToMaxExtent();


This code tells the map to zoom out to its maximum extent, which by default is the entire world. It is
possible to set a different maximum extent.
OpenLayers maps need to be told what part of the world they should display before anything will
appear on the page. Calling map.zoomToMaxExtent() is one way to do this. Alternative ways that offer
more precision include using zoomToExtent() and setCenter().

Step 6 (green): Loading the Map
<body onload="init()">


In order to ensure that your document is ready, it is important to make sure that the entire page has been
loaded before the script creating the map is executed.
We ensure this by putting the map-creating code into a function, and then calling it only when the
<body> element is fully loaded.
Task: Copy mapserver.html and save it in /var/www/openlayers/mydemo. View it at
http://localhost/openlayers/mydemo/mapserver.html. Experiment with the lat and lon values
(which are in the default coordinate system of EPSG 4326) and the zoom level to get your map
nicely centred and zoomed. Edit the descriptive text and experiment with the size of the map.
Check for problems using the firefox error console and firebug/httpfox.

Map controls and additional layers
Controls provide additional functionality for your map, such as zoom and scale bars, and mouse
position (the coordinates of your mouse cursor). These can be added as the map is drawn, using the
following syntax:
       map = new OpenLayers.Map('map', {
                     controls: [
                         new OpenLayers.Control.Navigation(),
                         new OpenLayers.Control.PanZoomBar()]}


or after the map is drawn using this syntax:
       map.addControl( new OpenLayers.Control.LayerSwitcher() );

TASK: Take a look at the various controls on offer in
http://localhost/openlayers/examples/controls.html and add some to your map.

Base Mapping
Displaying your data in a web map is one thing, but it looks much better when overlain on some base
mapping or satellite data. Openlayers can access base mapping from a variety of different mapping and
imagery providers, such as Google, Yahoo, Microsoft, OpenStreetmap and so on.
Overlaying data onto base mapping adds two additional factors:
    1. The base mapping and your data may not necessarily be in the same coordinate system. In fact
it is quite unlikely that they will be, as your data is likely to be smaller in scale and the base
        mapping is likely to be world-wide.
        Commercial base layers tend to be in Spherical Mercator coordinates, which has the EPSG
        code 900913 (it looks like GOOGLE as an aide memoire). Your data can be projected either
        server-side by the map server or client-side by the openlayers code.
    2. The layers you overlay onto the base mapping must have a transparent background.
    3. You need a (free) api key from the commercial mapping provider to display their maps on your
       web site.
We are going to cheat and use base mapping in standard EPSG:4326 format!
We need to add a number of additional parameters to our web page:
   1. An additional layer of basic base mapping free from metacarta and set that as the base layer:
       var wms = new OpenLayers.Layer.WMS( "OpenLayers WMS",
             "http://labs.metacarta.com/wms/vmap0", {'layers': “basic”},
             {'isBaseLayer': true} );

   2. Change our layer from OpenLayers.Layer.MapServer to OpenLayers.Layer.WMS, and add
      options for transparency:
       var layer = new OpenLayers.Layer.WMS(“UK Counties”,
       “http://localhost/cgi-bin/mapserv”,
       {'map': “/home/user/demo.map”, 'layers': “UK Counties”, 'transparent': true,
       'format': “png”});
   3. Change the addLayer directive to addLayers:
       map.addLayers([wms,layer]);


TASK: Add your base layers to your web page.

Beyond Openlayers
There are a number of alternative web mapping options than what have been demonstrated here. They
all have strengths and weaknesses and different approaches.
MapGuide Open Source (http://mapguide.osgeo.org) provides a pre-built web server and web mapping
application with a full Graphical User Interface for building your maps (in php or .net) without needing
to know any code.
Mapfish (http://mapfish.org/) builds on openlayers but adds additional libraries to allow you to build
more sophisticated applications, including working with the firefox google earth plugin.
Mapchat (http://mapchat.ca/) uses mapserver and ka-map, an alternative to openlayers and adds the
ability to do real-time chat, like a geospatial instant messenger.
Featureserver (http://featureserver.org/) is another cgi executable that works with openlayers to provide
full read- and write access to your data sources. In other words you can allow web-based editing of
your postgresql data, such as adding and deleting features or changing attribute text.

More Related Content

What's hot

Session 04 pig - slides
Session 04   pig - slidesSession 04   pig - slides
Session 04 pig - slidesAnandMHadoop
 
Import and Export Big Data using R Studio
Import and Export Big Data using R StudioImport and Export Big Data using R Studio
Import and Export Big Data using R StudioRupak Roy
 
H base introduction & development
H base introduction & developmentH base introduction & development
H base introduction & developmentShashwat Shriparv
 
Postgresql Database Administration- Day4
Postgresql Database Administration- Day4Postgresql Database Administration- Day4
Postgresql Database Administration- Day4PoguttuezhiniVP
 
Session 19 - MapReduce
Session 19  - MapReduce Session 19  - MapReduce
Session 19 - MapReduce AnandMHadoop
 
Manipulating Data using DPLYR in R Studio
Manipulating Data using DPLYR in R StudioManipulating Data using DPLYR in R Studio
Manipulating Data using DPLYR in R StudioRupak Roy
 
twp-integrating-hadoop-data-with-or-130063
twp-integrating-hadoop-data-with-or-130063twp-integrating-hadoop-data-with-or-130063
twp-integrating-hadoop-data-with-or-130063Madhusudan Anand
 
MongoDB Replication and Sharding
MongoDB Replication and ShardingMongoDB Replication and Sharding
MongoDB Replication and ShardingTharun Srinivasa
 
Recipe 5 of Data Warehouse and Business Intelligence - The null values manage...
Recipe 5 of Data Warehouse and Business Intelligence - The null values manage...Recipe 5 of Data Warehouse and Business Intelligence - The null values manage...
Recipe 5 of Data Warehouse and Business Intelligence - The null values manage...Massimo Cenci
 
Session 04 -Pig Continued
Session 04 -Pig ContinuedSession 04 -Pig Continued
Session 04 -Pig ContinuedAnandMHadoop
 
Sap basis administrator user guide
Sap basis administrator   user guideSap basis administrator   user guide
Sap basis administrator user guidePoguttuezhiniVP
 
Hive Apachecon 2008
Hive Apachecon 2008Hive Apachecon 2008
Hive Apachecon 2008athusoo
 
Data Migration in Database
Data Migration in DatabaseData Migration in Database
Data Migration in DatabaseJingun Jung
 
R data-import, data-export
R data-import, data-exportR data-import, data-export
R data-import, data-exportFAO
 

What's hot (20)

Comparing SAS Files
Comparing SAS FilesComparing SAS Files
Comparing SAS Files
 
Session 04 pig - slides
Session 04   pig - slidesSession 04   pig - slides
Session 04 pig - slides
 
Import and Export Big Data using R Studio
Import and Export Big Data using R StudioImport and Export Big Data using R Studio
Import and Export Big Data using R Studio
 
H base introduction & development
H base introduction & developmentH base introduction & development
H base introduction & development
 
Postgresql Database Administration- Day4
Postgresql Database Administration- Day4Postgresql Database Administration- Day4
Postgresql Database Administration- Day4
 
Session 19 - MapReduce
Session 19  - MapReduce Session 19  - MapReduce
Session 19 - MapReduce
 
Hspark index conf
Hspark index confHspark index conf
Hspark index conf
 
MongoDB DOC v1.5
MongoDB DOC v1.5MongoDB DOC v1.5
MongoDB DOC v1.5
 
Manipulating Data using DPLYR in R Studio
Manipulating Data using DPLYR in R StudioManipulating Data using DPLYR in R Studio
Manipulating Data using DPLYR in R Studio
 
twp-integrating-hadoop-data-with-or-130063
twp-integrating-hadoop-data-with-or-130063twp-integrating-hadoop-data-with-or-130063
twp-integrating-hadoop-data-with-or-130063
 
MongoDB Replication and Sharding
MongoDB Replication and ShardingMongoDB Replication and Sharding
MongoDB Replication and Sharding
 
The Google Bigtable
The Google BigtableThe Google Bigtable
The Google Bigtable
 
DataBase Management System Lab File
DataBase Management System Lab FileDataBase Management System Lab File
DataBase Management System Lab File
 
Recipe 5 of Data Warehouse and Business Intelligence - The null values manage...
Recipe 5 of Data Warehouse and Business Intelligence - The null values manage...Recipe 5 of Data Warehouse and Business Intelligence - The null values manage...
Recipe 5 of Data Warehouse and Business Intelligence - The null values manage...
 
Session 04 -Pig Continued
Session 04 -Pig ContinuedSession 04 -Pig Continued
Session 04 -Pig Continued
 
Unit 4 lecture-3
Unit 4 lecture-3Unit 4 lecture-3
Unit 4 lecture-3
 
Sap basis administrator user guide
Sap basis administrator   user guideSap basis administrator   user guide
Sap basis administrator user guide
 
Hive Apachecon 2008
Hive Apachecon 2008Hive Apachecon 2008
Hive Apachecon 2008
 
Data Migration in Database
Data Migration in DatabaseData Migration in Database
Data Migration in Database
 
R data-import, data-export
R data-import, data-exportR data-import, data-export
R data-import, data-export
 

Viewers also liked

Oswietlenie led-il003
Oswietlenie led-il003Oswietlenie led-il003
Oswietlenie led-il003skleporac
 
Pre Sell / Condo For Sale in Pasay City - La Verti Residences
Pre Sell / Condo For Sale in Pasay City - La Verti ResidencesPre Sell / Condo For Sale in Pasay City - La Verti Residences
Pre Sell / Condo For Sale in Pasay City - La Verti ResidencesJocel Victorino
 
Sybase ase solaris-containers
Sybase ase solaris-containersSybase ase solaris-containers
Sybase ase solaris-containersodd23
 
Moo cs for professional development tcea feb 2015
Moo cs for professional development tcea feb 2015Moo cs for professional development tcea feb 2015
Moo cs for professional development tcea feb 2015Kay Abernathy, Ed.D.
 
Player Performance Index
Player Performance IndexPlayer Performance Index
Player Performance IndexMattar M'Boge
 
Imprimir -first_alternative_-_solaris_10_zone_configuration_and
Imprimir  -first_alternative_-_solaris_10_zone_configuration_andImprimir  -first_alternative_-_solaris_10_zone_configuration_and
Imprimir -first_alternative_-_solaris_10_zone_configuration_andodd23
 

Viewers also liked (7)

Portable echo sounder
Portable echo sounderPortable echo sounder
Portable echo sounder
 
Oswietlenie led-il003
Oswietlenie led-il003Oswietlenie led-il003
Oswietlenie led-il003
 
Pre Sell / Condo For Sale in Pasay City - La Verti Residences
Pre Sell / Condo For Sale in Pasay City - La Verti ResidencesPre Sell / Condo For Sale in Pasay City - La Verti Residences
Pre Sell / Condo For Sale in Pasay City - La Verti Residences
 
Sybase ase solaris-containers
Sybase ase solaris-containersSybase ase solaris-containers
Sybase ase solaris-containers
 
Moo cs for professional development tcea feb 2015
Moo cs for professional development tcea feb 2015Moo cs for professional development tcea feb 2015
Moo cs for professional development tcea feb 2015
 
Player Performance Index
Player Performance IndexPlayer Performance Index
Player Performance Index
 
Imprimir -first_alternative_-_solaris_10_zone_configuration_and
Imprimir  -first_alternative_-_solaris_10_zone_configuration_andImprimir  -first_alternative_-_solaris_10_zone_configuration_and
Imprimir -first_alternative_-_solaris_10_zone_configuration_and
 

Similar to University of Leicester 2010: Databases and Web Mapping the Open Source way

Getting Started with PostGIS geographic database - Lasma Sietinsone, EDINA
Getting Started with PostGIS geographic database - Lasma Sietinsone, EDINAGetting Started with PostGIS geographic database - Lasma Sietinsone, EDINA
Getting Started with PostGIS geographic database - Lasma Sietinsone, EDINAJISC GECO
 
Spatial Data Integrator - Software Presentation and Use Cases
Spatial Data Integrator - Software Presentation and Use CasesSpatial Data Integrator - Software Presentation and Use Cases
Spatial Data Integrator - Software Presentation and Use Casesmathieuraj
 
map3d_mapguide_postgis
map3d_mapguide_postgismap3d_mapguide_postgis
map3d_mapguide_postgisPrasad PANDIT
 
LIFE GreenYourMove Project - GTFS data
LIFE GreenYourMove Project - GTFS data LIFE GreenYourMove Project - GTFS data
LIFE GreenYourMove Project - GTFS data LIFE GreenYourMove
 
Presentation data collection and gtfs
Presentation data collection and gtfsPresentation data collection and gtfs
Presentation data collection and gtfsLIFE GreenYourMove
 
Spatial query tutorial for nyc subway income level along subway
Spatial query tutorial  for nyc subway income level along subwaySpatial query tutorial  for nyc subway income level along subway
Spatial query tutorial for nyc subway income level along subwayVivian S. Zhang
 
B036407011
B036407011B036407011
B036407011theijes
 
Managing Big data Module 3 (1st part)
Managing Big data Module 3 (1st part)Managing Big data Module 3 (1st part)
Managing Big data Module 3 (1st part)Soumee Maschatak
 
TYBSC IT PGIS Unit II Chapter I Data Management and Processing Systems
TYBSC IT PGIS Unit II Chapter I Data Management and Processing SystemsTYBSC IT PGIS Unit II Chapter I Data Management and Processing Systems
TYBSC IT PGIS Unit II Chapter I Data Management and Processing SystemsArti Parab Academics
 
Hot-Spot analysis Using Apache Spark framework
Hot-Spot analysis Using Apache Spark frameworkHot-Spot analysis Using Apache Spark framework
Hot-Spot analysis Using Apache Spark frameworkSupriya .
 
PHP UK 2020 Tutorial: MySQL Indexes, Histograms And other ways To Speed Up Yo...
PHP UK 2020 Tutorial: MySQL Indexes, Histograms And other ways To Speed Up Yo...PHP UK 2020 Tutorial: MySQL Indexes, Histograms And other ways To Speed Up Yo...
PHP UK 2020 Tutorial: MySQL Indexes, Histograms And other ways To Speed Up Yo...Dave Stokes
 
Rendering OpenStreetMap Data using Mapnik
Rendering OpenStreetMap Data using MapnikRendering OpenStreetMap Data using Mapnik
Rendering OpenStreetMap Data using MapnikGraham Jones
 
Integrating hadoop-data-with-oracle
Integrating hadoop-data-with-oracleIntegrating hadoop-data-with-oracle
Integrating hadoop-data-with-oracleJyotirmoy Pramanik
 
BDAS Shark study report 03 v1.1
BDAS Shark study report  03 v1.1BDAS Shark study report  03 v1.1
BDAS Shark study report 03 v1.1Stefanie Zhao
 
Concepts and Methods of Embedding Statistical Data into Maps
Concepts and Methods of Embedding Statistical Data into MapsConcepts and Methods of Embedding Statistical Data into Maps
Concepts and Methods of Embedding Statistical Data into MapsMohammad Liton Hossain
 

Similar to University of Leicester 2010: Databases and Web Mapping the Open Source way (20)

Getting started with PostGIS geographic database
Getting started with PostGIS geographic databaseGetting started with PostGIS geographic database
Getting started with PostGIS geographic database
 
Getting Started with PostGIS geographic database - Lasma Sietinsone, EDINA
Getting Started with PostGIS geographic database - Lasma Sietinsone, EDINAGetting Started with PostGIS geographic database - Lasma Sietinsone, EDINA
Getting Started with PostGIS geographic database - Lasma Sietinsone, EDINA
 
Concect postgis
Concect postgisConcect postgis
Concect postgis
 
Spatial Data Integrator - Software Presentation and Use Cases
Spatial Data Integrator - Software Presentation and Use CasesSpatial Data Integrator - Software Presentation and Use Cases
Spatial Data Integrator - Software Presentation and Use Cases
 
map3d_mapguide_postgis
map3d_mapguide_postgismap3d_mapguide_postgis
map3d_mapguide_postgis
 
LIFE GreenYourMove Project - GTFS data
LIFE GreenYourMove Project - GTFS data LIFE GreenYourMove Project - GTFS data
LIFE GreenYourMove Project - GTFS data
 
Presentation data collection and gtfs
Presentation data collection and gtfsPresentation data collection and gtfs
Presentation data collection and gtfs
 
Spatial query tutorial for nyc subway income level along subway
Spatial query tutorial  for nyc subway income level along subwaySpatial query tutorial  for nyc subway income level along subway
Spatial query tutorial for nyc subway income level along subway
 
Cassandra data modelling best practices
Cassandra data modelling best practicesCassandra data modelling best practices
Cassandra data modelling best practices
 
B036407011
B036407011B036407011
B036407011
 
Managing Big data Module 3 (1st part)
Managing Big data Module 3 (1st part)Managing Big data Module 3 (1st part)
Managing Big data Module 3 (1st part)
 
TYBSC IT PGIS Unit II Chapter I Data Management and Processing Systems
TYBSC IT PGIS Unit II Chapter I Data Management and Processing SystemsTYBSC IT PGIS Unit II Chapter I Data Management and Processing Systems
TYBSC IT PGIS Unit II Chapter I Data Management and Processing Systems
 
Hot-Spot analysis Using Apache Spark framework
Hot-Spot analysis Using Apache Spark frameworkHot-Spot analysis Using Apache Spark framework
Hot-Spot analysis Using Apache Spark framework
 
ADAPTER
ADAPTERADAPTER
ADAPTER
 
PHP UK 2020 Tutorial: MySQL Indexes, Histograms And other ways To Speed Up Yo...
PHP UK 2020 Tutorial: MySQL Indexes, Histograms And other ways To Speed Up Yo...PHP UK 2020 Tutorial: MySQL Indexes, Histograms And other ways To Speed Up Yo...
PHP UK 2020 Tutorial: MySQL Indexes, Histograms And other ways To Speed Up Yo...
 
Rendering OpenStreetMap Data using Mapnik
Rendering OpenStreetMap Data using MapnikRendering OpenStreetMap Data using Mapnik
Rendering OpenStreetMap Data using Mapnik
 
Postgresql
PostgresqlPostgresql
Postgresql
 
Integrating hadoop-data-with-oracle
Integrating hadoop-data-with-oracleIntegrating hadoop-data-with-oracle
Integrating hadoop-data-with-oracle
 
BDAS Shark study report 03 v1.1
BDAS Shark study report  03 v1.1BDAS Shark study report  03 v1.1
BDAS Shark study report 03 v1.1
 
Concepts and Methods of Embedding Statistical Data into Maps
Concepts and Methods of Embedding Statistical Data into MapsConcepts and Methods of Embedding Statistical Data into Maps
Concepts and Methods of Embedding Statistical Data into Maps
 

More from Joanne Cook

Consuming open and linked data with open source tools
Consuming open and linked data with open source toolsConsuming open and linked data with open source tools
Consuming open and linked data with open source toolsJoanne Cook
 
Intro to the Open Source Geospatial Foundation
Intro to the Open Source Geospatial FoundationIntro to the Open Source Geospatial Foundation
Intro to the Open Source Geospatial FoundationJoanne Cook
 
Intro to Quantum GIS Desktop GIS
Intro to Quantum GIS Desktop GISIntro to Quantum GIS Desktop GIS
Intro to Quantum GIS Desktop GISJoanne Cook
 
Introduction to OSGeo:UK
Introduction to OSGeo:UKIntroduction to OSGeo:UK
Introduction to OSGeo:UKJoanne Cook
 
The Business Case for Open Source GIS
The Business Case for Open Source GISThe Business Case for Open Source GIS
The Business Case for Open Source GISJoanne Cook
 
Open Source and Open Data
Open Source and Open DataOpen Source and Open Data
Open Source and Open DataJoanne Cook
 
Consuming and Publishing Ordnance Survey Open Data with Open Source Software
Consuming and Publishing Ordnance Survey Open Data with Open Source SoftwareConsuming and Publishing Ordnance Survey Open Data with Open Source Software
Consuming and Publishing Ordnance Survey Open Data with Open Source SoftwareJoanne Cook
 
AGI 2010: It's all one big opportunity
AGI 2010: It's all one big opportunityAGI 2010: It's all one big opportunity
AGI 2010: It's all one big opportunityJoanne Cook
 
Worskhop OSGIS2010
Worskhop OSGIS2010Worskhop OSGIS2010
Worskhop OSGIS2010Joanne Cook
 
Worskhop Leicester 2010
Worskhop Leicester 2010Worskhop Leicester 2010
Worskhop Leicester 2010Joanne Cook
 
Lancaster University GIS Course 2010
Lancaster University GIS Course 2010Lancaster University GIS Course 2010
Lancaster University GIS Course 2010Joanne Cook
 
Databases and web mapping the Open Source way
Databases and web mapping the Open Source wayDatabases and web mapping the Open Source way
Databases and web mapping the Open Source wayJoanne Cook
 
How archaeologists use GIS
How archaeologists use GISHow archaeologists use GIS
How archaeologists use GISJoanne Cook
 
The Impact of Open Source
The Impact of Open SourceThe Impact of Open Source
The Impact of Open SourceJoanne Cook
 
Open Source GIS for Local Government
Open Source GIS for Local GovernmentOpen Source GIS for Local Government
Open Source GIS for Local GovernmentJoanne Cook
 
Foss4g Portable Gis
Foss4g Portable GisFoss4g Portable Gis
Foss4g Portable GisJoanne Cook
 

More from Joanne Cook (20)

Consuming open and linked data with open source tools
Consuming open and linked data with open source toolsConsuming open and linked data with open source tools
Consuming open and linked data with open source tools
 
Intro to the Open Source Geospatial Foundation
Intro to the Open Source Geospatial FoundationIntro to the Open Source Geospatial Foundation
Intro to the Open Source Geospatial Foundation
 
Intro to Quantum GIS Desktop GIS
Intro to Quantum GIS Desktop GISIntro to Quantum GIS Desktop GIS
Intro to Quantum GIS Desktop GIS
 
Introduction to OSGeo:UK
Introduction to OSGeo:UKIntroduction to OSGeo:UK
Introduction to OSGeo:UK
 
The Business Case for Open Source GIS
The Business Case for Open Source GISThe Business Case for Open Source GIS
The Business Case for Open Source GIS
 
Open Source and Open Data
Open Source and Open DataOpen Source and Open Data
Open Source and Open Data
 
Consuming and Publishing Ordnance Survey Open Data with Open Source Software
Consuming and Publishing Ordnance Survey Open Data with Open Source SoftwareConsuming and Publishing Ordnance Survey Open Data with Open Source Software
Consuming and Publishing Ordnance Survey Open Data with Open Source Software
 
AGI 2010 Notes
AGI 2010 NotesAGI 2010 Notes
AGI 2010 Notes
 
AGI 2010: It's all one big opportunity
AGI 2010: It's all one big opportunityAGI 2010: It's all one big opportunity
AGI 2010: It's all one big opportunity
 
Worskhop OSGIS2010
Worskhop OSGIS2010Worskhop OSGIS2010
Worskhop OSGIS2010
 
Worskhop Leicester 2010
Worskhop Leicester 2010Worskhop Leicester 2010
Worskhop Leicester 2010
 
Lancaster University GIS Course 2010
Lancaster University GIS Course 2010Lancaster University GIS Course 2010
Lancaster University GIS Course 2010
 
Databases and web mapping the Open Source way
Databases and web mapping the Open Source wayDatabases and web mapping the Open Source way
Databases and web mapping the Open Source way
 
How archaeologists use GIS
How archaeologists use GISHow archaeologists use GIS
How archaeologists use GIS
 
The Impact of Open Source
The Impact of Open SourceThe Impact of Open Source
The Impact of Open Source
 
Open Source GIS for Local Government
Open Source GIS for Local GovernmentOpen Source GIS for Local Government
Open Source GIS for Local Government
 
Bcs Talk Notes
Bcs Talk NotesBcs Talk Notes
Bcs Talk Notes
 
Gateway Seminar
Gateway SeminarGateway Seminar
Gateway Seminar
 
Foss4g Portable Gis
Foss4g Portable GisFoss4g Portable Gis
Foss4g Portable Gis
 
Bcs Talk Notes
Bcs Talk NotesBcs Talk Notes
Bcs Talk Notes
 

University of Leicester 2010: Databases and Web Mapping the Open Source way

  • 1. University of Leicester 2010: Databases and Web Mapping the Open Source way Introduction There is considerably more to web mapping than Google Mash-ups. While the approach outlined in this section is a bit more complicated, it's a lot more powerful, flexible and interesting, and you have complete control over your maps. There are three parts to this section: databases, map servers and web servers. It's not possible to go into full detail about any of these areas in the time available, but hopefully you will go away with an overview of how things work and an idea about where to go for more information. Hopefully, this section will allow you to load data into a spatial database and learn how to work with and interrogate it. You will then use a map server and web server to display it in a web-based map over suitable base data. There will be brief digressions into displaying your data in a desktop GIS, and into using web mapping services. Finally we'll look at some of the more advanced and interesting new developments in web mapping. Practicalities Please download www.archaeogeek.com/downloads/leics2010.zip and extract it to /home/user. Open Quantum GIS (geospatial/desktop GIS/Quantum GIS) and enable the following plugins: Mapserver Export, SPIT, and PostGIS Manager (Plugins/Manage Plugins then tick the boxes). Databases The database we are going to use is PostgreSQL (http://www.postgresql.org), which is a server-based multi-user database with support for storing spatial vector data such as points, lines, and polygons. The difference between server-based databases such as PostgreSQL and file-based databases such as Microsoft Access is that there is complete separation between the data, stored on the server, or host, and the software used to access the data on the remote pc, or client. The software used to access the data is usually one of the following: 1.PSQL- a command line package bundled with the server installation 2.PgAdmin3- with a graphical user interface allowing access to tables, management tasks and queries 3.PhpPgAdmin- a web-based user interface similar to PgAdmin3 4.Desktop GIS packages such as Quantum GIS and GvSIG 5.Map Servers PostGIS is an extension for PostgreSQL that provides additional functionality such as spatial queries (within, distance from, inside and so on) and coordinate systems. These allow you to do complex spatial queries and management of your data without needing any additional software. Basics Data is stored on the HOST server in a DATABASE, and accessed using a USERNAME, PASSWORD, and PORT. The HOST parameter is likely to be an IP address (numbers of the form 192.168.3.23 which
  • 2. form the unique address of the server on the network), a name, or “localhost” if you are connecting directly from the server. The port parameter is usually the default 5432. Each program needs these 5 parameters in order to connect to the database. The Detailed configuration of user security (who is allowed to access which database, and using which computer) is possible using text-based config files on the host. We are using the following parameters: host: localhost port: 5432 username: user password: user The way of communicating with the data in a PostgreSQL database is by using Structured Query Language, or SQL. This is text-based, and human-readable, with a very precise syntax and grammar. We will cover a small subset of SQL when we learn how to query the data in a later section. PostGIS has to be installed after PostgreSQL. It creates additional tables in the database to store information on coordinate systems. PostGIS can either be installed on an individual database, or in the template from which all new databases are created. Installing PostgreSQL and PostGIS are beyond the scope of this workshop, but the easiest way to tell if PostGIS is installed is to check for the presence of the geometry_columns and spatial_ref_sys tables in the PostgreSQL template database. Task: Open PgAdmin3 (applications/Development/pgAdmin3) and create a new empty database. Note the name that you give it, and choose user as the owner, and template_postgis as the template. Check it is created with the geometry_columns and spatial_ref_sys tables (expand schemas, public, tables). Loading data into PostgreSQL There are several ways of loading spatial data into PostgreSQL. The most common are two command line packages called OGR2OGR and SHP2PGSQL. We will, however, use the SPIT plugin, which is part of the Quantum GIS package. The plugin will not be enabled when you first open Quantum GIS. SPIT requires you to make a connection to the database that you created above, in order to access it. For this, it needs the following parameters: hostname, database name (whatever you called your database above), username and password. Task: Make a new connection to the database you created above and connect to it. Remember to check the “save username and password” boxes! To load data into the database using SPIT you need the following additional parameters: 1.The location and name of the shapefile 2.The coordinate system of the data 3.The name of the table you wish to load it into (it's easier if you make this all lower-case) The sample data supplied is in EPSG 4326 format. Task: Load the united_kingdom_counties shapefile into your database with an SRID of 4326 (untick the “use default srid” box). You can check that the data has loaded using PostGIS Manager- this will show you the text-based attributes of the data. The geometry of the data will be stored in the geometry column in the table,
  • 3. called the_geom. Note that other importers call it wkb_geometry. The different names do not matter, as the postgis-enabled database contains a table called geometry_columns that lists the geometry column used by each table, and the coordinate system in use. This is automatically filled in during the import. TASK: Use PostGIS Manager to check your shapefile has loaded, and also to check that the geometry_columns table has been filled in. Note that it is good database behaviour to ensure your data has a primary key (in this case called gid), and in the case of spatial data that it has a spatial index too. This has to be done in a query window (more below) and has the syntax: CREATE INDEX indexname ON tablename USING GIST(geometryfield); TASK: Use the SQL window in PostGIS Manager to add a spatial index to the united_kingdom_counties table, using the geometry field the_geom. Displaying Data Displaying data stored in PostgreSQL tables is easy in Quantum GIS, using the Layer/Add PostGIS Layer option. The same parameters are required to make a connection to the database, as with using SPIT. TASK: Display the united_kingdom_counties layer in Quantum GIS. Querying data The data in PostgreSQL can be queried using Structured Query Language as described above. The basic syntax is as follows: SELECT columns FROM table WHERE some condition is met; A complete guide to using SQL is beyond the scope of this workshop but good cheatsheets can be found at (for example) http://www.cheat-sheets.org/. In brief: • SELECT * FROM table; This will select all the data in all the fields from a given table. • Logical and Arithmetic operators can be used in the WHERE condition to choose data that meets a given parameter (or parameters). • In the WHERE condition, text is enclosed in commas 'like this' but numeric data is not. • The wildcard character % can be used in the WHERE condition LIKE, for example to find all names that end in 's': SELECT * FROM table_with_names_in WHERE “firstname” LIKE ('%s'); To display your query on the map (if appropriate), you need to create a view: CREATE VIEW your_view AS SELECT …; You can then add that to QGIS as if it's any other PostgreSQL layer. TASK: Use the PostGIS Manager to select all united_kingdom_counties that begin with L and display them on the map.
  • 4. Spatial Querying In a PostGIS-enabled database, it is possible to construct complex spatial queries, like buffers, intersections, unions, as well as management functions like re-projecting your data into a new coordinate system. The syntax is an extension to SQL, and many examples can be found in the PostGIS cheat-sheet and manual (http://postgis.refractions.net/download/postgis-1.4.1.pdf). Some examples: 1.Find the total extent of a layer: SELECT st_extent(the_geom) FROM united_kingdom_counties; This returns a box containing a pair of coordinates representing the upper left and lower right extents of the layer. 2.Find out which county a particular point is in: SELECT “PROV3NAME” FROM united_kingdom_counties WHERE st_within(geomfromtext('POINT(-1.4 52.6)', 4326), the_geom); TASK: Find out which county the above point is in. Map Servers A map server is a program that lives on a host server and accepts requests from a client computer (web or desk-based) and returns geographic data from a variety of defined data sources. These can be raster (base-mapping) or vector (points, lines and polygons). In its most simple form that map server packages up all the defined data sources and returns a simple flat image to the client. Given additional parameters, the map server can return an image of a limited subset of the data sources, and for a given bounding box or coordinate system. Types of Map Server There are two main open source map servers available. These are: 1.Minnesota Mapserver (usually abbreviated to Mapserver: http://mapserver.org/). This is a cgi executable, ie a program that runs when required by a web server. Configuration is done using a text file called a map file. 2.Geoserver (http://geoserver.org). This is a java-based package with a web interface for configuration. There are strengths and weaknesses to both packages. It is possible to run both packages on the one server, returning data from the same data sources. In this course we will just be using Minnesota Mapserver. Installation Detailed installation instructions are beyond the scope of this tutorial, but information can be found for all major operating systems at the mapserver website. In brief, the program comprises the main executable, mapserv, which must be placed in a location accessible to the web server, and a series of utility tools for working with maps and data. We can test mapserver at the command line and check it's version number, and the formats it can read by entering:
  • 5. /usr/lib/cgi-bin/mapserv -v We can also test mapserver using a web browser by going to: http://localhost/cgi-bin/mapserv The response “no query information to decode. QUERY_STRING is empty” actually means it's working. TASK: Check your mapserver installation using both these methods. Map File Syntax The map file is a text file with a .map extension. It must live in a location accessible to the web server. It comprises three basic sections, of which the second two are subsets of the first. 1. Map This defines global configuration options such as the name, the maximum extent, where to find the data, and supporting files, and the level of error reporting. 2. Web This contains the locations of the folders where mapserver stores temporary files and images. 3. Layer Each data source in the map is defined as a layer, along with it's type, connection information and styling. Below is a very simple map, with one single layer, showing the hierarchy of the sections. Note that each section ends with the keyword END. Note also that, although mapserver is not case-sensitive, it is standard practice to put keywords in capitals and text strings in quotation marks (numeric data should not be in quotation marks). If a map is to be used in a dynamic fashion, such as on a web site, it normally has a number of additional sections such PROJECTION, which defines it's coordinate system, and CLASS, which contains the styling information for a layer. MAP NAME "sample" STATUS ON SIZE 600 400 EXTENT -180 -90 180 90 SHAPEPATH "../data" IMAGECOLOR 255 255 255 # # Start of web interface definition # WEB IMAGEPATH "/ms4w/tmp/ms_tmp/" IMAGEURL "/ms_tmp/" END # # Start of layer definitions # LAYER NAME 'global-raster'
  • 6. TYPE RASTER STATUS DEFAULT DATA bluemarble.gif END END Visualising a map file There are a number of ways to quickly view a mapfile: 1.Shp2img This is a utility bundled with most mapserver installations that generates an image from a map file. The syntax is as follows (all on one line in a command window): shp2img -m /home/user/leics2010/demo.map -o /home/user/leics2010/demo.png 2.Checking using a browser This method is more complex but will generate an error message rather than a blank image in the event of an error, so can be more informative. The basic syntax is as follows, but note that this differs slightly from operating system to operating system (all on one line in your browser address window): http://localhost/cgi-bin/mapserv?map=/home/user/leics2010/demo.map&mode=map TASK: Test demo.map with the two methods above. Adding new layers to a map Adding additional layers is simply a case of adding additional layer declarations to the map file. Layers are drawn in the order they appear in the map file, so the ones you want to be drawn on top should be placed after your background layers. The STATUS keyword dictates whether the layer is drawn automatically (DEFAULT) or on request (ON/OFF). Use DEFAULT to test your map. The TYPE keyword indicates the geometry of the layer. Values include RASTER,LINE, and POLYGON. Note that the symbology in demo.map is very simple, and will only work for polygon layers. Normally it is necessary to define a symbols text file that includes your definition for the symbols you wish to use, and then to refer to that in your map file. TASK: Explore the shapefiles available in /home/user/shape_files using Quantum GIS. Find a suitable polygon layer to display on top of the united_kingdom_counties layer and add it to the map file. Test it using the browser or using Shp2Img. Remember to switch the STATUS to DEFAULT. Displaying single layers The STATUS keyword dictates whether a layer is always visible (DEFAULT), or should be requested (ON/OFF). ON/OFF are used in other applications where layers are toggled on or off- OFF dictates that the layer is initially switched off when the map is loaded. Set the Layer status to “ON” rather than “DEFAULT”, and then specify the layers as part of the URL:
  • 7. http://localhost/cgi-bin/mapserv?map=/home/user/leics2010/demo.map&layer=layer1 Connecting to a PostgreSQL data source using Mapserver Mapserver uses the 5 parameters defined earlier to connect to a PostGIS-enabled PostgreSQL data source. The CONNECTIONTYPE parameter must also be used, and set to PostGIS. Additionally the DATA parameter must include an SQL query that selects some or all of the geometry from the specified table. Note that the SELECT keyword is omitted from the query, and all the query terms must be in lower-case. LAYER NAME "UK Counties" STATUS ON TYPE POLYGON CONNECTIONTYPE POSTGIS CONNECTION "host=localhost port=5432 dbname=leics2010 user=user password=user" DATA "the_geom from united_kingdom_counties" END TASK: Change the united_kingdom_counties layer from a shapefile to PostgreSQL layer, and test it. Web Servers and Web Pages Introduction A web server is a program that sits on a host and responds to remote requests from a client pc using http (hypertext transfer protocol). This is normally in the form of a web page that is then displayed in a browser. The web page simply needs to be stored in a location accessible to the web server. The simplest form of web page generally has the suffix html (it is written in hypertext markup language). The client requests a page using its URL (Uniform Resource Locator- usually the name of the web site, plus the directory the page is stored in, plus the name of the page) and the web server sends that page to the client. The client browser renders the page, and sends off further requests for the additional resources referred to in the page, such as images, or styling information. Web servers look for web pages in a particular location. This varies (unfortunately) from installation to installation. In this case it is /var/www. The pages will be accessible via a web browser at the URL http://localhost. TASK: Allow write access to the web folder by typing the following at a command prompt: sudo chown -R user /var/www sudo chmod -R 755 /var/www and when it asks for a password type 'user ' (no quotes). Basic structure of a web page Web pages have a very simple pre-defined structure. Like map files, they contain a hierarchy of elements that perform different functions, in this case defined by opening and closing tags. There can be only one set of each tags. The basic structure is as follows: <html> <head>
  • 8. <title> My Home Page </title> </head> <body> HELLO WORLD! </body> </html> The <head> section is used to tell the browser information about the page and how it should be run. In addition to the title (which will appear in the title bar of the browser, it may contain metadata about the language of the page, and links to external files for styling or scripting. In the <body> section of the page, everything outside of a pair of angle brackets will be displayed on the web page. As well as plain text, such as in the example above, other attributes (keywords in angle brackets) can be used for styling, formatting text as a link to another web page, or showing an image. By using “view source” in a web browser, you can view the structure of any web page. Task: Use Mousepad to create a sample web page based on the syntax above and save as demo1.html in /var/www. View it in a web browser by going to http://localhost/demo1.html Scripting Scripting is a way of getting a web page to do more than display static text or images. For example, it is used to create forms to allow you to submit data to a web site, and to display data from a database or map. There are many scripting languages available, but there are only two types: 1. Server-side In this case the web page contains instructions that are interpreted using additional software on the web server. The result is then formatted as a standard web page and rendered as such by the client (so “view source” shows you the rendered structure rather than the original script). Examples of this are pages that display data from a database. The usual language (and software) used for this is PHP (PHP Hypertext Processor- this is not a mistake). These pages usually have the suffix php. This has the disadvantage that the page must be submitted back to the server for information (such as a filled-in form) to be processed. For example, these three lines of code in a page will give you a wealth of information about your web server environment. <?php phpinfo(); ?> 2. Client-side In this case the browser interprets the instructions after it has received the web page from the server. The most common language used is javascript. The script is either placed in the <head> element of a standard html web page, in it's own <script> tags, or in an additional file requested by the browser (these will usually have the suffix js). This is the approach that we are going to be using for our web mapping. Note that JavaScript is case-sensitive. Many functions use “Camel Case”, whichLooksLikeThis. Task: Use Mousepad to create a file in /var/www called phpinfo.php with the code above in it,
  • 9. and run http://localhost/phpinfo.php in your web browser. Debugging/Solving Problems Firefox comes with a number of very useful plugins to help debug openlayers problems. The error console (Tools/Error Console) will highlight syntax errors in your code (such as missing brackets). The extensions Firebug (http://getfirebug.com/) and HttpFox (https://addons.mozilla.org/en- US/firefox/addon/6647/) provide more comprehensive help with JavaScript and HTTP calls respectively. TASK: Install the add-ons above (you will need to restart your browser) And finally... Web Mapping Introduction There are a number of different open source web mapping options, generally using either client or server-side scripting to work. We are going to concentrate on openlayers (http://openlayers.org), which is written in javascript. The information below is largely taken from the excellent opengeo openlayers workshop (http://workshops.opengeo.org/openlayers-intro/), which also goes into more advanced mapping. Task: Go to http://localhost/openlayers/examples in your web browser and take a look at some of the options. The simplest one is lite.html. Right-click on the web page and select “view source” to see how the page is put together. Making a map You don't really need to know much/any javascript to use openlayers in its basic form, but you do need to respect the syntax and the order in which the components are declared and used. This works in the following way: 1. The OpenLayers library is loaded using the script tag. 2. There is a positioning element (<div>) in the body of the page, with the id “map” that contains the map. 3. A map is created using JavaScript. 4. A layer is created using JavaScript, and then added to the map. 5. The map zooms out as far as possible. 6. The JavaScript function gets called only after the body tag’s onLoad event. For example: <html> <head> <script src="http://www.openlayers.org/api/OpenLayers.js"></script> <script type="text/javascript"> var map; function init() { map = new OpenLayers.Map('map'); mylayer = new OpenLayers.Layer.MapServer( "World Map", "http://localhost/cgi-bin/mapserv.exe", {map: 'C:/world_mapfile.map'}); map.addLayer(mylayer);
  • 10. map.zoomToMaxExtent(); } </script> </head> <body onload="init()"> <div id="map" style="width: 600px; height: 300px"></div> </body> </html> Step 1 (red): Getting the OpenLayers Library <script src="http://www.openlayers.org/api/OpenLayers.js"></script> The URL points to the location of the JavaScript file that loads OpenLayers. It can be a local file, or the version hosted on the openlayers web site. Step 2 (orange): Positioning the map in the web page <div id="map" style="width: 600px; height: 300px"></div> The <div> tag is a container with the id “map” for the map that we are creating in our page. Later, when we initialize the JavaScript map object, we will give it this id. The size of the map is determined by the size of the element that contains it. Here, we have set the size of the map to be 600 pixels by 300 pixels. Step 3 (brown): The Map Object map = new OpenLayers.Map('map'); There are lots of "maps" in this statement! This line creates a new OpenLayers.Map object with the to go in the div 'map' and makes it a variable called map so it can be referred to elsewhere in the code. The OpenLayers.Map object, tells the openlayers javascript that we are using the Map class. This allows us to access a whole library of pre-built code without needing to write the code ourselves. Step 4 (blue): Creating a Layer mylayer = new OpenLayers.Layer.MapServer( "World Map", "http://localhost/cgi-bin/mapserv.exe", {map: 'C:/world_mapfile.map'} ); map.addLayer(mylayer); Like a Mapserver map file, OpenLayers organizes a single map into several layers or data sources. In this case, we are adding a new object of the Mapserver layer class, telling it the location of the mapserver executable, the location of the map file, and the layer we want from it (“World Map”). We are then assigning it the variable name mylayer, which is then used to load the layer in the map. Different layer classes need different parameters to be given. Examples of many different layer types can be seen on the openlayers web site (http://openlayers.org/dev/examples/)
  • 11. Step 5 (purple): Positioning the Map map.zoomToMaxExtent(); This code tells the map to zoom out to its maximum extent, which by default is the entire world. It is possible to set a different maximum extent. OpenLayers maps need to be told what part of the world they should display before anything will appear on the page. Calling map.zoomToMaxExtent() is one way to do this. Alternative ways that offer more precision include using zoomToExtent() and setCenter(). Step 6 (green): Loading the Map <body onload="init()"> In order to ensure that your document is ready, it is important to make sure that the entire page has been loaded before the script creating the map is executed. We ensure this by putting the map-creating code into a function, and then calling it only when the <body> element is fully loaded. Task: Copy mapserver.html and save it in /var/www/openlayers/mydemo. View it at http://localhost/openlayers/mydemo/mapserver.html. Experiment with the lat and lon values (which are in the default coordinate system of EPSG 4326) and the zoom level to get your map nicely centred and zoomed. Edit the descriptive text and experiment with the size of the map. Check for problems using the firefox error console and firebug/httpfox. Map controls and additional layers Controls provide additional functionality for your map, such as zoom and scale bars, and mouse position (the coordinates of your mouse cursor). These can be added as the map is drawn, using the following syntax: map = new OpenLayers.Map('map', { controls: [ new OpenLayers.Control.Navigation(), new OpenLayers.Control.PanZoomBar()]} or after the map is drawn using this syntax: map.addControl( new OpenLayers.Control.LayerSwitcher() ); TASK: Take a look at the various controls on offer in http://localhost/openlayers/examples/controls.html and add some to your map. Base Mapping Displaying your data in a web map is one thing, but it looks much better when overlain on some base mapping or satellite data. Openlayers can access base mapping from a variety of different mapping and imagery providers, such as Google, Yahoo, Microsoft, OpenStreetmap and so on. Overlaying data onto base mapping adds two additional factors: 1. The base mapping and your data may not necessarily be in the same coordinate system. In fact
  • 12. it is quite unlikely that they will be, as your data is likely to be smaller in scale and the base mapping is likely to be world-wide. Commercial base layers tend to be in Spherical Mercator coordinates, which has the EPSG code 900913 (it looks like GOOGLE as an aide memoire). Your data can be projected either server-side by the map server or client-side by the openlayers code. 2. The layers you overlay onto the base mapping must have a transparent background. 3. You need a (free) api key from the commercial mapping provider to display their maps on your web site. We are going to cheat and use base mapping in standard EPSG:4326 format! We need to add a number of additional parameters to our web page: 1. An additional layer of basic base mapping free from metacarta and set that as the base layer: var wms = new OpenLayers.Layer.WMS( "OpenLayers WMS", "http://labs.metacarta.com/wms/vmap0", {'layers': “basic”}, {'isBaseLayer': true} ); 2. Change our layer from OpenLayers.Layer.MapServer to OpenLayers.Layer.WMS, and add options for transparency: var layer = new OpenLayers.Layer.WMS(“UK Counties”, “http://localhost/cgi-bin/mapserv”, {'map': “/home/user/demo.map”, 'layers': “UK Counties”, 'transparent': true, 'format': “png”}); 3. Change the addLayer directive to addLayers: map.addLayers([wms,layer]); TASK: Add your base layers to your web page. Beyond Openlayers There are a number of alternative web mapping options than what have been demonstrated here. They all have strengths and weaknesses and different approaches. MapGuide Open Source (http://mapguide.osgeo.org) provides a pre-built web server and web mapping application with a full Graphical User Interface for building your maps (in php or .net) without needing to know any code. Mapfish (http://mapfish.org/) builds on openlayers but adds additional libraries to allow you to build more sophisticated applications, including working with the firefox google earth plugin. Mapchat (http://mapchat.ca/) uses mapserver and ka-map, an alternative to openlayers and adds the ability to do real-time chat, like a geospatial instant messenger. Featureserver (http://featureserver.org/) is another cgi executable that works with openlayers to provide full read- and write access to your data sources. In other words you can allow web-based editing of your postgresql data, such as adding and deleting features or changing attribute text.