![[email_address] Mashup Competitions & Data Portals - Where Next? Jose Diacono 6 th October, 2010 Attribution-Share Alike](https://image.slidesharecdn.com/mashupsanddataportalswherenextspatialgov1-101010222139-phpapp01/85/Mashups-and-data-portals-where-next-spatial-gov-1-320.jpg)
![[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
More Related Content
Similar to Mashups and data portals where next? (spatial@gov)
Similar to Mashups and data portals where next? (spatial@gov) (20)
Recently uploaded
Recently uploaded (20)
Mashups and data portals where next? (spatial@gov)
- 1. [email_address] Mashup Competitions & Data Portals - Where Next? Jose Diacono 6 th October, 2010 Attribution-Share Alike
- 19. Home › Blogs › Lisa Price's blog 1ST JULY 2010 Owing to overwhelming demand by apps that use the service, the London Underground feed has had to be temporarily suspended. We hope to restore the service as soon as possible but this may take some days. We will keep everyone informed of progress towards a resolution.
- 20. One feed could provide an entire picture broken down by line and lead car ID with destination & location text once ….. THU, 07/01/2010 12:41 I second the idea of having a call that returns the status of all the trains for feed's 4 and 5, just without the time to destinations. FRI, 07/02/2010 09:45 Let's do this : - each train is an RSS/Atom feed. New events like "train 1234 got in Paddington Station" are the feed entries
- 21. Any news on when access might return? An update would be nice. What's happening TFL? The service has been down for a week now - could we get an update? Over the two week mark of the service being down, any updates at all?
- 22. Hi - it seems these comments are being moderated so perhaps somebody reading this could comment on the status of this API. We're only asking because we like it so much...... 20 TH July So this is it? In true British fashion - it was a resounding success so they turned it off... genius!
- 23. TFL communication is seriously overdue on the APIs future availability… 26 TH AUGUST Why are you being so rude …Any news is better than no news. 3 RD SEPTEMBER
- 25. We are aware from all the comments... a lot of frustration in the developer community Discussions involve a number of stakeholders There is a cost associated with TFL Restored in a sustainable way... within the next 8 weeks... We will tweet and post
- 27. Blogs
- 29. Get ready, the data hounds are coming!!
- 33. United States
- 34. London Datastore
- 36. The data hounds are coming
Editor's Notes
- My interest in mashups and data portals stems from a combination of a technical and marketing background in the GIS industry, website design and development though a family business and data sharing from my work with the Victorian Spatial Council’s data custodianship program. It also stems from impatience. I get very frustrated by unfriendly websites with unattractive interfaces, broken links or that assume knowledge that I do not have. When I heard about government 2.0 taskforce and mashup australia competition I got very excited. I felt it was very relevant to the spatial industry so I wrote a couple of articles for Position Magazine and then got more and more interested in the doors they are opening for us. More recently I have been working with the spatial infrastructure division of the Department of Sustainabililty and Environment in Victoria on the stakeholder communications for a new Spatial Metadata System. This has really opened my eyes to what non specialists need in order to discover our data.
- Why encourage the creation of mashups? The more people have access to more data, the more likely you are to get something really good. There may also be some „unexpected or inconvenient truths‟. Why create them? They are fun, they are useful, they are quick to create – and you can win fame and fortune. Overall it is a satisfying thing to do.
- This was a winning entry in the Victorian mashup competition Appmystate. By Mohammad Shahiduzzaman The standard of presentation is fantastic. A very slick website with uplifting classical music, images that slide through. Very classy, huge amount of talent and effort. Works on iphone or android. RAW – What is it? RAW is Remind AnyWay. Put simply, it is a universal reminder system with special emphasis on location. What’s your motive behind RAW? The motive is simple – we want you to concentrate on your tasks and leave the reminding part to RAW. There are two triggering aspects of any event/task i.e. time, location. Up until now, the time part has been covered by many applications and the location part has been recently covered by only a few. Well, we want to put a full stop on this reminding business. We are offering a universal reminding system. Let’s explain with some common scenarios: Sally went to Box Hill Railway station to drop off a friend. After coming back to home, she just found out that she could have been gone to nearby Coles supermarket to buy some vegetables. Oh! damn! – what a silly mistake! Next time Sally will be reminded by her iphone when she gets close to the Railway Station. Uses Victorian Govt. location data regarding schools, parks, universities, railway stations, markets, hospitals and other landmarks. This includes some of the Vicmap lite data such as railway stations which were released under creative commons licence for the competion.
- This mashup was in the national competition. It allows you to report problems in your area to the appropriate local council, without actually having to know who that is. A the moment they don’t go anywhere, but you can display what other people have reported by local government area. A website called fixmystreet is up and running in the UK.
- A more politically correct victorian version was an iphone app called Snap Send Solve I like this mashup because it is fun and irreverant, yet would be a huge timesaver in reporting things that are broken and affect you personally and getting them fixed. Getting a pothole fixed could save an elderly neighbour falling into it. I also like it because it uses another mashup – and that is the true spirit of mashups. We should be standing on each others shoulders, not reinventing the wheel. It uses the geo2gov engine
- “ Map your address or latitude,longitude into your relationship with government”. Adam Kennedy and Jeffrey Candiloro’s spatial engine Geo2gov won the MashupAustralia Transformation Prize for a mashup that solves a data problem for other hackers. Most hackers were spending an inordinate amount of their time dealing with locating, downloading, cleaning, formatting and otherwise overcoming problems getting access to the data they wanted to use in their application before they even got to start building it. Geo2gov was used by itsbuggeredmate and anyone can use it. Over 36 hour hackfest in Canberra, most of mashups created in last 10 hours. geo2gov takes a location in a variety of different formats (address, postcode, suburb, place name, ip address, etc) and converts them to a GPS location, drills through a whole host of layers to tell you what local government area and ward you are in, who your federal and state elected representatives are so you can link through to OpenAustralia.org to find out what they are saying and doing. It does all this in around a quarter of a second. It gives you a link through to ABS statistics for the page and census information. Geo2gov lives in the cloud (it is hosted on Amazon for the cost of a weekly squash game). But how many people know about it? My son works for a web development company and his colleague was mashing up ABS data so I told him about it but its all a bit haphazard. I interviewed Adam Kennedy. http://www.communica.com.au/mashup-australia/interview-with-a-mashup-hacker-2/
- Too many other apps to cover but look on http://www.communica.com.au/mashup-australia/favourite-mashups/ or http://www.premier.vic.gov.au/app-my-state/view-application-submissions.html?view=category&layout=category While only 2 of the entries in Mashups Australia were mobile, about half in Victoria were. The iphone developers really jumped on the bandwagon. They not only produced great apps, they had videos on youtube and slick websites to promote them. Dunny Directories uses the national public toilet dataset. The location of public toilets was one of the two most asked for government datasets in the UK Vicwaterapp A quiz on water usage Water storage levels The microphone listens out for when you turn the water on and off in the show to time it (and tell you how much water you used and how much it costs) Mymarkets Tells you where the markets are, opening times, what food is available, lets you rate markets and individual stall holders and it is a social tool so you can communicate with other market goers in real time. Uses markets in victoria data from the department of planning. Ideas sections The Victorian and NSW competitions added an ideas section for those who have a vision but don't have the time, skills or data to create an actual app. Many of the ideas related to public transport, mobile phone apps that allow you to find out how to get from A to B. They could be hugely beneficial to irregular users of public transport or visitors. People wanted to know where things were in relation to other things – such as restaurants, BBQ areas, museums or public toilets along bicycle routes.
- There weren’t many entries from traditional GIS people in the competitions. I think they passed us by, which is why I am so keen to explain how relevant they are. But there are plenty of mashups being done by GIS people within their organisations. This example was presented by Geomatic Technologies at our GITA Mashup workshop. It mashes property boundaries with before and after aerial photography. The benefits are clear from the newspaper article.
- Energy Australia have also embraced mashups. Daniel Hansen and Craig Hersant gave a paper at the GITA conference, explaining how mashups have raised the profile of GIS in their organisations. Dan also gave a more detailed explanation in the GITA workshop of how he and a young web developer have mashed up the EA network, customer information and operational data with Dial Before you Dig requests and google maps for internal management briefings and media briefings.
- Energy Australia have raised the profile of GIS in that organisation immensely through mashups for senior management and media briefings.
- The website on which the mashup entries were posted was very user friendly. You could search by category of mashup, comment on a mashup or vote for it, report a bug, see what datasets it used, who created it. Very good feedback for the developers LobbyLens LobbyLens correlates data about Federal Government business over the last 18 months. It shows the connections between government contracts, business details, politician responsiblities, lobbyists, clients of lobbyists and the location of these entities. Very much in the spirit of Government 2.0 transparency
- Data is categorised and documented – mostly very simple from a spatial viewpoint, may be addresses or lat,long. Metadata variable, SA frog atlas has a very good explanation in the readme sheet in the spreadsheet, then all the locations of frog sightings, whether it was seen or heard or captured in a trap. The site is run by Australian Government Information Management Office AGIMO, drawing on experience of a UK model called showusabetterway. What sorts of data? All sorts. from Sustainability Victoria Adopt a roadside . Locations where individuals, organisations, community groups and businesses help maintain sections of roadside within Victoria’s arterial road network in regional Victoria. Dataset includes hours spent in maintaining roadsides and cubic metres of litter collected. Locations for Byteback ™ – a free computer recycling service that lets Victorian residents and small businesses dispose of old, unwanted and non-functioning computers safely. This service is currently being trialled in Victoria only. New South Wales Crimes by offence type, month and Local Government Area. Road crash statistics from SA Suburb of residence of school students in ACT and BBQ locations Rainfall, evaporation, temperature, pressure and wind data of all the capital cities plus Cairns, Broome and Alice Springs for 2008 from the BOM A KML file of Victorian Hospital locations Locations of playgrounds, public BBQs, fire brigades, boat ramps Most visited datasets on the site were centrelink offices, and medicare offices national public toilet map, federal election boundaries, frog atlas and crime data. One I really liked was public internet locations in Victoria – with addresses, opening times, free or fee, support for disabled access (speech synthesis, wheelchair access, vision impaired).
- Two types of data Raw data – 92 datasets mostly csv files, some KML (the Vicmap lite files for features and railway stations). Public internet locations, arts organisations, microbreweries, Melbourne Water use by postcode , where you can dispose of unwanted chemicals or computers. A whole host of useful information. Department of health published hospitals. The website, created by the Dept of Innovation, industry and Regional development is in beta form. I am not sure where it will go from here. DSE published a KML version of its Vicmap Lite products for the competition under creative commons licenses. They were downloaded over 600 times. I am not sure how many mashups used them but Transportle did. Mostly under creative commons, Metlink data more restrictive. But all just for mashup competition. You can still download the data but no updatesDIIRD want developers to give feedback. Really not very much at all on the discussion forums. Data Tools e.g. Heritage Database. This is a fully searchable (no doubt huge) database containing information about all Victorian Heritage Places and Precincts, including statements of significance, physical descriptions, builder, architectural style, photographs and heritage overlay number. Developers can either do a full download of core data in a text file or access more detailed (and always current) information about an individual place in JSON format through a live API. It is well explained and the interface is simple. There is a blog, a developers forum a “suggest a dataset” option, which plenty of people did use.
- National mashup comp tapped into OpenAustralia.org and their forum. Seemed to work very well. Volunteers hugely enthusiastic about transparent government. Really picked up the ball and ran with it. The developers forum on the victorian site wasn’t very widely used but there were some good discussions and you can still participate.
- Interviews with Adam Kennedy and Daniela Fernandez http://www.communica.com.au/mashup-australia/interview-with-a-mashup-hacker-2/ http://www.communica.com.au/mashup-australia/interview-with-a-mashup-hacker-1/ At the 32-hour long Canberra GovHack event, we (the MyReps team, Jeffery and myself) noticed that most of the teams present ended up spending between one and two thirds of their time dealing with locating, downloading, cleaning, formatting and otherwise overcoming problems getting access to the data they wanted to use in their application. Most of the applications that were ultimately produced were built largely in the last 10 hours of the event. If the data was immediately accessible in well known and easily usable formats, we might well have doubled the productivity of the event. Daniella Fernandez of Suburbanmatchmaker would have liked crime statistics in a standard format from all states – or better still via a web service so they didn’t have the responsibility of keeping them up to date.
- Choice magazine survey
- Live train map of the London underground. This mashup was created in a few hours at a UK science hack day. It uses a London Transport live feed of the location of London’s tube and was reported in the Sunday observer. While this app only made 10 requests every 2 minutes, others must have built their own mashups resulting in the London transport feed being overwhelmed by requests for data. It ballooned from 180,000 to 10million and the feed ground to a halt. Go to mashup favourites on communica.com.au to follow the links to the video and feedback discussions
- In the comments to the blogpost, there are some useful suggestions for about how to improve the service while easing the strain on its (well, the LDS's) servers: more partitioning of feeds with less data per feed, and more caching. These are obvious to developers - not so obvious to an organisation which has lived its life functioning, as one developer described it to me, as "a black box that people pour money into and which then spits out travel".
- 7 weeks later
- We can and should apply a lot of the lessons learned from general data portals to spatial portals. A portal may seem easy to use to the specialist but get non specialists to test them, they will type in search words you haven’t thought of, they won’t read instructions and they will be impatient if they don’ t find the information they want in a few keystrokes. If you are asking them to publish their own data, it has to be easy, give Test in different browsers. But even if you test it to death, expect feedback and criticism. Welcome it – it means people are interested. We must remember as spatial people that a portal is just that – a door, not all things to all men – direct people through to specialist data sites such as geoscience australia or statistical data. Automate the populating of the fields in the portal such as date updated e.g. example of communications towers on data.gov.au. Make sure there is ongoing resourcing for portals, so that you can respond to feedback and chase agencies who are slow to publish data. Talking of broken links, I would like to give a gong for the web master of the Victorian appmystate website. I have a collection of favourite mashups on my website, I posted images of the mashups and linked these to the mashups on appmystate. I then found to my dismay one day (while trawling my website to check it was all working) that the links were broken. It turned out they had added a /yourvoice in all the urls related to appmystate not realising it woudl affect everyone who was linked to them. I sent them a feedback form that evening by mid morning the next day it was fixed and I got a very nice reply. Support for crowdsourcing is an obvious extension. But don’t try or expect to do everything at once. The more functionality there is the more potential there is to confuse your users.
- Put a human face on the portal. They are rather faceless – I found out the ladies behind the NSW portal quite by chance at an ABS seminar. Start a blogs – and maintain it. This is Craig Thomlers Blog Online Communications Director at Department of Health and Ageing
- Job satisfaction – knowing who is using your data, what for and what are the community or economic benefits Helps you plan and justify spending on maintenance and additional data capture You can notify users of updates and changes, new datasets like amazon “people who downloaded this dataset also downloaded...” You have all these poeple ‘out there’ who can spot errors and tell you about them. Expect questions and criticism. Connected with the public and developer community. Extra work – yes, but by putting data on a website, you will reduce individual requests. NSW legislation the first. Others will follow . In US each government department has to share 3 datasets. Need to have this new responsibility acknowledged in job descriptions. Do your own mashups. As custodians you are not just putting in, you can trawl the portals for data that is useful to you. Communications between government departments are not perfect now, so it is highly likely there is data in other departments that will be hugely beneficial to your department – if only you could find it and get access. Less red tape.
- The data hounds are coming for your data. They will be persistent. Be ready to give them something. It doesn’t have to be perfect. How is this for serendipity? I googled “the hounds are coming” and found this image of the Sinnington Hunt The hounds are coming away from the meet at Wether Cote farm. The cars on the right belong to hunt followers. Not particularly special you may think, except I was born and grew up in Sinnington, a tiny village of in North Yorkshire whose hunt had more members than the entire population of the village.
- Spatial portals that are inviting, slick, easy to find your way around In addition to the obvious, robust, slick, upto date, consistent, easy to find your way around Lively discussion like the openaustralia google group Well resourced. Not just in terms of people but efficient use of skills and tools. New area, training, collaboration. Some duplication inevitable, best ideas float to the top but make sure we talk to each other. Catalogue of public and private webservices and tools – such as geo2gov or the Numaps demographic drapes that won a prize in the NSW mashup competition. Brad Spencer of Numaps is speaking at this conference. How to get started. If I want to develop an iphone app what do I need to do, what do I need to know or where do I find the skilled people? What are the pitfalls. At the GITA conference we had a mapping mashups workshop. Energy Australia and Geomatic Technologies showed what you can do and talked about how to do it but we didn’t have time to get our hands dirty. SSSI have discussed it. It would be good to collaborate with other associations such as the Australian Computer Society ACS – there was an openstreetmap crowdsourcing seminar in Woolongong. Early adopters and very technical just pile in and do it – don’ t mind spending hours figuring something out, but majority need a bit more help. Also the technical person is not necessarily the right person to create the user interface.
- This is the alpha version and they are doing lots more work on it and have set aside funds for the user interface and usability testing.
- Spoke to Peter Alexander, assistant secretary at AGIMO responsible for the portal First priority is just get the data out in some form – even a table in a pdf (same philosophy in NSW government) If you cannot find it, it doesn’t matter how good it is. Then improve the format – the platform will support flexibility 5-10 attributes for every dataset Link through to specialist portal for geo-spatial data (eg. Geoscience Australia) or statistical. Different metadata. Appreciate importance of consistency and resourcing blog etc. But will have a ‘lag response policy’. Don’t jump in straight away with answer, let others comment. Otherwise it kills the conversation. Peter has had some discussions with ANZLIC and states. The same technology platform can be used by states. Working with Information Managers in agencies to get data up on the site. Agencies will self publish. Short timescale, mid September had not taken final decision on platform, plan to be in production october/november. Have a Government 2.0 steering group
- The sunlight foundation is a not for profit organisation that is collecting data and useful tools from all levels of government and the private sector. Nice introductory video.
- Star ratings for datasets like amazon.com
- So in summary, mashups are very exciting and useful. You can’t talk about mashups without talking about data in the same breath. Different types of data will stimulate different parts of the economy. For example, releasing accessibility data removes barriers to work, leisure and tourism for disabled people. Releasing transport data, especially live feeds , improves travel experiences and increases passenger numbers (or encourages them to adopt more sustainable travel practices). But the benefits may be even more far-reaching than the obvious. Knowing how long till the next bus means you can pop into the shops while you wait. Or choose to walk if it means getting to your destination quicker. Or wait for the next, less crowded bus – particularly useful for people with reduced mobility. Closer to home Premier John Brumby announced that the 170 entries to the Victorian mashups comp had created $1 million worth of apps for Victorians for an investment of $100,000 in prize money (plus the cost of setting up the portal, running the hackfests, organising the promotion ) They have their place alongside serious GIS. Because they are familiar and non-threatening, they are embraced by managers, non specialist colleagues and the public because they help them to understand issues and better make better, quicker decisions. The lifeblood of mashups is data. You have to be able to find it and so the easier we make it for people to find our data and the more flexible formats we offer it in, the more it will be used and the more it will be of benefit and if there are errors in it, people will tell us about them so we can fix them. Join our mapping mashups linkedin group formed after the GITA mashup workshop. We will post about seminars, online training, share mashups we have done. Open Australia.org is a non-partisan website run by a group of volunteers which aims to make it easy for people to keep tabs on their representatives in Parliament. They really got behind the mashup australia competition and there was a very active discussion forum. In your own organisation: Find a bright eyed bushy tailed web developer