1. Assessing User
Satisfaction for the
Digimap Service:
Surveys, spread sheets and a different
kind of chart.

2. The 2011 Winter Survey
• Required as part of the JISC services portfolio review
• Assess user satisfaction as part of the review of all their
services
• Common questions across all JISC services
• In house design for interface and database
• Bristol Online Surveys used in the past
• Detailed review for Geo-Services in 2011-12
• Survey made more prominent in a pop-up
• Users made to wait 10 seconds for the pop-up to disappear
• Results were made publicly available online:
http://edina.ac.uk/impact/

3. Considerations before starting a survey
Planning
Surveys are exercises in Public Engagement, be prepared!
• Don’t carry them out lightly, have a plan
• Survey are an inconvenience, weigh the costs with the benefits
• Plan carefully how you are going to carry out the survey
• Plan how you are going to give feedback
• Plan how you are going to handle complaints
Good and swift feedback can turn someone with a negative view
of your service into your best customer; if they have been part of
an improvement they will often help promote your services.
“Tell me and I'll forget; show me and I may remember;
involve me and I'll understand.” Chinese Proverb

4. Considerations before starting a survey (2)
Timing
Survey fatigue can dent your response rate and
increase the amount of overly negative responses you
get through misrepresentation:
• When possible, avoid times that you know your customers/users
are having to fill in lots of other surveys.
• Don’t survey too often / balance with length.
• Find out the best time of year to survey your users.
Length
Keep it as short as possible, and use yes/no answers
and options as much as you can to minimise the time
users need to fill it in.

5. Considerations before starting a survey (3)
Misrepresentation
This can be further mitigated by the amount of information you
give respondents:
• Tell them why they are being surveyed
• Tell them how there responses will be used
• If possible, show how previous results have affected the service
• Feedback the results of the survey
• Feedback how the results have changed the service
The more the respondents understand why they are doing the
survey and what impact it is going to have, the more useful the
responses they are likely to give.

6. The Questions – Who are the users?
1. Where do you study/work?
2. In which role are you using Digimap?
Student, Postgraduate, Lecturer, Researcher, Information professional
3. Please indicate your subject area:
Agriculture, food, and forestry Law
Architecture and planning Mathematics and computer science
Biological sciences Medicine including dentistry
Business and management studies Modern languages and area studies
Communication and media studies Nursing, midwifery and allied health
Creative and performing arts Physical sciences
Education and research methods Psychology
Engineering Social sciences
Geography and environment Veterinary medicine
Humanities
Automatically answered for registered users

7. The Questions – How do you find the service?
Q1. On average, how often do you use this service?
– Several times a week, Once a week, Once or twice a month, Less often, This is my first use
Q2. How do you find using Digimap?
– Very easy, Easy, Neither easy nor difficult, Difficult, Very difficult
Q3. My work would take longer if Digimap were not available.
– Strongly agree, Agree, Neither agree nor disagree, Disagree, Strongly disagree
Q3b. [optional] Why do you say that?

8. The Questions – What else is there?
Q4. Would you use an alternative if Digimap was not available?
– Yes, No, Don’t Know
Q4b. If yes, what would you use?
Q5. I would recommend Digimap to others.
– Strongly agree, Agree, Neither agree nor disagree, Disagree, Strongly disagree
Q5b. [optional] Please tell us why you would, or would not,
recommend this service:

9. The Questions – What do you use Digimap for?
Q6a. What do you use Digimap for? Tick all that apply to you:
– For Research
– For teaching the use of maps, spatial data or GIS
– For Post Graduate Thesis
– For teaching materials in a general subject
– For Undergraduate Thesis
– For supporting students use of maps, spatial data or GIS
– For Coursework
– For personal interest
Q6b. [optional] We’d really like to know more about what you are
doing with Digimap, please give as much detail about what you
use the service for below:
Can we contact you about using your example as a case study?

10. The Questions – What do you use the most?
Q7a. Rate the following elements of the service in terms of how
important they are to your studies, research or work:
Essential, Important, Useful, Little value, I don’t use it
Viewing Online Maps
Downloading Data
Accessing Streamed Maps
Gazetteer and Postcode Query tools
Support Materials
Q7b. [optional] We’d really like to know how important the service
is to you; please share any stories you may have about how
Digimap has helped you:

11. The Questions – What do you want in the future?
Q8a. What would you like Digimap to provide next? Rate the
following options in terms of how useful they would be to your
studies, research or work:
Essential, Important, Useful, Little value, I wouldn’t use it
Mobile Maps
More Data Products
More Data Formats
Advanced Online GIS Tools
More printing options
More mapping support
More GIS support
Q8b. [optional] Please give us any details about the choices you
would use from the table above such as which products,
formats, options or support you would like to see:

12. The Results
• 8537 responses received
• The published results are based on the 7000+ responses from
returning users (omitting those who indicated this was their
first time):
• OS = 4969
• Historic = 1462
• Geology = 776
• Marine = 117
• Results in full: http://edina.ac.uk/impact/
• Thank you to anyone who filled one in!

13. Doughnut Charts
Digimap User Disciplines 2010-11 & 2011-12
Agriculture, food, and forestry
Architecture and planning
Biological sciences
Business and management studies
2011 - 12
Communication and media studies
Creative and performing arts
Education and research methods
Engineering
Geography and environment
2010 - 11 Humanities
Law
Mathematics and computer science
Medicine including dentistry
Modern languages and area studies
Nursing, midwifery and allied health
Physical sciences
Psychology
Social sciences
Veterinary medicine
Academic Support / Other
Mmm... Donuts!

14. Diverse Users
Agriculture, food, and forestry
OS Architecture and planning Historic
Biological sciences
Business and management studies Architecture and planning
8% Communication and media studies 12%
Creative and performing arts Geography and environment
Education and research methods
Engineering Humanities
Geography and environment
Humanities
19% Law 22%
Mathematics and computer science
53% Medicine including dentistry 47%
Modern languages and area studies
Nursing, midwifery and allied health
Physical sciences
Psychology
Social sciences
Veterinary medicine
Academic Support
English Lit, History, Fine Art, Equine Studies, Medicine
Geology Architecture and planning
Marine
Architecture and planning
Engineering 14% Biological sciences
20%
Geography and environment
30% Engineering
Physical sciences
41% Geography and environment
19%
Physical sciences
15%
17% 19%
13%

15. Easy To Use?
OS Historic
Very Easy 1% Very Easy
8% 1% 16% 5%
16%
Easy Easy
Neither Neither
31%
30% Difficult Difficult
Very Difficult Very Difficult
46% 47%
Geology YES Marine
1%
4% Very Easy 1% 9% Very Easy
19% 14%
Easy Easy
26%
Neither Neither
Difficult Difficult
Very Difficult 32% 44% Very Difficult
50%

16. Saves Effort?
OS Historic
0% Agree Strongly 0% Agree Strongly
1% 1%
8% Agree 8% Agree
Neither Neither
Disagree Disagree
49%
52% Disagree Strongly Disagree Strongly
38% 42%
Geology YES! Marine
1% 0%
1% Agree Strongly 2% Agree Strongly
10% 14%
Agree Agree
43% Neither 40% Neither
Disagree Disagree
Disagree Strongly Disagree Strongly
45%
44%

17. Would You Recommend Digimap?
OS Historic
1% Agree Strongly 0% Agree Strongly
1%
1%
8% Agree 7% Agree
Neither Neither
45% Disagree 47% Disagree
Disagree Strongly Disagree Strongly
45% 45%
Geology YES! Marine
1% 1%
1% Agree Strongly 2% Agree Strongly
9% 7%
Agree Agree
36% Neither
42% Neither
Disagree
Disagree
Disagree Strongly
Disagree Strongly
47%
54%

18. Difficulties Users Experienced
• About 62% of respondents found the service easy or very
easy to use… but what about the others?
• These were the main areas of concern:
• Waiting for maps
• Printing
• Complex interfaces
• Registration

19. The Solutions
Digimap is Slow?
• Problem = Bandwidth
• Solution = Shrink JavaScript & Compress Map Images
Printing?
• Problem = Previews & Legends
• Solution = A new interface
Complex Interfaces?
• Problem = MasterMap Download & Carto
• Solution = New Data Download, Roam+ & USeD
Registration?
• Problem = Complicated & the wait
• Solution = The next Digimap licence

20. Priorities for the Future
OS Historic Geology Marine
1. Printing 1. Printing 1. Printing 1. GIS Tools
2. Mapping Support 2. Mapping Support 2. Mapping Support 2. GIS Support
3. Data Formats 3. Data Products 3. GIS Support 3. Data Formats
Over 300 people commented in this section

21. Priorities for the Future (2)
• The Print interface needs to be enhanced.
• New functionality: Legends, 2up maps & more formats
• We need to offer downloads in many more formats.
• Pre-generate data & on the fly transformations
• Mapping and GIS Support
• The new home page resource centre
• New Datasets
• We are continually looking to add to Digimap
• Environment Digimap

22. Survey Results Summary
• The service used by an incredibly broad range of people
from across academia.
• The service is well liked and very well used:
...but...
• There is more that can be done to make the service:
• Faster
• Easier to use
• Meet all it’s users requirements
• We continually need feedback to make sure Digimap
continues to meet its users needs.

23. Lessons Learned from the survey
JISC want us to carry out the survey every year.
Changes we need to make:
• No survey at all for first time users
• Easier to ignore
• Tighten up the wording of certain questions
• Timing of the appearance of the pop-up
• More thorough testing
Important considerations before making changes:
• Surveys need to be comparable to previous years (especially
questions 1-6)
• We need a good amount of responses

24. Beyond the Annual Survey
Continual Feedback Via:
• Helpdesk
• No news is good news!
• “Comment on this service” links
• A mechanism for people to let us know about something while they
are doing it
• Surprising number of glowing reviews from this
• Shorter, one question pop-ups to gauge opinion on certain
features etc. are also being considered.
• This has proved successful in the past for users waiting for downloads
• Social media
• Popular with certain groups of users
• Needs careful monitoring as it is all public

25. Beyond the Annual Survey (2)
More in-depth user engagement during service development
• Usage statistics are being used to create personae
• Users who fit into the categories are asked to test services in
development
• Iterative testing and design phases are used to hone the user
experience
• USeD Project: http://used.blogs.edina.ac.uk/