1. www.vestforsk.no
- Size of Municipalities and Quality of Websites
Small is Beautiful or Bigger is Better?
- by Svein Ølnes, Western Norway Research Institute (Vestlandsforsking)
- ECEG 2013, Como, Italy
4. www.vestforsk.no
Main question answered
Does quality of municpality websites vary with size?
Which underlying factors can explain the observed variation?
Whose quality are we really measuring?
5. www.vestforsk.no
Benchmarking systems
Norwegian benchmarking system from 2001
all public websites
yearly evaluations (expert evaluations)
a single set of indicators
Denmark: Also since 2001
tried different methods (single set of ind., multiple sets, self eval.
gradually moved to more end user involvement
2012 benchmarking accessibility screening + user survey
6. www.vestforsk.no
Benchmarking systems
EU’s benchmarking system
Conceived at the Lisbon conference i 2000
Until 2012 based on a set of 20 services
• 12 services for citizens
• 8 services for businesses
As the “e-readiness” approached 100 % when measuring the digital
availability of the 20 services, new approaches had to be found
7. www.vestforsk.no
Method
1. Results from the Norwegian benchmarkings of public
websites (municpality part) 2007 – 2011
2. A survey to web administrators (of the same websites as
above)
3. A broad citizen survey conducted in 2009/2010
8. www.vestforsk.no
Norwegian benchmarking system 2007-2011
2007 2011
No. of ind. Max score No. of ind. Max score
1. Accessibility 11 27 11 28
2. Usability 14 37 12 30
3. Useful services 7 28 10 35
Total 32 92 32 93
9. www.vestforsk.no
Norwegian benchmarking system (2013)
1. Accessibility
13 indicators, max score 36 points
Example: Ind. 1.11 – Correct use of HTML
2. Usability
8 indicators, max score 28 points
Ind. 2.2 – The website is functional also with mobile devices
3. Useful services
9 indicators, max score 39 points
Example: Ind. 3.7 – The website has published open data
10. www.vestforsk.no
Results
2007 20011 Change
Small municipalities (< 5.000) 46,6 58,4 11,8
Medium sized mun. (5.000-20.000) 50,7 66,1 15,4
Large municipalities (> 20.000) 58,4 72,0 13,6
Average, all municipalities 49,5 62,7 13,2
11. www.vestforsk.no
What do the webmasters say?
What is your knowledge of the quality evaluation system?
Small Medium Large
No knowledge/little knowledge 36 % 15 % 3 %
Good/Very good knowledge 59 % 85 % 97 %
12. www.vestforsk.no
What do the webmasters say?
How easy or difficult is it to understand the quality indicators?
Small Medium Large
Difficult/Very difficult 37 % 34 % 13 %
Easy/Very easy 58 % 66 % 87 %
13. www.vestforsk.no
What do the people say about the quality
of public services?
A survey to 30.000 citizens above 18 years
Satisfaction with public services (12.659 asked, 6.646 answers)
Results:
No significant difference in perceived quality of municipality services in
relation to size of municipality
Planning and building services and elderly care got higher scores from
citizens of small municipalities
For services like kindergarten and primary school larger municipalities
scored better than smaller
Is it Jakob Nielsen’s first rule of usability?
“Don’t listen to the users, whatch them work!”
14. www.vestforsk.no
Whose quality are we measuring?
Better quality on public websites and digital public services
is the objective
But whose quality?
The end user (the citizen) is too far away and end user
quality is not enough taken into account
Benchmarking systems like the Norwegian need to be more
user oriented
Usability testing and user surveys should be key parts in
developing benchmarking systems