A whitepaper examining the application of Big Data approaches within the UK government and identifying gaps in order to make recommendations for the future.

1. Big Data 2.0 ­ How the UK Government Can Optimise Its Use of Big Data
This whitepaper examines the application of Big Data approaches within the UK government and
identifies gaps in order to make recommendations for the future.
Big Data Innovations To Date
The UK government recognises Big Data as one of the government’s “Eight Great Technologies” ­
areas of focus and investment in the UK tech economy ­ with £198m in funds issued to support 1
Big Data development since 2013. This has lead to the creation of many new data resources
including Data.gov.uk, an open database of any government expenditure of £25,000 or more, as
well as the initiative “A Right To Data” which opened access to previously guarded data sets, such
as maps and postcodes ­ formerly sold at profit ­ in order to boost innovation , free of charge. 2
Rather than embrace this national unified data set approach, there are many competing Big Data
projects at local government level such as Open Data Bristol, Cambridgeshire Insight, the Greater
London Authority London DataStore, Open Data Institute Nodes network, and the Trafford
Innovation and Intelligence Lab . 3
Defining The Problem: Unrealised Potential 4
1. Whilst on the surface these Big Data
initiatives are an encouraging sign, Big Data in
government has not yet realised its potential to
increase operational efficiency and targeting of
service delivery , lagging behind the private 5
sector in terms of implementation.
Utilised to its full extent, Big Data should be a
revolutionary and disruptive force, increasing
governmental efficiency, unlocking new avenues
in life­saving research and creating huge
opportunities for innovation. According to the UK
free market think tank, Policy Exchange, the
government ​could​ save up to £33bn a year by
using public Big Data more effectively, and Digital
Catapults estimates £200bn being unlocked
through sharing of data . 6
1
​https://www.gov.uk/government/publications/eight­great­technologies­infographics
2
​http://www.policyexchange.org.uk/images/publications/a%20right%20to%20data%20­%20mar%2012.pdf
3
​http://www.v3.co.uk/v3­uk/news/2461469/council­leaders­urged­to­embrace­big­data­despite­extreme­skills­shortage
4
https://gcn.com/microsites/2012/download­big­data/~/media/GIG/GIG%20Custom/2012%20Custom%20Reports/Q4Images/Big%20Da
ta%20Download%20charts/1b_chart.ashx
5
​http://www.publications.parliament.uk/pa/cm201516/cmselect/cmsctech/468/468.pdf
6
​https://digital.catapult.org.uk/about/

2.
2. Despite the government’s funding efforts to
elevate the importance of Big Data, a culture
change has not been effected. ​91% of
executives and council leaders say they are
confident of their capabilities in data analytics,
yet a much lower proportion are actually using
data analytics to inform decision making and
strategy .​ There is an absence of a central 7
owner for nationwide IT infrastructure, with a
clear focus and objectives to create a
seamless experience for the end user, be it a
government worker, a business user, an
average citizen, or another intergovernmental
office ​. ​Big Data advocates even note that 8
“Councils do not publish the data people want”
and “systems used in local government are
not very good at making data available
externally .” 9
3.This is symptomatic of an entrenched challenge with Big Data as the government itself notes
“Data is increasingly being produced at a rate that means that current techniques are insufficient
to fully exploit it .” 10
Data by Default ­ Our Strategy and Tactics To Optimise Big Data Use
Our recommendations to optimise Big Data usage centre around three key pillars.
1. Create a Digital Steering Committee
Creating an overseeing Digital Steering Committee including a Department of Data and dedicated
Minister with a brief to unify Big Data initiatives at a National and Local level is crucial to driving
the strategy forward. Local government also acknowledges the importance of insight being driven
by clear leadership . The existing Digital Advisory Board primarily consists of members from 11
various sectors, but formal meetings are limited to twice a year . 12
2. Address the Data Skills Shortage
A severe shortage of analytical and information management skills is preventing government from
deriving benefits from key data to improve services. Skilled workers are so rare that they have
been likened to Unicorns . Our recommendation is to stimulate the growth of analytical and 13
7
​http://www.computerweekly.com/news/450297218/Councils­must­bridge­digital­gap­to­remain­relevant
8
​http://www.mckinsey.com/business­functions/business­technology/our­insights/public­sector­digitization­the­trillion­dollar­challenge
9
​http://www.computerweekly.com/feature/Using­open­data­in­local­government
10
​http://www.publications.parliament.uk/pa/cm201516/cmselect/cmsctech/468/46805.htm#_idTextAnchor012
11
​http://www.local.gov.uk/home/­/journal_content/56/10180/3511511/ARTICLE
12
​https://gds.blog.gov.uk/2012/04/25/introducing­the­digital­advisory­board/
13
​http://identity.governmentcomputing.com/news/socitm­big­data­potential­hampered­by­extreme­skill­shortage­4921218

3. information management skills by funding training at degree level, offering reduced fees and loan
terms for students opting to study data science. Adding data science modules to non­data related
courses could also facilitate the rise of Citizen Data Scientists ­ people whose specialism is
outside of the field of data but whose value is understanding the benefits of its application . 14
3. Standardisation of Data Infrastructure
Local government officials have already highlighted the benefits of a collaborative approach to Big
Data sharing . Streamlining the access to data that government departments hold and removing 15
barriers to access will further harness Big Data capabilities and maximise its potential. At the
forefront of this innovation will be redesigning processes, with the end user in mind . Matt 16
Hancock, Minister of Cabinet Office tasked with digital transformation, highlights that “Data is the
fuel for the digital revolution. The very best policies and services are developed around
information that’s current [and] relevant... .” At the centre of this will sit a data sharing strategy to 17
overcome ‘silo’ mentality within government departments. A lucrative example is the ‘Nowcasting’
technique which uses information from social media sentiment in combination with government
data to make more accurate judgments and future predictions . 18
Risks and Challenges of Our Strategy and Tactics
Government­led projects historically run over budget ­ and technology infrastructure quickly
becomes outmoded. A joint study by McKinsey and Oxford University found that public sector IT
projects requiring business change were six times more likely to experience cost overruns and
20% more likely to run over schedule than such projects in the private sector . 19
Most of the technologies capable of transforming day­to­day life have been around for over twenty
years, sometimes longer. The reason they have not been applied is a matter of skills: particularly
across our fragmented and siloed public sector planning and funding hierarchies.
Developing shared services will therefore require standardisation, process re­engineering and
consolidation to deliver effective services, and it will be important for the government to provide
local councils with a framework of incentives to share information (de­anonymised), as well as
indications of funds saved through this new technology.
The most suitable way to store this amount of data is via cloud computing, which comes with its
own risks. According to a report released by MIT, cloud security has two main threats, firstly that
the data may be lost due to equipment or software failure, or stolen. The second threat is that the
data may be mishandled by the cloud provider due to technology gaps, and the information would
also be able to be extracted from the cloud through a court issued summons . 20
14
​http://www.informationweek.com/big­data/big­data­analytics/citizen­data­scientists­7­ways­to­harness­talent/d/d­id/1321389
15
http://www.local.gov.uk/documents/10180/11553/Transforming+public+services+using+technology+and+digital+approaches/ab9af2bd­
9b68­4473­ac17­bbddf2adec05
16
​https://gds.blog.gov.uk/2015/08/18/mapping­new­ideas­for­the­digital­justice­system­2/
17
​https://www.gov.uk/government/news/launch­of­new­data­sharing­consultation
18
​https://www.theguardian.com/technology/2014/jun/14/big­data­google­economics­twitter
19
​http://www.mckinsey.com/business­functions/business­technology/our­insights/public­sector­digitization­the­trillion­dollar­challenge
20
​ ​https://www.technologyreview.com/s/416804/security­in­the­ether/

4.
We recommend that the government push ahead and begin to centralise all governmental data
with encrypted and permissions­based access. Using high­level cryptography and possibly using
aspects of Blockchain ­ recently created to encrypt Bitcoin to verify access and permissions ­ will
address security and data access issues . 21
Regulatory Implications of Big Data
Our recomendations face specific regulatory challenges.
Applying any changes regarding the use of Big Data in government
are potentially hamstrung by the pending implementation of EU Data
Protection Regulation, which supersedes the UK’s own Data
Protection Act. The law has already been passed, and will be rolled
out in the next two years. The government now faces a challenge to update its own Data
Protection Act in the interim period between the European law being implemented, which could
slow down or even bring a stop to our recommendations for Big Data 2.0.
The EU Data Protection Regulation enforces stricter rules regarding the requirement of consent
for data use, and has been flagged as a potential roadblock for the Big Data industry in the UK by
the Science and Technology Committee, especially in the field of medical research, which could
limit the scope and impact of our Big Data 2.0 strategy . 22
Additionally, an overarching challenge that applies to any project in government is the change in
personnel who make up the various branches. This has been recognised as a significant
challenge to data related projects in particular, with senior staff turnover doubling between
2015­16 , leading to delays on flagship projects such as Universal Credit. 23
Conclusion
The government’s early adoption of Big Data is commendable, but without a concerted effort to
address structural and cultural challenges, the unprecedented possibilities that Big Data can offer
government and its citizens will not be realised. Moreover, an opportunity to place Big Data at the
heart of efficient decision­making will be lost. The appropriate use of Big Data 2.0 must meet
these challenges and more to future­proof government against an increasingly digital and
data­driven world.
21
​https://datafloq.com/read/7­big­data­trends­for­2016/1699
22
​http://www.publications.parliament.uk/pa/cm201415/cmselect/cmsctech/245/245.pdf
23
​http://www.computerweekly.com/news/4500271427/Digital­methods­will­improve­government­projects­says­civil­service­chief