SPRINT 13 Workshop 5 Using Open Data and Open Source to show how european legislation has spread into UK legislation over time and how TNA approaches big data. Oliver Morley- The National Archives
2. Most-used museum, library or archive web-site in the UK
(Google Trends/ Alexa)
150m records downloaded
19m unique users for legislation
40m web-site redirects for whole govt
So, the three ways to get access. Or show me the way/ Get directions from someone who knows/ Or stop wasting my time and give me the whole map. Point one. Unlike preservation, good presentation merges the physical and digital seamlessly together, but makes it clear which is which. I want to be able to see all the documents relating to Edward VIII’s Abdication, but as not all of them are digitised it should be clear which is which. Note that our previous presentation system did not do this. Point two. These three routes are complementary to each other. Our intention is that they democratise access to our information, but not that they are exclusive.
Text, images, film, social media Scalability – up to 50 million records by 2015 High cost of maintenance and support Need to replace other differently structured legacy systems Current catalogue assumes knowledge of records structure New features required eg user participation
Options we considered Great community of users SQL solution – too complex XML database – not good at updating No SQL, big data solution MONGO DB offered best balance between reads and writes
Agile Team of up to 11 people First release in 11 months Moving to monthly releases Continuous integration and delivery
Graph shows SIs made under the European Communities Act, by type (Regulation, Rule or Order), by Dept, in terms of the size of legislation (using the page count, not the number of SIs) Domestic SIs, by type (Regulation, Order, Rule) Points to note Huge number of temporary Orders made by DfT Orders in Council made shown as Privy Council (so exercise of Royal Perogative powers)
Points to note Substantial portion of new regulation does come from Europe Impact of devolution to Wales (the Welsh Government bubbles – including a substantial amount of the new EU legislation) Defra and BIS have been the large implementors of new EU driven regulations