Overview of Digital Archiving at Historic Environment Scotland

1. ARA Summer Seminar – Edinburgh 7 June 2019
Digital Archiving at Historic Environment Scotland
Stuart Macdonald | Digital Archivist
Archives & Engagement | Heritage Directorate
Historic Environment Scotland | Àrainneachd Eachdraidheil Alba
stuart.macdonald@hes.scot

2. ARA Summer Seminar – Edinburgh 7 June 2019
Historic Environment Scotland (HES)
HES core functions
Archival collections
Canmore
Digital archiving
Processes
Future developments
Questions

3. ARA Summer Seminar – Edinburgh 7 June 2019
Historic Environment Scotland (HES)
HES is the lead public body established to investigate, care for and promote Scotland’s
historic environment.
A non-departmental public body with charitable status, governed by a Board of Trustees
appointed by Scottish Ministers.
• 300+ properties and sites in care
• 5 million visitors to our staffed sites in 2018
• 5 million+ drawings, photographs, negatives and manuscripts relating to Scotland’s historic environment
• tens of millions of aerial photographs of historic events and locations worldwide
• 200,000 members
• £41.6 million commercial income per year
• 1,280 staff across Scotland

4. ARA Summer Seminar – Edinburgh 7 June 2019
Core functions include:
• Heritage management: maintain listings of world heritage sites, scheduled monuments, listed
buildings, gardens and designed landscapes, battlefields.
• Advice and guidance: on the impact of planning and development on Scotland’s historic environment
(on land and marine).
• Climate change: exploring affects on the historic environment and how to limit impact.
• Conservation: maintain over 300 monuments and buildings in our care.
• Research: understand, protect, value, impact and track changes in our historic environment.
• Historic Environment Scotland is an Independent Research Organisation with AHRC & part of the Scottish
Cultural Heritage Consortium
• Survey and recording: building and industrial surveys, archaeology field surveys, aerial surveys,
maritime surveys – data management.

5. ARA Summer Seminar – Edinburgh 7 June 2019
Archival collections include:
• National Collection of Aerial Photography (NCAP) - one of the world’s largest collections of historic
aerial photography.
• Scran - an online public learning service containing images and media from museums, galleries, and
archives.
• Scotland’s Urban Past - database of historical places, buildings and artifacts celebrating Scotland's
urban environments and how these have changed over time.
• Britain from Above – historic aerial images of Britain taken from 1919 to 1953.
• PastMap - interactive online map to locate Scotland’s archaeological and historic sites.
• Canmore - the online catalogue of the National Record of the Historic Environment.

6. ARA Summer Seminar – Edinburgh 7 June 2019
Canmore - https://canmore.org.uk/
Contains more than 330,000 records and 1.4 million catalogue entries for archaeological sites,
buildings, industry and maritime heritage across Scotland including physical and digital assets.
Contains information and collections from HES survey and recording work, as well as from a wide range
of other organisations, communities and individuals who are helping to enhance the national record.
• Sites - location of individual sites, buildings or wrecks.
• Events – excavation, walking survey, field visit, LIDAR survey, pollen analysis, side beam sonar
survey.
• Catalogue of physical archival material - historic photographs, drawings, manuscripts, models.
• Catalogue of digital archival material (“data”) – digital photographs, GIS data, CAD drawings,
photogrammetry.

7. ARA Summer Seminar – Edinburgh 7 June 2019
Digital Archiving at HES
Data assets received from a variety of sources including:
• with physical accessions
• architect firms
• archaeological units
• private donations
• field surveys
• digitisation efforts
• Funded research projects
Floppy disk, CD, secure file share, email, ftp, external hard drive, pen drive.
Digital files include: digital photographs images, documents, CAD drawings, geophysical data, raw
text, spreadsheets, databases, audio and video.
In a range of accepted proprietary and standard formats predefined by the digital archive.

8. ARA Summer Seminar – Edinburgh 7 June 2019
Digital Archive processes
Quarantine:
Data are transferred to non-networked quarantine PCs from original media using Robocopy.
Data are then virus-checked and transferred to a Digital Archive external hard-drive for 30 days.
After 30 days the data on the Digital Archive hard-drive are then virus-checked again.
When two virus scans are passed data are safe to copy onto a networked PC for ingest into an
Accessions Database.
For data received on large floppy discs digital archivists use a tool called Kryoflux to take an
image of the disk and digital content. Decisions then need to be made about file migration if
formats are obsolete.

9. ARA Summer Seminar – Edinburgh 7 June 2019
Accession:
Virus-free data are given an accessions number and ingested from a pre-defined folder into a
holding Accessions Database via a process in the Oracle Sites and Archive System (SAS).
Digital Archive staff are alerted when this is complete.
All files are given individual MD5 checksums for integrity checking & fixity.
Data can then be uploaded from a network folder for cataloguing using the Oracle SAS.
Currently a backlog of 571,435 files (2.4TB) being addressed as part of a 4-year Digitisation
project.

10. ARA Summer Seminar – Edinburgh 7 June 2019
Catalogue:
Incoming data can come in many digital file formats.
Before data are catalogued manual checks are done on files to organise the data.
Tools including DROID, FME, QGIS are used to identify and validate new file formats and their
characteristics.
Data are then ingested and catalogued according to ISAD-G archival metadata standard and
linked to relevant site, event and physical archival information.

11. ARA Summer Seminar – Edinburgh 7 June 2019
Control over whether metadata or metadata and data are published are influenced by
issues such as copyright, privacy etc.
Minimum Standard Records: On-going work to standardise site, event, archive (physical
and digital) metadata capture.
Once catalogued Oracle SAS run scheduled overnight jobs to transfer data to external
Digital Archive Storage.
Data e.g. digital image becomes visible on Canmore.
IT administer storage and security processes such as disc-space, back-up schedules,
disaster-recovery plans in line with operational policies.

12. ARA Summer Seminar – Edinburgh 7 June 2019

13. ARA Summer Seminar – Edinburgh 7 June 2019
Searching Canmore

14. ARA Summer Seminar – Edinburgh 7 June 2019

15. ARA Summer Seminar – Edinburgh 7 June 2019
Digital Archive future development
1. 4-year Digitisation Project (2017-2020) funded by HES Investment Plan (£250,000 p.a.)
Pilot year:
• 175,000 digital images catalogued .
Years 2-4:
• creation of 125,000 new digital images from physical material.
• catalogue 125,000 items from Digital Archive backlog.
• c. 1 million digital images accessible on Canmore.
2. HES Digital Archive applying for Core Trust Seal Trusted Digital Repository Accreditation – October
2019.
• 16 requirements express the core characteristics of a trustworthy data repository or archive
• CTS offers core accreditation and is a solid foundation for higher-level certification including nestor-Seal DIN 31644 (34
metrics) and the formal level TDR certification ISO 16363 (109 metrics).

16. ARA Summer Seminar – Edinburgh 7 June 2019
• The application process will clarify and articulate HES’s digital archivalpractices.
• Help determine strengths/weaknesses.
• Help to identify service gaps.
• Promote trust and confidence between the three stakeholders in the data supply chain.
• ensure all are working to a common set of standards or principles.
• Easier to conduct systematic review of technical / human processes in future.
• As/when new tools, technologies, standards emerge the archive will be better equipped to respond to
changes in data stewardship workflows.
• Raise the profile of the archive and digital preservation with HES senior managers.
• Highlights areas of interworking between archival colleagues for purposes of streamlining operations.
• Improves national and international recognition and reputation.
2. cont’d CTS - organisational and community benefits

17. ARA Summer Seminar – Edinburgh 7 June 2019
3. Digital Preservation & data migration
HES are committed to using an enterprise instance of Preservica as the preservation platform for the
Digital Archive.
Testing on integrating with Oracle on existing infrastructure done.
Digital Archive data and servers moving to state-of-the-art secure data facility – Data Vita (June 2019).
Work to commence testing and integrating Preservica into new technical environment (end summer
2019).

18. ARA Summer Seminar – Edinburgh 7 June 2019
4. Policies & procedures
• Work on-going to craft digital archive policies e.g. Collections and appraisal, Mission
statement, and to update existing policies e.g. Preservation Policy.
• Work on-going to formalise operational procedures through guidance and procedural
documents.
• Public-facing documents to support TDR best-practice.
• Re-evaluate and define Digital Archive within HES as core service e.g. using consistency of
vocabulary/terminology plus outreach and engagement
Lots to do!!

19. ARA Summer Seminar – Edinburgh 7 June 2019
Many thanks!
Questions
e: stuart.macdonald@hes.scot