Dan Gillean and Justin Simpson from Artefactual Systems gave a presentation on their open source digital preservation applications AtoM and Archivematica. Artefactual develops these applications using open standards and makes their code freely available under open source licenses. They provide technical support and services to help institutions implement and use AtoM for archival description and Archivematica for digital preservation workflows based on the OAIS model. Examples of institutions using AtoM and Archivematica were provided.
Dev Dives: Streamline document processing with UiPath Studio Web
Dan Gillean and Justin Simpson Present on AtoM, Archivematica, and Artefactual Systems
1. Dan Gillean and Justin Simpson
Higher Education Archive Programme Network Meeting
London, UK– December 2, 2016
An Introduction to
AtoM, Archivematica, and
Artefactual Systems
2. Who are we?
Justin Simpson
Director, Archivematica Technical Services
Dan Gillean, MAS, MLIS
AtoM Program Manager
3. Who is Artefactual?
Artefactual Systems is dedicated to producing
standards-based open source web applications
for digital preservation, description, and access.
We are the lead developers of AtoM (for archival
description and access) and Archivematica (for
managing digital preservation workflows).
4. We are archivists, librarians, system administrators,
and developers that are passionate about open-
source and standards-based systems. We are also
one of the most hard-working and respected
technical crews in the archives and library community.
We put out and support several Archivematica and
AtoM releases a year and deliver technical services to
a growing list of satisfied clients.
Our Team_
6. Our Development Approach
• Standards-based / follows best practices
• Open source / Creative Commons
• Multilingual support whenever possible
• Generalize specific use cases
• Include all features in public release
• Iterative development via multiple
contributions over subsequent releases
Community-based
Development
7. Our Business Model
The “Bounty Model”
of Open Source Business
• Maintain our applications under A-GPLv3
licenses
• Provide and maintain the documentation
under CC-BY-SA licenses
• Produce additional resources (e.g. videos,
presentations, webinars) whenever possible
• Participate actively in the user forums
• Offer additional paid services
• Always include development in public project
8. Our Services
• Hosting
• Remote technical
maintenance
• Consultation
• Data migration
• Training
• Application theming
• Software development
https://www.artefactual.com/services/
9. Our Applications
https://www.accesstomemory.org
Web based application for
standards-based description and
access in a multilingual, multi-
repository environment
Web based application to
support digital preservation
workflows based on the OAIS
reference model
https://www.archivematica.org
10. Our Applications
https://www.accesstomemory.org
Web based application for
standards-based description and
access in a multilingual, multi-
repository environment
Web based application to
support digital preservation
workflows based on the OAIS
reference model
https://www.archivematica.org
12. Web based: Once installed, you only need a
modern web browser and an internet
connection to connect to the application from
anywhere
Standards-based: AtoM implements existing
national and international archival content and
metadata transmission standards and best
practices, so your data is standards compliant
and interoperable
Multilingual: Supports translations of both
content and user interface elements, can be
used as a multilingual catalogue
Multi-repository: Can be used as a portal site,
content aggregator, or union catalogue to
provide access to content from multiple
institutions
What is AtoM?AtoM stands for
Access to Memory
It is a web-based, open
source application for
standards-based archival
description and access in
a multilingual, multi-
repository environment.
13. Web-based:
platform independent
Browser-based user interface.
• Anyone with access to a browser (e.g., Chrome,
Internet Explorer, Firefox, Safari etc.) has access to all
the features and functionality of the AtoM application.
Platform independent application.
• The application runs on a web server that can be
installed and run on many platforms.
14. Open source:
free as in freedom
AtoM is built with open source tools
(NGINX, MySQL, Symfony,
Elasticsearch), rather than proprietary
(i.e., closed-source) ones.
The underlying software code of
AtoM is open source.
Free to use
Free to modify
Free to share
15. Documentation is freely available
Public User Forum for troubleshooting
and community-building
Wiki with community resources, example
users, etc.
YouTube video tutorials and webinar
recordings
Open source:
free as in freedom
20. 2009 2010 2011 2012 2013
0.X-BETA 1.0-BETA
FIRST
NON-BETA
RELEASE
1.1 2.01.31.2
TRILLIUM THEME
AtoM’S DEVELOPMENT
21. 2016 2017 2018
1.x 2.22.0 2.42.3
JOB SCHEDULER
2.0.1 2.1.1
20152014
2.1
2.2.1
2012
1.3.1
CLIPBOARD
AtoM 2.4
• Full bulk import / export via the user
interface
• Search index improvements
• Authority records and repositories on
the Clipboard
AtoM’S DEVELOPMENT
23. You can find many more examples of AtoM sites from around the world on our wiki Users page. If
you don’t see your institution’s AtoM installation listed here and would like to, please send us an
email and we will be happy to add it!
https://wiki.accesstomemory.org/Community/Users
25. What is Archivematica?
Archivematica is a web-
and standards-based,
open-source application
which allows your
institution to preserve
long-term access to
trustworthy, authentic
and reliable digital
content.
Standards based
Open source
Customizable
Integrated w 3rd
party systems
Active community
26. 2009 2010 2011 2012 2013
2007: UNESCO REPORT 0.1-ALPHA
DASHBOARD
INTRODUCED
0.7
1.0
RELEASED!0.9
0.8
Bradley, K., Lei, J., Blackall, C.
Towards An Open Source
Archival Repository and
Preservation System (2007)
Planning and development begin.
Initial Funding via UNESCO MotW
Subcommittee, IMF Archives, City of
Vancouver Archives
0.6-ALPHA
February 2010
May 2010
February 2011 February 2012
PREMIS
in
METS
0.10
April 2013
August 2012
STORAGE
SERVICE 0.2
January 2014
Archivematica’s development
29. PREMIS in METS XML
Packaged according to BagIt specifications
Virus scan, normalization report, extraction log, etc
For browsing in Archivematica
Original + normalized
objects, submission
docs, original metadata
included at SIP creation
Archivematica AIP structure
31. Micro-services Handshakes Partnerships
Gears – Joe DeSousa.
https://www.flickr.com/photos/mustangjoe/22711070429
Metal Handshake – Grey Geezer.
https://commons.wikimedia.org/wiki/File:Metal_Handshake.jpg
Hands Passing Baton - tableatny,
https://www.flickr.com/photos/53370644@N06/4976497160
Do one thing well…Inside and Out
38. archivesDIRECT
• Partnership with DuraSpace
• U.S. Based
• Launched August 2014
• Secure storage and
monitoring via DuraCloud
• Artefactual provides AM
technical support
http://archivesdirect.org/
39. Perpetua
• Partnership with Arkivum
• U.K. Based
• Launched July 2016
• Secure storage and
monitoring via Arkivum
• Artefactual provides AM
technical support
http://arkivum.com/perpetua/
40. ArchivesCANADA
Digital Preservation Service
• Partnership with The
Canadian Council of
Archives (CCA)
• Canada Based
• Launched September 2016
• Artefactual provides AM
technical support, storage,
monitoring
http://archivescanada.ca/ACDPS
41. Some examples of how AtoM and Archivematica are used by our global communities
AtoM and Archivematica
Use Cases
42. AtoM Use Cases
Medium to Large Deployment
Model (Artefactual):
• ES and MySQL deployed on separate
server
• Internal read/write and testing
webservers
• Separate public read-only front-end
web server – content proxied via firewall
and replication script
53. Archivematica Use Cases Simon Fraser University
Building a Digital Repository with Archivematica and AtoM
54. Archivematica Use Cases Simon Fraser University
Building a Digital Repository with Archivematica and AtoM
http://www.sfu.ca/content/dam/sfu/archives/DigitalPreservation/FormatPolicyRegistry.pdf
56. RESOURCES
AtoM homepage: https://www.accesstomemory.org
AtoM demo: http://demo.accesstomemory.org
Roadmap: https://wiki.accesstomemory.org/Releases/Roadmap
Documentation: https://www.accesstomemory.org/docs/latest/
User forum: https://groups.google.com/forum/#!forum/ica-atom-users
57. RESOURCES
AM homepage: https://www.archivematica.org
AM demo: http://sandbox.archivematica.org
Wiki: https://wiki.archivematica.org
Documentation: https://www.archivematica.org/docs/
User Forum: https://groups.google.com/forum/#!forum/archivematica
Notas do Editor
AtoM is short for Access to Memory. It is a web-based, open source application for standards-based archival description and access in a multilingual, multi-repository environment. Now that’s a mouthful, so we’re going to break down each of these elements a bit more in the following slides.
------
Web based: Once installed, you only need a modern web browser and an internet connection to connect to the application from anywhere
Standards-based: AtoM implements existing national and international archival content and metadata transmission standards and best practices, so your data is standards compliant and interoperable
Multilingual: Supports translations of both content and user interface elements, can be used as a multilingual catalogue
Multi-repository: Can be used as a portal site, content aggregator, or union catalogue to provide access to content from multiple institutions
Web-based: All user interactions with the system (add, view, search, edit, and delete actions) take place through a web browser. Users access HTML pages on the web server; clicking a button or link triggers a PHP script that sends a command to the database and returns the output as HTML back to the user’s browser. Once installed, you can view or edit AtoM descriptions from anywhere, with nothing but a web browser and an internet connection.
Platform independent: AtoM was built using open-source tools that can be installed or virtualized on any number of platforms. The team at Artefactual prefers to use Nginx as our web server in both development and production environments, but Apache has been tested and used successfully by our community as well.
The underlying AtoM code is itself open source, with the source code freely available for use or modification as users or other developers see fit (under the A-GPL version 3 license). The Affero General Public License version 3 makes the source code available for users, enables collaboration between developers of web services and other networked software. Therefore there is no cost to download any of the software required to run the AtoM application – use it with or without Artefactual support. You are also free to study the code, modify it, distribute your modifications, and even offer your own additional services for AtoM.
We also do everything we can to enrich our user community with additional resources and ensure there are no barriers or paywalls to using the application. Our documentation is also freely available under a Creative Commons Attribution-ShareAlike license (CC BY SA), and we maintain and actively participate in a free public User forum. The creative commons license means that, like the application itself, you are free to share, copy and redistribute our docs in any medium, and you can adapt the materials for any purpose, so long as you give appropriate credit and distribute your contributions under the same license.
We also maintain a wiki with additional resources, including a list of those created by our community, and try to regularly provide free webinar recordings and video tutorials via our YouTube channel.
Standards based: Part of ensuring that AtoM metadata remains interoperable and embracing a larger community is to provide compliance with international and national content standards.
AtoM aims to simplify description – content standards such as ISAD(g) and others are internally mapped to metadata standards (e.g. EAD or XML) so users can enter data in familiar fields. We crosswalk each additional standards based template, so you can easily switch to a different display standard (such as the Canadian RAD standard, the U.S DACS standard, MODS, or Dublin Core) without having to change your data. Helpful tooltips drawn straight from the standards are provided with every field. Users can even set different display standards within the same repository – and even within different levels of the same archival unit!
Multilingual: All user interface elements and database content can be translated into multiple languages, using our built-in translation interface. At installation, AtoM also includes many default user interface translations, all generously provided by volunteer translators from the AtoM User Community. If you are interested in contributing translations to our user community, we have resources on our wiki to help you get started!
In AtoM you can create a multi-repository portal in which each archival institution can customize its own page with a logo and banner, a custom background color, and custom content using HTML and in-line CSS.
Select a background color with an intuitive color picker
Add an API key and enable Dynamic Google Maps to identify the location of your repository from the address information entered
Repository logo and background color will appear on all related descriptions throughout AtoM to distinguish them in a multi-repository environment
The Search box can toggle between global search and repository-specific searching.
Archivematica integration: And of course, AtoM comes with built-in integration with the other open source application developed by Artefactual Systems: Archivematica. Archivematica will allow you to ingest diverse digital materials and run them through standards-based workflows that follow the OAIS reference model, to produce interoperable, system agnostic Archival Information Packages (AIPs) for long-term preservation. Archivematica will also normalize your digital object masters into smaller, user-friendly access formats for dissemination, based on user-adjustable policy rules. This Dissemination Information Package (DIP) can then be uploaded to an existing description in AtoM for arrangement, further description, and access. AtoM will store a Unique Universal Identifier (UUID) for both the original file and the source AIP, preserving a link between your access copy and your preserved content.
AtoM was originally developed out of a collaborative effort between the International Council on Archives and UNESCO, with seed funding support from institutions such as The Hogeschool Van Amsterdam Archiefschool, Direction des Archives de France, the World Bank Group Archives, and the United Arab Emirates Centre for Documentation and Research. The 1.0-beta release of the application was launched at the 2008 ICA Congress in Kuala Lumpur, and the first non-beta release, version 1.1, was released in late 2010 after a long period of beta testing with dozens of institutions.
We’ve come a long way since then! In October of 2013, we released the 2.0 version of AtoM, which included an improved search index for better scalability and performance, a new user interface look and feel, faceted search and browse, and more. We also launched a new website, and began overhauling the project documentation. The project continues to grow and improve with every major release, and we are currently preparing for our 2.4 release in 2017.
The AtoM project has a vibrant international community of users. This map shows all the different locales where AtoM has been installed, up to version 2.1 – and these are just the places we know of.
You can find many more examples of AtoM sites from around the world on our wiki Users page. If you don’t see your institution’s AtoM installation listed here and would like to, please send us an email and we will be happy to add it!
Standards based: OAIS, PREMIS, METS, BagIt, Dublin Core
Open source: A-GPLv3 license, free to study, use, modify, etc
Customizable: Add/change/remove FPR rules as needed
Integrated: dSpace, CONTENTdm, Islandora, LOCKSS, AtoM, DuraCloud, OpenStack, Archivist’s Toolkit, Arkivum, ArchivesSpace… etc
Active community:
From source systems
Hand-off to access and description systems
Hand-off for archival storage – repositories or other secure storage
Administrative hand-off
From source systems
Hand-off to access and description systems
Hand-off for archival storage – repositories or other secure storage
Administrative hand-off