Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
The New DRS: Plan for Metadata Migration
1. The New DRS:
Plan for Metadata Migration
Harvard Library & Library Technology Services
February 26, 2014
2. Agenda
Welcome and introduction …... Franziska Frey, Tracey Robinson
Migration challenges .…………... Randy Stern
Creating the plan ………………..... Kate Bowers
Walkthrough of plan …………….. Andrea Goethals
Imaging Services:
minimizing disruption……….… Bill Comstock
Wrap-up & next steps ………...… Kate Bowers, Andrea Goethals
Q & A ……………………………….…... All
4. The DRS Advisory Group…
…provides oversight and guidance during the rollout
phase of the DRS2 project and ensures that the user
community of active DRS depositors and content
owners contributes to decisions about the rollout.
Amy Benson
Kate Bowers
Bill Comstock
Franziska Frey (chair)
Andrea Goethals
Wendy Gogel
Tracey Robinson
Randy Stern
5. Why a New DRS?
• Upgrade to best-in-breed technologies
• Adopt digital preservation best practices
and standards
• Preserve metadata better
• Improve collection management
• Support preservation planning & activities
• Improve access to content & metadata
• Support more formats & genres
6. Preservation Capability Before and
After the DRS2 Project
Level One Level Two Level Three Level Four
Storage & Geographic
Location
File Fixity and Data Integrity
Information Security
Metadata
File Formats
= already compliant = will be compliant after the DRS2 project
Based on the NDSA Levels of Digital Preservation
7. Evolution of the DRS
2000 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 20122001
Current
DRS in
production
New DRS in
productionDRS enhancements
New DRS infrastructure
development
2013 2014 2015
New DRS metadata
migration
& user adoption
8. New DRS - Completed
2009 2010
2011 2012
convened
DRS
Advisory
Group
software in
production
2013 2014 2015
users trained,
phase 1
hardware in
production
migrated content
to new hardware
Infrastructure
Development
Metadata
Migration
& User
Adoption
Fedora
assessment
DuraCloud
pilot test
early release beta 1 beta 2
beta 3
first object
deposited
to the new
DRS
9. New DRS - Upcoming
2009 2010
2011 2012 2013 2014 2015
Infrastructure
Development
Metadata
Migration
& User
Adoption
metadata
migration tools
created
metadata
migrated
users moved
14. New DRS Data Model
• Not a simple metadata conversion
• A new DRS object is a logical intellectual
entity that unifies multiple DRS files
– Still image objects - archival and production
masters, and deliverables including thumbnails
– Audio objects - archival and production
masters and deliverables
– PDS objects - page image and text files
15. Object Descriptors
• METS files generated for each object
– Standards-based internal schemas (PREMIS,
MODS, MIX, etc.)
• Metadata gathered from multiple sources
– Current DRS database
– Every content file
– HOLLIS records
16. Technical Challenges
• Many formats
– Images, audio, text, digitized books, web
sites, documents, biomedical image stacks,
opaque files
• Unique migration rules per format
– technical metadata, roles, relationships
• Large (>5000 file) PDS documents
• 45+ million DRS files
17. Technical Challenges
• At 1 sec/file, 45 million files would take
520 days!
• We are designing the migration
software tools for parallel processing
• We are configuring multiple servers to
run the migration
20. Formulating a Migration Strategy
• Analysis of:
– DRS content
• Technical (relationships, etc. for building objects)
• Volume and type by repository
–metadata for mapping
–user activity in the DRS
–survey of highest volume, active users
–training and testing registration lists
21. Migration Strategy Factors
• Combines needs of users with technical
requirements
• User sequencing will be based on:
– Current deposit & administrative activity
– Level of preparation (training and participation
in beta testing)
26. Migrating Content in 5 Stages
Tier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 PDS Document, Still Image
3 Audio, Text (SMIL)
4 Web Harvest, Opaque Container
5 Biomedical Image; Google Document Container 1, 2, 3
27. Migrating Content in 5 Stages
Tier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 PDS Document, Still Image
3 Audio, Text (SMIL)
4 Web Harvest, Opaque Container
5 Biomedical Image; Google Document Container 1, 2, 3
28. Migrating Content in 5 Stages
Tier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 PDS Document, Still Image
3 Audio, Text (SMIL)
4 Web Harvest, Opaque Container
5 Biomedical Image; Google Document Container 1, 2, 3
Tiers 1, 3, 4, 5: Migrate across all DRS owner codes at one time
Tier 2: Migrate one DRS owner code at a time
29. Tier 2: Sequence by DRS Owner Code
• Migrate just your unit’s PDS document and
still image content
• Minimize the amount of time the content
you manage the most is in 2 different
systems
31. Timing
• Current estimates:
–Building & testing migration tools: Now
–Begin Tier 1 content: Spring 2014
–Begin Tier 2 content: Summer 2014
• Units will be contacted about their Tier
2 migration schedule
32. After Your Tier 2 Migration
• You, and anyone depositing on your
behalf, will begin depositing only to the
new DRS
• All of your management tasks will be
done only in the new DRS
34. Minimizing Disruption
• Testing by Imaging Services
• Uninterrupted services
• Migration sequencing
• Participating as a “pioneer”
35. Testing by Imaging Services
Alpha and beta testing:
• Depositing processes
• DRS content maintenance tools
–Searching and assembling content for
download
–Editing PDS objects
36. Uninterrupted Services
Providing services before and after your
migration
• Content needs to be deposited
• Content needs to be searched
• Content needs to be assembled
• may need to be edited
• may need to be downloaded
37. Migration Sequencing
• We will synchronize deposits with your
migration
–start depositing for you in the new DRS
after your Tier 2 content is migrated
38. Imaging Services as Pioneers
As pioneers, we:
• Learn to use the new tools
• Refine the new depositing workflows
• Identify bugs
• Suggest improvements
• Create a group of local experts that can support
those that follow
We’ll wear the scars so that you can stay pretty!
40. Nine Pioneers
• Limited number of first depositors to new
DRS
• Factors
– New DRS-ready content from new systems
• EAS (Electronic Archiving Service), ACORN
(Weissman Preservation Center conservation
treatments), DASH (for ETD)
– Prepared and trained staff
– No content to migrate
• HUA example: opaque objects
41. First Deposit in the New DRS
大藏經 Da Zang Jing - Buddhist sutra, Qing dynasty (1644 -1911), China, Tibetan language
44. Coming Attractions
• Open meetings
– Technical aspects brown bag (March)
– Digital preservation & DRS intro (Summer)
• Training and instruction
– Refresher training
– New training
– Onsite assistance
