Metadata Harvesting And Validation
•
0 likes•938 views
The document discusses metadata harvesting and validation. It describes a validation service that checks metadata against an application profile using XML schema and Schematron rules. An online validation demo is also presented. The harvesting infrastructure uses the OAI-PMH protocol and includes components for harvesting from multiple targets, validating metadata, and generating validation reports. An ARIADNE Harvester manages the harvesting process and interfaces with repositories, validation services, and other systems.
Recommended
More Related Content
Similar to Metadata Harvesting And Validation
Similar to Metadata Harvesting And Validation (20)
More from Bram Vandeputte
Recently uploaded
Recently uploaded (20)
Metadata Harvesting And Validation
- 1. Metadata Harvesting and Validation Bram Vandeputte K.U.Leuven 1
- 3. Overview • Validation Service • Online Validation Service • OAI-PMH • Harvesting Infrastructure 3
- 4. Validation Service • Interoperability : Application Profile (AP) • Manual check : very time consuming • Need a tool for enforcing an AP => validation scheme Best practices derived from previous projects • A set of validation rules such as MELT and MACE Reusable : modular + • Reusable & extendable inheritance possible 4
- 5. Validation Service • Components : • XML schema : structure • schematron : • mandatory/conditional elements • empty fields • vocabularies (auto generated) • ... • Vcard component 5
- 6. Validation Service component : atomic block which does specific validation checking scheme : collection of components that • Terminology : ensures validity against a whole AP • Validation Component URI : unique identifier of a scheme • Validation Scheme • Validation Scheme URI : • http://aspect-project.org/validation/ASPECTv1.0/core 6
- 7. Validation Service 7
- 8. Validation Service ASPECTv1.0/ ASPECTv1.0/ LOM loose recommended core recommended lomloose.xsd vocabulary bank schematron rules core schematron vcard validator Legend rules uses empty attribute fields extends IMS ILOX ASPECT validationScheme vcard validator validation component 8
- 9. Validation Service 9
- 10. Online Validation Service demo 10
- 12. validation to lre AP refer to lre ap document
- 13. OAI-PMH • Client - Server model • Pull mechanism • options : • selective harvesting (date and set) • incremental harvesting • Metadata-agnostic 13
- 14. OAI-PMH • Verbs : Identify, ListRecords, GetRecord • Parameters : • baseUrl • from & until date • metadataPrefix • sets 14
- 15. Harvest Component • Multiple targets • Each target separate properties (sets, date granularity, metadataPrefix, ...) • Storing metadata (SPI, Filesystem, APP, ...) • Extra features : • Incremental harvesting • harvesting scheduling • Metadata validation + reporting • OAI-PMH Target validation • (User Friendly) GUI 15
- 16. invalid : discarded or identifier recorded for next harvesting 16
- 17. The Harvest component invalid : discarded or identifier recorded for next harvesting 16
- 18. ARIADNE Harvester invalid : discarded or identifier recorded for harvester log next harvesting 16
- 19. ARIADNE Harvester ASPECT Repository SPI SQI invalid : discarded or identifier recorded for harvester log validation service next harvesting 16
- 20. ARIADNE Harvester ASPECT Repository SPI SQI External Repository OAI OAI-PMH LOM LOM LOM invalid : discarded or identifier recorded for harvester log validation service next harvesting 16
- 21. ARIADNE Harvester ASPECT Repository OAI-PMH SPI SQI External Repository LOM LOM LOM OAI OAI-PMH LOM LOM LOM invalid : discarded or identifier recorded for harvester log validation service next harvesting 16
- 22. ARIADNE Harvester ASPECT Repository OAI-PMH SPI SQI External Repository LOM LOM OAI OAI-PMH LOM LOM LOM LOM invalid : discarded or identifier recorded for harvester log validation service next harvesting 16
- 23. ARIADNE Harvester ASPECT Repository OAI-PMH SPI SQI External Repository LOM LOM OAI OAI-PMH LOM LOM LOM LOM LOM invalid : discarded or identifier recorded for harvester log validation service next harvesting 16
- 24. ARIADNE Harvester ASPECT Repository OAI-PMH SPI SQI External Repository LOM LOM OAI OAI-PMH LOM LOM LOM LOM Validation Msg invalid : discarded or identifier recorded for harvester log validation service next harvesting 16
- 25. ARIADNE Harvester ASPECT Repository OAI-PMH SPI SQI External Repository LOM LOM OAI OAI-PMH Validation LOM Msg LOM LOM LOM invalid : discarded or identifier recorded for harvester log validation service next harvesting 16
- 26. ARIADNE Harvester ASPECT Repository OAI-PMH SPI SQI External Repository LOM LOM OAI OAI-PMH LOM LOM LOM LOM Validation Msg invalid : discarded or identifier recorded for harvester log validation service next harvesting 16
- 27. ARIADNE Harvester ASPECT Repository OAI-PMH SPI SQI External Repository LOM LOM OAI LOM OAI-PMH LOM LOM LOM Validation Msg invalid : discarded or identifier recorded for harvester log validation service next harvesting 16
- 28. ARIADNE Harvester ASPECT Repository OAI-PMH SPI SQI External Repository LOM LOM LOM OAI 2 6 LOM OAI-PMH 1 Validation LOM Msg LOM LOM LOM 4 5 3 Validation Validation LOM Msg Msg harvester log validation service 17
- 29. Harvester Screenshot or live demo 18
- 33. Validation Reports • After harvesting -> report generated and put online • report has 4 “levels” : • full log (incl. metadata) • reporting log • Grouped Errors • Error Summary
- 34. • Questions ? 23
- 35. References • SPI : http://ariadne.cs.kuleuven.be/lomi/index.php/ SimplePublishingInterface • IEEE LOM : http://ltsc.ieee.org/wg12/ • OAI-PMH : http://www.openarchives.org/