DuraSpace Solutions: “The Features and Updates of Fedora 3.6”
Thursday, October 18, 2012
11:00 AM - 12:00 PM (Eastern Time)
Presented By:
Edwin Shin (Fedora Release Team Leader, Managing Partner & Technical Director of MediaShelf)
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Fedora 3.6 webinar slides 10 18-12
1. DuraSpace Solutions
The Features & Updates of
Fedora 3.6
Presented by Edwin Shin,
Fedora Release Manager,
Managing Partner & Technical Director of
MediaShelf
18 October 2012 Fedora 3.6 1
2. Using the Webinar
Platform
• 2-way audio for all
participants is muted
• We‟ll utilize the Chat
Window for the Q&A
portion or you may use it if
you are having technical
difficulties
• You may type your
question here & hit „enter‟
18 October 2012 Fedora 3.6 2
5. MediaShelf
• Est. 2006 • REST API • Steering Group
MediaShelf
Fedora
Hydra
• Core Services • Fedorazon • HydraCamp
• Strategy • FeSL • Hydra Components
• Training • 3.6 • ActiveFedora
• Development • 3.6.1 • Solrizer
• Support • Fedora Support • OM
• Offices Services • Hydra Heads
• U.S.A. • Fedora Futures • Libra
• Singapore • Hydra@Hull
• HyPATIA
• ScholarSphere
5 Fedora 3.6 18 October 2012
6. Fedora 3.6 Timeline
Fedora 3.5 Fedora 3.6.1
18 August 2011 6 September 2012
Fedora 3.6
7 August 2012
6 Fedora 3.6 18 October 2012
7. Fedora 3.6 Contributors
1. Aaron Birkland*, Cornell University
2. Adam Soroka*, University of Virginia
3. Asger Askov Blekinge*, State & Univ. Library of Denmark
4. Ben Armintor*, Columbia University
5. Chris Beer, Stanford University
6. Chris Wilper*, DuraSpace
7. Edwin Shin*, MediaShelf
8. Frank Asseg*, FIZ Karlsruhe
9. Graham Triggs, BioMed Central
10. Gregory Jansen, University of North Carolina at Chapel Hill
11. Jerry Pan, Oak Ridge National Laboratory
12. Jiri Kremser, Masaryk University
13. Jonathan Green, Discovery Garden
14. Stephen Bayliss*, Acuity Unlimited
7 Fedora 3.6 18 October 2012
8. 3.6: Web Services Framework
Replaces Axis 1.3 with CXF
MTOM Support
REST Support
Spring Configurable
8 Fedora 3.6 18 October 2012
9. 3.6: More Spring in Your Repository
Newly Spring configurable components
• Server module
• SOAP APIs
• REST APIs
• Security
– Policy Enforcement
– Authentication
9 Fedora 3.6 18 October 2012
10. Low-level Storage Metadata
• A new hints provider API for Akubra
• Allow, for example, the low-level storage
module to make decisions about where
objects or datastreams are stored based
on the provided hints
10 Fedora 3.6 18 October 2012
11. Upstream AuthN
• New, “experimental” authentication filter
– Enable integration with Single-Sign-On Services
11 Fedora 3.6 18 October 2012
12. fedora-batch
• A batch controller for Fedora‟s REST API
– http://mediashelf.github.com/fedora-batch/
• Adds a new REST API endpoint, e.g.:
– http://localhost:8080/fedora/batch/getDatastre
ams?pid=demo:1&pid=demo:2&dsid=DC
12 Fedora 3.6 18 October 2012
13. rightsmetadata-policyfinder
• A XACML PolicyFinderModule for Hydra
rightsMetadata
– https://github.com/projecthydra/rightsmetadat
a-policyfinder
13 Fedora 3.6 18 October 2012
14. End of Life
Legacy FeSL
AuthN AuthN
fcrepo- fedora-
client- client-
messaging messaging
14 Fedora 3.6 18 October 2012
15. Should I Upgrade?
① Simple, binary upgrade from 3.5, no rebuild
② 3.6.1 is the best, recommended version of
Fedora today
① Already in production use
② Already part of hydra-jetty
15 Fedora 3.6 18 October 2012
16. Commercial Support
• Fedora Support Services
– Global support in the U.S., Europe and Asia,
staffed exclusively by Fedora committers
– 20% of subscription fees directly fund ongoing
Fedora development by the committers
16 Fedora 3.6 18 October 2012
17. What’s Next?
① Java 7
② Servlet 3.0
③ Fedora 3.6.2
④ Fedora 3.7
⑤ Fedora Futures
17 Fedora 3.6 18 October 2012
18. Fedora 3.6.2
• Bugfixes
• Release Manager
– Adam Soroka, University of Virginia
• Code Freeze
– TBD, but ~ 1st November
• Release Date
– TBD, but ~ end of November
18 Fedora 3.6 18 October 2012
19. Fedora 3.7 aka 3-dot-last
• Focus
– Ease long-term maintenance
• Remove “legacy” code that lacks an active
commitment to maintain
• OpenJDK 7 support
• Servlet 3.0
• Java 7
• Release Manager
– Ben Armintor, Columbia University
19 Fedora 3.6 18 October 2012
20. Fedora Futures
Opportunities & Challenges
• Research Data
Management
• Linked Data
• Performance
• Scalability
20 Fedora 3.6 18 October 2012
21. Fedora Futures, cont’d
• Product Manager, Pro Tem
– Edwin Shin
• CNI Panel Presentation
– 10 - 11 December 2012
21 Fedora 3.6 18 October 2012
22. Get Connected
• Weekly developer calls
• Thursdays 9:00 AM U.S. Eastern
• Mailing lists
• fedora-commons-users@lists.sourceforge.net
• fedora-commons-developers@lists.sourceforge.net
• IRC
• chat.freenode.net #duraspace
22 Fedora 3.6 18 October 2012
24. So long,
and thanks for all the fish!
24 Fedora 3.6 18 October 2012
25. Q&A
Thanks!
edwin.shin@yourmediashelf.com
25 Fedora 3.6 18 October 2012
Editor's Notes
… managing partner at MediaShelfA very brief background about my history w/ Fedora, just to provide some context for what’s to come.
…[ok, done w/ introductions and background]
Fedora 3.6 was released on August 7th, nearly one year after the release of version 3.5.That was really far too long between releases and so we’re endeavoring to release on a more rapid cycle.3.6.1, by contrast, was released just one month later. And we are anticipating a release of 3.6.2 for November
I’ve included a list of the contributors to the 3.6 & 3.6.1 releases. As I generated this list from our commit logs, it is notably missing the folks who provided valuable contributions in the form of patches, bug reports, documentation and testing. In particular, I would call out Dan Davis at Cornell and Scott Prater at Wisconsin-Madison, who are both committers, for their help in getting the releases out.By way of legend, the institutional affiliations listed reflect the institutional affiliation at the time of the contribution, rather than present-day affiliation. The stars following some of the names indicate folks who are current Fedora committers.
One of the headline features of 3.6 was the updated web services framework.As part of the 2011 Google Summer of Code, we had a student, Jiri Kremser, a doctoral student at Masaryk University in the Czech Republic, and mentored by our own Adam Soroka and Chris Wilper, work on one of the biggest refactorings of Fedora in recent history. The project was to replace the aging Axis 1.3 web services library with a modern web services framework, namely Apache CXF.Ultimately, this is plumbing within what is itself middleware, so the external benefits might be a bit opaque. Notably, this now allows Fedora to support the MTOM (the W3C Message Transmission Optimized Mechanism) which provides a 33% size efficiency over the Base64 encoding of binary data supported by earlier Fedora versions.Two of the immediate benefits of the switch to CXF: Spring configurability and REST support, I’ll return to later with an example.
By way of background:Fedora’s original Server and Module classes were designed in 2002 to provide a means for the major functional components of the repository to be plugged in, configured, started and stopped. But Fedora’s module framework is homegrown and better, more widely-understood frameworks have arrived. Fedora’s module framework also makes testing difficult, which in turn slows down development of new features.Spring is a popular dependency-injection framework (and more) that lets usResolve inter-moduie dependencies via dependency injectionProvide hooks to initialize modules when the webapp container starts and stopsAllow re-configuration and plugging in of alternative modules without re-compilingThe Spring-ification of Fedora started with version 3.5. Thanks to some pretty herculean efforts by Ben Armintor, 3.6 has even more Spring support than ever.This allowed us to shift the management and configuration of various aspects of Fedora from the webappsweb.xml and fedora.fcfg to the Spring 3 Framework.The benefit of these changes is probably most evident to the Fedora developers, but depending on your deployment environment, there are a number of externally visible benefits:Pushing configuration out of web.xml means we can keep the Fedora WAR static, which is a welcome change for most sysops teams.It allows Fedora to be much more easily integrated into or with other applications that are already Spring-ified.It lets you extend or add features to Fedora with little or no changes to the webapp itselfI’ll provide some examples of adding features to Fedora that take advantage of Fedora’s new spring-iness later on.
Fedora now includes a hints provider API, developed and contributed by Jerry Pan, then at Oak Ridge National Laboratory.A hints provider implementation could collect such metadata as mimetype or collection membership and pass that to the low-level storage module. This would allow, for example, the low-level storage module to make decisions about where objects or datastreams are stored based on the provided hints or metadata.Note that there is no actual ready-made implementation of this API yet. It is very much an exercise left to the reader. But it provides hooks that were wholly absent prior.
An experimental facility available for "upstream authentication", which is meant to allow integration with SSO services such as Shibboleth or CAS.Many of these systems will add attributes in HTTP headers to indicate that a user has authenticated. This work enables Fedora to access authN/Z attributes provided by such an upstream attribute supplier. The attributes would indicate that the user has successfully authenticated, and the attributes would be used to authorize access to objects. As with LDAP authN, this would reduce the need to Fedora to maintain its own lists of user credentials, allow Fedora to use institution-wide information, such as user groups and roles, and also allow users to perform authN with their institutional credentials.
fedora-batch is a proof-of-concept implementation of a batch api for Fedora.Say, for example, you want to display the metadata for 10 different Fedora objects on a web page. For the sake of example, let's assume we're working with 10 Fedora objects which each represent a video resource and that the metadata we're interested in is contained in each object's DC datastream.Currently, you might tackle this by issuing 10 separate getDatastreamDissemination requests to Fedora. However, what if you could just issue a single batch request to Fedora and have Fedora assemble and return a single response?So, I bring up fedora-batch, not so much because of any inherent utility (although I think it’s kinda cool), but because it’s an illustration of how you can now extend Fedora functionality without touching the core. I can install fedora-batch into my copy of Fedora running on Tomcat 7 by dropping in a single jar file and it will add the /batch API endpoint and provide all sorts of new functionality that was hitherto unexposed. Fedora-batch takes advantage of Fedora 3.6’s new CXF web services stack and improved Spring-iness
This is *very* much a work in progress, but I thought worth highlighting to this audience as another example of the kind of flexibility that Fedora now affords.rightsMetadata is very simple schema for access rights developed by the Hydra Project. It was developed very much as a reaction to the complexity of XACML policies. It’s just a bit of XML persisted in a Fedora datastream that declares what users or groups can discover, read or edit a resource.For years now we’ve talked about implementing some means of getting Fedora to respect the access rights asserted in rightsMetadata. Well last month, at the Hydra Partners meeting at Notre Dame, Ben Armintor & I stole away for part of a day and knocked this out. Actually, Ben did all the work, I just played Salt N Pepa every time he pushed a commit to github.So with a little FeSL and Spring magic, you can actually have Fedora use the vastly simpler rightsMetadata as Fedora’s policy language instead of XACML.
I would be remiss if I didn’t provide the warning of things that are not much longer for this world as of 3.6:The legacy authentication module has been deprecated since Fedora 3.4 and is planned to be phased out in the next release.The fcrepo-client-messaging library has been deprecated as of 3.6 and is planned to be phased out in a subsequent release.
Absolutely.
Many of you will have seen Jonathan Markow’s email last month about “Fedora Futures”:At Open Repositories this summer in Edinburgh, a group of Fedora stakeholders with representation from large institutions and major projects like Hydra, Islandora, APTrust, and eSciDoc self-organized to plan some post-conference, informal discussions about the future of Fedora.Today, Fedora is in use around the world, and serves as a durable repository for hundreds of institutions, with years of proven, production-level software supported by a widecommunity. But as Fedora has matured, new needs are emerging for scaling, performance and the ability to integrate into wider ecosystems. Research data management, linked data, and ease of incorporation into frameworks like eSciDoc, Hydra, Islandora and microservice-based architectures have become paramount concerns.Last month, in DC, that group of stakeholders gathered again to discuss how to meet these emerging needs and position Fedora to not just survive but thrive in the face of these opportunities. The outcome was very positive, the group found a lot of common ground and interest on possible approaches to advancing Fedora. Most importantly, a commitment was expressed to mobilize resources in the short term to organize a sustained effort to scale up the development of Fedora.
The very latest news to which you as the audience to this webinar are the first to be privy to is that DuraSpace and MediaShelf have come to an agreement where I will serve as the Interim Product Manager for Fedora. Details are scarce at the moment, but we’ll have more at CNI this December in D.C.At this stage, my near-term priority is to develop the product vision. This includes
As everyone knows, Chris Wilper recently left the DuraSpace team. I just wanted to take a moment to acknowledge Chris’ many, many contributions to Fedora and to our community over the years.Chris joined the Fedora Project in 2002 and played a tremendous role in stewarding Fedora these past ten years through not only his technical engagement but also in building the developer community that supports Fedora today.I hope you’ll all join me in wishing him the very best in his new adventures at Thompson Reuters.