The document summarizes a paper presented at the World Conference 2013 in Dubai from October 25-28. The paper reports on achievements of the FIMS (Framework for Interoperable Media Services) task force, a joint effort between EBU and AMWA to define common approaches for integrating media components. It provides an overview of the FIMS 1.0 specification and previews developments in FIMS 1.1, including new repository services for managing digital media archives and an extended metadata set. FIMS aims to simplify interoperability and reusability of media services through a service-oriented architecture.
2. G. Dimino et al.
Service Oriented Architecture (SOA)[1] has become the reference design paradigm for
system integration in many business environments and is becoming popular in the media
environment as well, as many success stories can attest to. SOA is a design
methodology focused on guiding implementers in the realisation of lean systems that are
easy to manage and re-configure, despite their intrinsic complexity. Great attention is
devoted to the decomposition of systems into business processing units that possess a
potentially high degree of reusability.
These business processing units in SOA terminology are called services, and
represent any functionality in the system that adds business value to the product, or in
other words that is part of the value chain. A service must publicly expose a well-defined
interface and behaviour, the service contract. The service contract contains all the
information needed by a consumer to use the service and it represents the only link
between service producers and service consumers. Interfaces must be kept separate
from the implementation and kept stable as much as possible. Two services that expose
the same interface are to be considered interchangeable from the consumer perspective
and equivalently two services that implement the same business function must expose
the same interface. This greatly reduces the number of interfaces that must be
implemented to interconnect different components on a composite system. Another
important prescription of SOA is to keep business process logic separate from service
implementation[2].
THE FIMS TASK FORCE
In 2009 the EBU (European Broadcasting Union) and the AMWA (Advanced Media
Workflow Association) decided to join forces and established a Task Force called FIMS
(Framework for Interoperable Media Services) with the goal of defining a framework
based on SOA that can simplify the interoperability of software based components for the
processing of media through standard interfaces.
After the issue of a Request for Technologies in 2010, a first specification
waspublished in 2011. Itprescribes the common behaviour of services, including job
management, queuing and message exchange patterns and a simple data model to
describe media content. A first set of three services (capture, transfer and transform) has
also been included. In 2012, FIMS was awarded the IBC Judges’ Prize. In 2013 a new
set of services was defined to interface media repositories. It will be included in the new
revision of the FIMS specification, to be officially released in 2014, after clearing the
approval process prescribed by the two sponsor organizations EBU and AMWA.
The FIMS project has a formal governance structure and is managed at two levels.
The Business Board evaluates the business relevance of new project proposals and
defines the requirements. It also has the task of prioritizing the work between different
projects which may compete for the same resources. The Business Board represents the
interests of the end users. The Technical Board estimates the scale of resources
required to deliver a project and provides technical guidelines and requirements to
ensure a consistent approach in the development of products. The Technical Board
represents the interests of the developers for the projects.
The Project Teams are composed of technical experts from a wide rangeof companies
andresources. Anyone who has interest in any specific project is welcome to join the
development team to create the requested deliverables based on the project
requirements. Members of the project team must sign the Participation Agreement (PA)
before contributing to the project. The PA prescribes the rules that each FIMS participant
must follow to comply withthe FIMS copyright policy; that is all resulting products are
compensation/royalty free.As of today, more than 90 companies have signed the FIMS
PA, including manufacturers, system integrators and broadcasters.
2
FIAT/IFTA World Conference 2013 in Dubai
3. Out-of-the-box interoperable components for the design of digital media archives
FIMS works on a voluntary basis and all the efforts are freely funded by the
participants through provision of manpower or donations to hire experts when needed.
THE FOUNDATIONS: SPECIFICATION 1.0
The FIMS 1.0 (v1.0.7) specification was formally approved by the AMWA and the EBU in
September 2011, and comprises Part 1, the General Description[3], and Part 2, a multisection document describing the Base Schema[4]and the Transfer[5], Transform[6] and
Capture Services[7]. An accompanying package of schema files [8]is also available.
This specification defines a common approach to the integration of software
components in modern media production facilities, based on an overall framework for
integration of reusable components for multimedia content production. This framework for
media services includes specific detailed definitions of common media service interfaces.
Figure 1 shows the overall reference model of the FIMS framework. In simple words
the role of FIMS is to provide an abstraction layer between media processing
components and the orchestration engine that implements the business processes
required by the users, who interact with the system via an application layer, out of scope
from the FIMS perspective.
Figure 1. The FIMS Reference Architecture
SOA-based media workflows are often long running processes, sometimes active for
hours, days, or even weeks. Therefore, the FIMS specification defines two different service
invocation patterns:
Job management services, that run fast, can be invoked synchronously, that is the service
response is issued immediately after the request
Processing services, that usually require considerable time to run, are invoked
asynchronously, that is the reply contains only an acknowledgement of the request, while the
response due at the end of processing will be autonomously issued by the service at the
address provided by the client in the request to notify of process completion or failure.
A command is called “job”. All the services must implement the same job management
lifecycle, that includes queue management, prioritization and execution at specified points in
FIAT/IFTA World Conference 2013 in Dubai3
4. G. Dimino et al.
time or upon specific events occurrence. A job in a queue can be canceled or its priority can
be modified. While in running state it can be paused and resumed or cancelled.
A job request contains all the parameters needed for its execution, that is a reference and
optionally, a description of the media content on which the command operates, and the
execution parameters grouped into profiles. An extension mechanism allows for the inclusion
of vendor-specific parameters where needed. This is schematized in Figure 2.
Figure 2.The data model.
As the FIMS approach is to be media format independent,system integrators are
responsible for choosing service implementations that support the required media formats.
To simplify the interaction between services and orchestrators and for early detection of
format mismatch, the FIMS data model allows for a complete description of video, audio and
container technical parameters, so that services don’t have to download bulky content files
before realizing that the user is requesting an operation on a media format that is not
supported. Also, services are required to announce on demand the list of features that are
actually implemented, to simplify dynamic binding of services.
Profiles are a nice feature of FIMS: a set of parameters needed to execute a specific
operation, e.g., transcoding some content to a given format, can be included in a profile and
referenced in subsequent usage via the profile name. This can be used to provide presets for
operations that have to be executed often or to hide vendor specific extensions.
The Business Media Content (BMContent) provides the basic information to reference and
locate media files together with their technical parameters. It can also contain descriptive
information about the content, using a subset of the EBUCore [9] set of descriptors.The
EBUCore is a standard defined by the EBU by adapting the Dublin Core [10] descriptors to the
media production, publishing and archiving needs. See Figure 3.
4
FIAT/IFTA World Conference 2013 in Dubai
5. Out-of-the-box interoperable components for the design of digital media archives
Figure 3. The BMContent object
The FIMS Data Model has been designed to be protocol agnostic, allowing either
SOAP/RPC or REST approaches to implementation, howeverin FIMS 1.0 only SOAP and
XML are fully supported. The mapping to REST and JSON will be included in FIMS 1.1.
As well as defining a core set of common Base schemas, FIMS 1.0 includes definitions
of the interfaces for three basic media services:
Transfer Service: to copy or, optionally, move one or more files to another location (or
to several locations). Five different transfer protocols are permitted: HTTP, HTTPS,
FTP, SFTP and FILE. A Transfer Service may implement one or more of these
protocols.
Transform Service: to alter essence and container formats. This interface builds on
the Transfer Service interface, adding elements of transformation. The main intended
usage is transcoding/transwrapping of media files but it can be easily extended to provide
other types of transformations, like cropping, filtering or color grading.
Capture Service:to convert stream-based real-time input such as HD-SDI or RTP to
one or more files. It is based on the transform service adding provision for real time input
handling, including manual start/stop or bound-to-time events.
The original reason for the inclusion of these three services in the specification was
that of testing the Base schema on some representative real life services. Besides that, it
has soon become evident that they are already sufficient, maybe complemented with
some custom extensions, to cover a useful set of use cases in the media production
environment.
THE NEXT STEP: SPECIFICATION 1.1
As of time of writing, the new version of the FIMS Specification is in its final draft form. It
will be officially published in the first half of 2014, upon clearing the formal acceptance
process in the sponsororganizations, EBU and AMWA. Meanwhile, advanced preview
drafts can be found on the project wiki[11].
This new version contains, besides some minor updates of the Base schema, a new
set of services for the interfacing of media repositories and the mapping of the full FIMS
schemas onto the REST protocol.
FIAT/IFTA World Conference 2013 in Dubai5
6. G. Dimino et al.
THE REPOSITORY SERVICE
The Repository Service is certainly the most relevant addition to FIMS. It provides the
basic functionalities needed to manage persistent contents and their related metadata. It
is not meant for exposing all the features that can be expected from a Media Asset
Management system, but rather to interface generic repositories that can be found at any
stage of the production/archiving chain in any media organization as shown in Figure 4.
FIMS Repository Service
Federation Service
FIMS Repository Service
FIMS Repository Service
Media Asset
Management
FIMS Repository Service
Broadcas
t
System
Metada
ta
databas
e
Video Editing System
Media
Workflow
FIMS Repository Service
Near
Line
Storage
Metada
ta
databas
e
FIMS Repository Service
Archive
Storage
Metada
ta
databas
e
Figure 4. The role of the Repository Service
The main features of the Repository Service are:
– Provide interface for basic media operations, like Ingest, Create, Read,
Update, Delete (CRUD)
– Expose a way to manipulate content and metadata
– Enable a query interface to retrieve media assets
– Represent a service interface to be consumed by a workflow engine
Media Objects are identified and referenced within a repository via their GUID, a
unique identifier provided by the user or generated by the repository itself. Each Media
Object is made of two parts, the BMContent(a descriptor that carries all the metadata
related to the object) and the set of files that contain the media essence. A repository
therefore must implement at least a database to store and index the BMContent
descriptors and a file system to persist the essence files. Any CRUD operation takes as
an argument either a BMContent or a reference to one or more essence files. The
interface is deliberately modelled at a rather low level as it is meant to be used to
interconnect repositories to workflow engines and not be exposed to the users at an
application level.The creation of a new object happens in two steps: first the BMContent
descriptor is ingested, then all the related media files are uploaded. The deletion goes in
the reverse order, first all the media files are deleted, then the BMContent is cancelled.
Read and update operations can be applied to either of them.
Events can be automatically generated to notify when the status of any object in the
repository changes.In the FIMS repository model any object is an entity on its own; the
6
FIAT/IFTA World Conference 2013 in Dubai
7. Out-of-the-box interoperable components for the design of digital media archives
relations between objects, typical of MAM systems, have to be handled at a higher level.
Events notifications help to keep in synch the repository status with external sources of
information about the contained objects.
As the configuration and management of a repository, even a small one, is a complex
operation, a Repository Capabilities Registry (RCR) has been defined to store
information about the behavior and implementation level of the FIMS interface. Querying
the RCR it is possible to know the implemented capabilities of a repository, its status,
available storage space, performance and all the vital information needed to properly
monitor the health state of the system.
Vendors can leverage the flexible and generic FIMS interface to expose their
products’APIs as a standardized and robust interface that seamlessly integrates with
other compliant products for media exchange.Media organizationscan minimize
integration costs and system complexity through standardizing operations for storing
media assets in a vendor independent way.
EXTENDED DESCRIPTIVE METADATA SET
In the context of the Repository Service, the FIMS data model role changes. While in
version 1.0 it was used only to convey transient information needed for the service
execution, now it will carry also persistent information about the media object, whose
usage may be in the future. Therefore the data model has been extended to cover all the
elements defined in the EBUCore standard. Namely, the most relevant extensions are
about the element parts that is used to attach metadata related to a temporal region
identified on the essence timeline and the element relation that describes relations
between resources like is-part-of, is-replaced-by or is-version-of.
With these additions the FIMS data model becomes suitable to the exchange of media
objects between archives including their attached documentation. If custom descriptors
are still needed, the extension element allows for the inclusion of any size of “dark”
metadata.
REST
REST (REpresentational State Transfer) is an architectural approach for the design of
web services that is consistent with how the web works, and is becoming widely adopted
within the IT industry. For example, many web streaming and cloud-based transform
services provide a REST API. It is beneficial to both users and manufacturers if FIMS can
be consistent with such an approach, and make use of commodity web development
tools and frameworks. Services can be scaled using off-the-shelf Internet technology,
and can be tested simply with a web browser.
In a RESTful system, a service provides its clients with representations of resources.
The Task Force has from the outset planned to provide REST bindings and so the data
model was designed with resource representation in mind. Examples of resources in
FIMS are a description of video clips and a transfer job. The service makes available
common and limited HTTP operations for creating, reading and modifying resources
dependent on their state. While using a different syntax the REST version of the FIMS
services is fully equivalent to the SOAP one.
As the REST approach is generally paired with the use of JSON (JavaScript Object
Notation) instead of XML, the FIMS data model has been mapped to JSON, allowing
consistent operation with the XML version of the SOAP implementation.
SUPPORT FOR PRODUCT DEVELOPMENT
Correctly implementing the FIMS specification, as any other of the same kind, is certainly
not a trivial task. Therefore the FIMS work is not limited to drafting and publishing the
paperwork but a number of side actions are being undertaken to help implementers to
smooth the learning curve and to test their implementations.
FIAT/IFTA World Conference 2013 in Dubai7
8. G. Dimino et al.
The approval process set up by AMWA requires that a reference implementation be
provided together with any candidate specification. FIMS 1.0 provides with open source
license a reference implementation of the interfaces based on the SOAP/XML protocol. It
can be downloaded from the project wiki. The new version 1.1 will include a new
reference implementation based on REST/JSON, again provided with open source
license.
The FIMS Test Harness is a simulator for both FIMS clients and FIMS services. It
allows FIMS orchestration systems and FIMS vendors to create “simulated” clients and/or
services for testing without having actual vendor’s hardware and/or software. The FIMS
Test Harness is configurable and data driven, in that the simulations can respond and
behave differently based on configured data in requests and request types. It also
validates submitted FIMS requests and provides samples for various job types that can
be used as a reference. It can be obtained from the project wiki.
As several manufacturers have announced products that implement the FIMS
interface, the Task Force is debating which level of certification of compliance can be
provided. A first attempt will be the organization of plugfests, where manufacturers can
check the interoperability of their products under the guidance of the FIMS Technical
Board team. Based on this experience the Group will decide if there is the need to move
to a more formal certification process.
Finally, an Implementation Guidelines document is being compiled, containing several
examples of message exchange and of implementation scenarios.
WHAT’S ON THE HORIZON
A new technical activity has just been started on the definition of an interface for Quality
Analysis of media content.
The Quality Analysis Service Interface will expose capabilities oriented around
analysis and reporting of asset properties. The FIMS Business Board has completed a
draft of the Quality Analysis Service Charter, which was sent to the FIMS Technical
Board to begin work on the specification.
Currently, the FIMS Business Board is evaluating several options for the next service
to be chartered. The initial list of 20+ proposed services has been narrowed down to five;
this discussion is one of the main agenda items for the FIMS Business Board call in
September.
FIMS believes that media industry input on FIMS activities is critical. In order to
promote, recommend, and contribute to FIMS activities we welcome new members to the
FIMS Business Board, for which membership is limited to end users. The Business
Board, which has global representation, evaluates the business relevance of new project
proposals and defines the high-level service requirements. It also has the task of
prioritizing the work between different projects which may compete for the same
resources. The Business Board’s mission is to represent the interests of the industry as a
whole. The board meets regularly the first Thursday of every month; ad-hoc meetings are
scheduled as needed. If you are interested in joining the FIMS Business Board, please
send an email to abbe.wiesenthal@turner.com. More information can also be found at
the FIMS web site [12].
CONCLUSIONS
Software-based media production and archiving system integration is still providing
challengesforboth users and vendors due tolack of interoperability between products.
FIMS is finally bringing standards for the interfacing between orchestration engines and
processing components. Some important achievements have already been reached with
several companies that are already successfully implementing the FIMS specifications.As
an example, Bloomberg Television, together with their technology partner Triskel, have
based on FIMS the integration of their corporate production, rundown and archive
8
FIAT/IFTA World Conference 2013 in Dubai
9. Out-of-the-box interoperable components for the design of digital media archives
platforms. The reported benefits of the FIMS approach sum up to a reduction of 80% in
the time required to integrate in their platform a new third party component, thanks to the
use of standardized and well known interfaces that shield the implementation
peculiarities of each product.
FIMS 1.0 and shortly 1.1 is immediately usable for the design of file based archives.
The new Repository service is a cornerstone for the interoperability of storage
subsystems of any size, providing flexibility and uniform access between vendors.
Transfer, Transform and Capture services provide a generalized interface for the main
components needed to implement ingest and delivery functionalities together with the
forthcoming Quality Analysis service.
The main short term advantages of FIMS for the users are clean, consistentand
vendor-independent design of component interfaces. More substantial advantages are
expected in the midtermwith the growing of the market adoption of FIMS.Standardized
interfaces provide ease of maintenance, scalability and repurposing of systems with the
freedom to always choose best of breed products.
FIMS is gaining a growing interest in the industry but there is still a lot to do as the
media production world is vast and manifold. User support and active participation is
crucial for FIMS success.To see how to contribute please visit the FIMS web site [12].
REFERENCES
[1] Organization for the Advancement of Structured Information Standards (OASIS).2006.
Reference Model for Service Oriented Architecture v1.0http://www.oasisopen.org/committees/soa-rm - 2006
[2] J. Footen, J. Faust.2008. The Service-Oriented Media Enterprise:
SOA, BPM, and Web Services in Professional Media Systems.FocalPress
[3] European Broadcasting Union. 2012. EBU Tech 3356: FIMS Media SOA Framework
V.1.0.7, Part 1: General Description
[4]European Broadcasting Union. 2012. EBU Tech 3356: FIMS Media SOA Framework
V.1.0.7, Part 2 S0: Base Schema
[5]European Broadcasting Union. 2012. EBU Tech 3356: FIMS Media SOA Framework
V.1.0.7, Part 2 S1: Transfer Service
[6] European Broadcasting Union. 2012. EBU Tech 3356: FIMS Media SOA Framework
V.1.0.7, Part 2 S2: Transform Service
[7] European Broadcasting Union. 2012. EBU Tech 3356: FIMS Media SOA Framework
V.1.0.7, Part 2 S3: Capture Service
[8] European Broadcasting Union. 2012. EBU Tech 3356: FIMS Media SOA Framework
V.1.0.7, Schemas
[9] European Broadcasting Union. 2013. EBU Tech 3293: EBU Core Metadata Set
(EBUCore) V1.4
[10] Dublin Core Metadata Inititiative (DCMI).2012.Dublin Core Metadata Element Set,
Version 1.1
[11] FIMS Technical Board Wiki.http://wiki.amwa.tv/ebu
[12] Framework for Interoperable Media Services official web site.http://www.fims.tv
FIAT/IFTA World Conference 2013 in Dubai9