Presentation given by Limecraft at the 2012 EDM Metadata Hands-on Workshop organised by the EBU. Summary: The Material Exchange Format (MXF) has by now become the de facto standard for file-based media exchanges. Even though MXF is incredibly rich in features and flexibility, handling MXF files, and in particular, the metadata that describes the essence within them is still a complex matter as there are many ways in which metadata can be associated with MXF (e.g., as side-car information, as dark metadata or as properly encoded and standardized metadata sets, …). In this presentation we will discuss and demonstrate a recent initiative, supported by EBU, that aims to make the association and processing of MXF metadata easier and more accessible for application developers. In particular, an open source MXF SDK is demonstrated that is able to serialize and deserialize EBUCore metadata natively in MXF audiovisual containers.

1. 2012 EDM Metadata Workshop
Embedded metadata in MXF
Dieter Van Rijsselbergen
EBU, Geneve – 20/06/2012

2. Who are we?

3. Contents
• Metadata exchanges?
• Metadata in MXF
• How to put metadata in MXF?
• Putting EBUCore into MXF
• EBU MXF SDK
• EBU MXF SDK – Demo
• EBU MXF SDK – Improvements & Community

4. Metadata exchanges?
• When exchanging audiovisual essence…
– We must also exchange its identity,
its context, its creators,….
– i.e., the semantics
behind the essence
– in the form of
metadata

5. Metadata exchanges?
• 2 ways to exchange metadata during the
production and broadcasting process
• Out-of-band • In-band /w essence
– Centrally stored – Tightly coupled and
– Optimized for doesn’t get lost
metadata (search, – Often optimized for
augmentation, …) essence storage

6. Metadata exchanges?
• Out-of-band is interesting within the facility
– Metadata is quickly accessible
– Metadata can be indexed, annotated, …
• Embedded metadata is interesting in B2B
– Side-car metadata doesn’t get lost,
doesn’t need a dedicated channel, …
– E.g., news exchange scenarios
• No more need for NewsML XML,
just serialize it directly into the MXF file

7. Contents
• Metadata exchanges?
• Metadata in MXF
• How to put metadata in MXF?
• Putting EBUCore into MXF
• EBU MXF SDK
• EBU MXF SDK – Demo
• EBU MXF SDK – Improvements & Community

8. Essence Exchange: MXF
• The Material Exchange Format is here to stay
• Standardized by SMPTE and EBU
• One of the most versatile, flexible and
powerful container formats defined
– Not sure there’s anything it can’t do…
– But makes it complex and harder to use

9. Metadata in MXF
• MXF has become the de-facto standard for
audiovisual material production and exchange
• Fortunately, it has significant support for
embedded metadata

10. Metadata in MXF
• 2 kinds of metadata in MXF
– Structural metadata
• Defines the structure of the file
– Track layout
– Describes the essence,
which tracks,
which codecs and
parameters per track
– References to other/older files

11. Metadata in MXF
• 2 kinds of metadata in MXF
– Descriptive metadata
• Exchange of essence semantics
– e.g., a continuity report
• At its simplest,
a means of identification
– “let the descriptive metadata reference
identifiers I use internally in my organisation”
• This is the kinds we’re interested in!

12. Descriptive Metadata in MXF
• Standardized MXF descriptive metadata exists
– Descriptive Metadata Scheme-1 (DMS-1)
• Standardized as SMPTE 380M
• Production characteristics, rights information,
annotations, person and entity identification, …
• Aligned with MPEG-7, TV-Anytime
– Is DMS-1 being used for real?
• Perhaps due to the fact that there is no DMS-1 notion
outside of the MXF ecosystem?
– Hard to obtain a viable and complete mapping

13. Descriptive Metadata in MXF
• Which other descriptive metadata are
interesting to consider?
– General identification and description: EBUCore
– Format-specific descriptions: IPTC NewsML-G2
– Acquisition metadata (state of the recording
device): EBU Tech 3349
– Feature detection results: AVDP profile of MPEG-7

14. Contents
• Metadata exchanges?
• Metadata in MXF
• Putting EBUCore into MXF
• How to put metadata in MXF?
• EBU MXF SDK
• EBU MXF SDK – Demo
• EBU MXF SDK – Improvements & Community

15. How to put metadata in MXF?
• Structure of MXF files is KLV elements
– Key to identify the kind of element
– Length of the value that follows
– Value of the element
Key Length Value ...
...
BER-encoded
BER-encoded
16 bytes
16 bytes Length length
Length length
length
length
– KLV is for both essence and metadata
• Typically, for essence a single frame per KLV

16. How to put metadata in MXF?

17. How to put metadata in MXF?
• Use KLV to embed metadata
<?xml version="1.0" encoding="UTF-8"?>
<ebuCoreMain xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:dc=http://purl.org/dc/elements/1.1/
xmlns="urn:ebu:metadata-schema:ebuCore_2011"
xmlns:default=http://iptc.org/std/nar/2006-10-01/
xmlns:ebu=http://ebu.org/nar-extensions/ version="1.3"
dateLastModified="2011-01-31">
<coreMetadata>
<title>
<dc:title xml:lang="fr-can">Berlin Wall 50th</dc:title> Key Length Value
</title>
<alternativeTitle typeLabel="headline"> BER-encoded
BER-encoded
16 bytes
16 bytes Length length
Length length
<dc:title>Berlin Wall 50th</dc:title> length
length
</alternativeTitle>
<creator entityId="EBUNMSSRVCodeSource:TBA"/>
<subject>
<dc:subject/>
<subjectCode>EBUNMSCategoryCode:EXCH</subjectCode>
</subject>
<subject>
<dc:subject>In or about Europe and EU</dc:subject>
<subjectCode>EBUNMSWebcatCode:EUROPE</subjectCode>
</subject>
<description typeLabel="pubStatus">
<dc:description>NMSStatusCode:TBC</dc:description>
</description> Dump the document as a
<description typeLink="EBUNMSDescription:dopesheet">
<dc:description> The Berlin Wall (German: Berliner Mauer) was whole into the KLV value?
a barrier constructed by the German Democratic Republic
(GDR, East Germany) starting August 13, 1961, that completely Not the best approach…
cut off West Berlin from surrounding East Germany…

18. How to put metadata in MXF?
• On top of this raw KLV structure is a
conceptual model
– Elements can be grouped into
logical metadata sets identified
by a single key, i.e., “classes”
– Logical metadata sets can be
reconstructed by means of a dictionary
<Identification base="InterchangeObject" detail="Identification set" type="localSet" baseline="yes" key="06 0e 2b 34 02 53 01 01 0d 01 01 01 01 01 30 00">
<ThisGenerationUID use="required" type="UUID" key="3c 09" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 01 00 00 00"/>
<CompanyName use="required" type="UTF16String" key="3c 01" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 02 01 00 00"/>
<ProductName use="required" type="UTF16String" key="3c 02" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 03 01 00 00"/>
<ProductVersion use="optional" type="ProductVersionType" key="3c 03" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 04 00 00 00"/>
<VersionString use="required" type="UTF16String" key="3c 04" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 05 01 00 00"/>
<ProductUID use="required" type="AUID" key="3c 05" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 07 00 00 00"/>
<ModificationDate detail use="required" type="Timestamp" key="3c 06" globalKey="06 0e 2b 34 01 01 01 02 07 02 01 10 02 03 00 00"/>
<ToolkitVersion use="optional" type="ProductVersionType" key="3c 07" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 0a 00 00 00"/>
<Platform use="optional" type="UTF16String" key="3c 08" globalKey="06 0e 2b 34 01 01 01 02 05 20 07 01 06 01 00 00"/>
</Identification>

19. How to put metadata in MXF?
• On top of this raw KLV structure MXF defines
a conceptual model
– Metadata sets can reference other sets
– Metadata set can be
constructed as trees or graphs

20. How to put metadata in MXF?
• Don’t just throw the metadata in there
– Link the descriptive metadata
to the structural metadata
The DM Framework hooks
custom descriptive
metadata to (a part of ) the
MXF structural timeline

21. How to put metadata in MXF?
• KLV encoding + conceptual data model +
timeline model
• MXF has a proper way of structuring
metadata natively
– Whenever we can, we should use it!

22. Contents
• Metadata exchanges?
• Material Exchange Format
• How to put metadata in MXF?
• Putting EBUCore into MXF
• EBU MXF SDK
• EBU MXF SDK – Demo
• EBU MXF SDK – Improvements & Community

23. Putting EBUCore into MXF
• A case study in embedding MXF metadata and
with practical merit
– Embed an EBUCore XML document in MXF file
using a proper KLV mapping and encoding
– Build software that handles
MXF container and
its embedded
ITM504483
EBUCore metadata _EBUcore.
XML
as a whole

24. EBU MXF SDK
• Initiative by EBU to provide community
with reliable and open-source
software to manipulate MXF files
– Handle metadata
– Handle essence and work on
conformity of MXF files
– Available as free and open source software
• First repository: https://github.com/Limecraft/bmx

25. EBU MXF SDK
• Based on BBC libraries
libMXF, libMXF++ and bmx
– Reference implementations for MXF app. specs.
• AMWA AS-02 (Versioning) and
• AMWA AS-11 (MXF for Contribution)
– Written in C/C++
• Milestone #1
– Mux and de-mux EBUCore metadata
in MXF containers

26. EBU MXF SDK – Demuxing/Reading
Compressed
Compressed
Read Compressed
Audio Decode Audio
Indices Audio
Audio Audio
Samples Samples
Samples
Samples Playback
Read
Partitions / Compressed
Structural
Read Essence Compressed
Video
Compressed Decode Video
Metadata Video Video
Frames
Video Frames
Frames
Frames Metadata
FFmpeg Display/
Read
Descriptive
(libavcodec) Analysis
Metadata
Metadata EBU
MXF SDK (EBU Core) Player/GUI
MXF Analysis
Report
SMPTE 434 XML
Representation (Files)

27. EBU MXF SDK – Wrapping/EBUCore
EBUCore EBUCore
XML Schema KLV Dictionary
XSD Code gen_classes
Generator Code Generator
Schema-based Bi- KLV-based
EBUCore
EBUCore directional Compressed
EBUCore
Compressed
Parser
object model mapping Audio
object model
Audio
EBUCore Samples
Samples
Document
Structural
Essence Compressed
Metadata
Compressed
Processor Video MUX
Essence Video
Frames
Frames
Other
<<Other
KLV-based
metadata
parsers>> metadata
object models
MXF SDK

28. EBU MXF SDK – Demo
EBU Core
KLV Dictionary
Standard MXF
ITM504483 KLV Dictionary
_EBUcore.
XML
MPEG-2 LG MXF SDK mxflib
video.raw
PCM Audio
raw2bmx mxfdump
a0.raw
MXF Dump
PCM Audio Report
a1.raw
PCM Audio
a2.raw playout
PCM Audio vlc
a3.raw

29. Contents
• Metadata exchanges?
• Material Exchange Format
• Metadata in MXF
• Putting EBUCore into MXF
• EBU MXF SDK
• EBU MXF SDK – Demo
• EBU MXF SDK – Improvements & Community

30. EBU MXF SDK – Improvements
• Input from the metadata community is
welcomed and encouraged
• Experiment, use, and improve the existing
code
– Instructions on how to use, compile and embed in
the coming weeks

31. MXF SDK – Metadata Improvements
• Implementation of 377-1 revision changes
– Extension of Descriptive Metadata elements with
grouping elements to aid identification of DM
elements
• Foreign objects can be identified, but what
about their semantics??
– Can we store the dictionary in the MXF container?
• Solution in proposed
SMPTE 377-2 – KLV Encoded Extension Syntax

32. Conclusions
• In B2B exchanges of AV material, embedded
metadata makes sense
• In MXF files custom descriptive metadata,
modeled as a graph of metadata set classes, can
be embedded
• As an example, we illustrated how we can map
EBUCore to a KLV-encoded representation
• This reference implementation SDK, along with
other MXF manipulation tools, is being released
as open source software by EBU

33. Questions? Contact Us!
• http://www.limecraft.com
• http://twitter.com/limecraft
• dieter.vanrijsselbergen@limecraft.com