The AudioVisual Description Profile a.k.a. ISO/IEC 15938-9:2005/Amd.1 a.k.a. MPEG-7 AVDP profile a.k.a. the EBU MPEG-7 profile Dr. Alberto Messina R&D Area Coordinator Multimedia Information Engineering RAI – Centre for Research and Technological Innovation Turin (ITALY)

1. The AudioVisual Description Profile
a.k.a. ISO/IEC 15938-9:2005/Amd.1
a.k.a. MPEG-7 AVDP profile
a.k.a. the EBU MPEG-7 profile
a.k.a. the ultimate metadata profile
a.k.a. …
Dr. Alberto Messina
R&D Area Coordinator
Multimedia Information Engineering
RAI – Centre for Research and Technological Innovation
Turin (ITALY)
Centro Ricerche e Innovazione Tecnologica

2. Why a new MPEG-7 profile?
Existing profiles insufficient for a number of reasons
Centro Ricerche e Innovazione Tecnologica

3. MPEG-7 - AVDP
Origin
AVDP was originated in EBU in the context of the technical
group MIM/SCAIE concerned with the study of automatic
techniques in media production
Requirements
Target application analysis was the starting point to define AVDP
requirements
Content Summarisation
Text Recognition
Semantic Segmentation
Copy/Repetition detection
Personality Identification
Keywords Extraction
Subject Classification
Centro Ricerche e Innovazione Tecnologica

4. Sources of the work and standardisation
process
JOANNEUM Research’s Detailed AudioVisual Profile
NHK’s Metadata Production Framework
Process
Proposed to MPEG in July 2010
Went through standardisation process in 2011
PDAM
DAM
FDAM
Officialised as a standard in April 2012
Part 11 (Schema) is now at its final stages too
Centro Ricerche e Innovazione Tecnologica

5. AVDP General Requirements
Number Requirement
1 The metadata model must have the ability to identify a feature extraction tool (e.g., name),
version and institute (e.g. name of a company or university/affiliation).
2 The metadata model must have the ability to identify contributors who have participated in the
test and their respective roles.
3 The metadata model should allow results from different extractions being combined if related to
a common timeline event.
4 The metadata model must have provisions for the date and time on which the results of the
feature extraction tool were generated.
5 The metadata model must be able to identify and describe (e.g. “title”, “genre”, “language”) one
or more assets or parts of assets (e.g. using a standard identification format), the associated
type (e.g. MIME type) and location (e.g. URL), to which the feature extraction relates to.
6 The metadata model should have the ability to describe the content on multiple timeline (such as
video and audio timeline).
7 The metadata model should be have the ability to add confidence levels attached to the results
for each feature extracted by any feature extraction tool at the appropriate level of
granularity
Centro Ricerche e Innovazione Tecnologica

6. AVDP semantics
TD : TemporalDecomposition AVS : AudioVisualSegment
Mpeg7 STD: SpatioTemporalDecomposition AS : AudioSegment
MSD : MediaSourceDecomposition VS: VideoSegment
Description type=“ContentEntityType” SD: spatialDecomposition SR : StillRegion
MR : MovingRegion
MultimediaContent type=“AudioVisualType”
AudioVisual
TD AVS AVS AVS ( An experiment/ criteria=shot )
TD AVS AVS AVS ( An experiment/criteria=ASR )
TD AVS AVS AVS ( An experiment/criteria=Face)
( Container)
T
T TD
TD AVS-2nd TD
TD AVS-3rd T
AVS-1st
V V
MSD
MSD AVS
VS STD
STD MR
VideoText
VS A TD VS V
Same duration AS TD VS
T Text
AS
VS-key TD AS A
V Video feature + Text TD AS
A Audio feature + Text AS
AS-key
SR
SR V SD SR
SD V
A frame ImageText
Centro Ricerche e Innovazione Tecnologica

7. Implementation example
Centro Ricerche e Innovazione Tecnologica

8. Conclusions
AVDP is the new standard reference for low level automatically
extracted metadata
Grounded on a thorough requirements analysis made by
experts of the media domain
EBU
Several strategic impacts foreseen
EBU members Internal projects
FIMS (Framework for Interoperable Media Services)
XML Schema of AVDP going to be standardised soon
Guidelines being prepared by MIM/SCAIE about usage of AVDP
Stay tuned!
Centro Ricerche e Innovazione Tecnologica

9. Acknowledgements
Masanori (Masa) Sano (NHK)
Excellent skills and knowledge of the MPEG procedural rules
Continued passion for AVDP
Very nice person
Jean-Pierre Evain (EBU)
Continued support from within EBU Technical
Co-chaired the MPEG AhG with Masa
Dissemination and support of AVDP throughout the world
Werner Bailer (JOANNEUM Research)
Top-level expertise of MPEG-7 technicalities and definitions
All MIM/SCAIE members for having supported AVDP
in its infancy
Centro Ricerche e Innovazione Tecnologica

10. a.messina@rai.it
Dr. Alberto Messina
R&D Area Coordinator
Multimedia Information Engineering
RAI – Centre for Research and Technological Innovation
Turin (ITALY)
Centro Ricerche e Innovazione Tecnologica