1. MULTIMEDIA DATABASES
AND MPEG7
Rahmi Volkan Başar
Department of Computer Engineering
METU
May, 2013

2. Multimedia Databases
• Introduction
• Capabilities of DB Types
• Search on MMDB
• Multimedia Content Description
• Research Fields

3. Multimedia Data
• Text: using a standard language (SGML, HTML)
• Graphics: encoded in CGM, postscript
• Images: bitmap, JPEG, MPEG
• Video: sequenced image data at specified
rates
• Audio: recordings in a string of bits in digitized
form

4. Database vs Multimedia Database
• Databases
– well structured data organization
– efficient storage of large amounts of data
– querying
– transactional support for concurrent users
– numbers, strings
• Multimedia Databases
– large content
– different structures
– not easily searched/queried

5. Use Cases
• Repositories: central location for data
maintained by DBMS, organized in storage
levels
• Presentations: delivery of audio and video
data, temporarily stored, ‘VCR-like
functionality’
• Collaborative: complex design, analyzing data

6. Capabilities
• Relational Databases
– Atomic / Tables
– Data relation – Common Foreign Keys
– Record: Content – No meta information
– A predefined set of domains for columns
• Hard to extend
• BLOB data type exist

7. Capabilities
• Object Oriented Databases
– Schema is “Class”
– All data is “Object”
– References
– New data types
• Easy. New class is a new data type.
– Appropriate for multimedia data

8. Capabilities
• Object Relational Databases
– In addition to RDBMS
• Object references
• New types
– Multimedia
– MMDBMS
• Extensible ORDBMSs

9. Search
• Collection of data. How to search?
– Any standards?
– Workarounds?
• Search: Retrieve similar images…
– Fast, Correct
• Content-based
– New techniques?

10. Search
• Content Based Retrieval Facilities
– Supported by MMDBMS
• Organize and Manage accordingly
– Compare based on a number of features
• Shape/Color/Texture
• Meta-Data?
– Always.

11. Content Based Retrieval
• Accurate representation of the multimedia
objects in the database
– For accuracy and efficiency
– Combination: Different features
• Similarity Search
– High-dimensional feature vectors
• Special multi-dimensional indexing structures
• Dimension reduction methods.

12. Multimedia Content Description
Standard: MPEG-7
• Influential XML based multimedia meta-data standard
• Description of the storage media:
– Format, Image Size, Audio Quality, Video Frames etc.
• Creation and production information:
– Creation date and location, title, genre, etc.
• Content semantic description:
– Events, concepts, objects, etc.
• Content structural description:
– Shot and key frames with color, texture and motion
features, etc.
• Metadata about the description:
– Author, version, creation date, etc.

13. MPEG-7
• Expression of multimedia data
• Missing: Search for Implicit Data
– The meaning of the structure: Not expressed
– Ex. A video: length, format, name, dates etc.
• Gender: Documentary, Interview, Movie
• Theme: Science, Sports, Horror
• No consideration on search engines

14. MPEG-7
• Search:
– XPath, XQuery
– Semantic Views Query Language

15. Simple MPEG7 Example
<Mpeg7>
<Description xsi:type="SemanticDescriptionType">
<Semantics>
<Label>
<Name> Car </Name>
</Label>
<Definition>
<FreeTextAnnotation>
Four wheel motorized vehicle
</FreeTextAnnotation>
</Definition>
<MediaOccurrence>
<MediaLocator>
<MediaUri> image.jpg </MediaUri>
</MediaLocator>
</MediaOccurrence>
</Semantics>
</Description>
</Mpeg7>

16. MPEG7 Details
• Standardizes 3 parts:
– Description tools
• Descriptors (D)
• Description Schemes (DS).
– Description Definition Language (DDL)
• To specify these schemes
– System tools

17. MPEG7 Details
• Descriptors (D)
– Representation of a feature
• Syntactic and Semantic
– Low-level audio or visual features
• Color, motion, texture etc
– Audiovisual content
• Location, time etc
• Objects can be described
– Several descriptors.

18. MPEG7 Details
• Description Schemes (DS) describe
– Specification of the relations
• Between Descriptors
• Between Description Schemes
– Relations can be structural and semantics
– High-level audiovisual (AV) features
• Regions, segments, events etc

19. MPEG7 Details
• Description Definition Language
– Based on XML
• Defines the structural relations between descriptors
– Creation and modification of description schemes
– Creation of new descriptors.

20. MPEG7 Details
• System Tools
– Deal with Descriptor management
• Binarization
• Synchronization
• Transport
• Storage

21. MPEG7 Details - Overview

22. MPEG7 Details
• Next Slide
– Description of a Video Segment

24. MPEG7 Details
• How to extract semantics?
– i.e. Intelligent Information Retrieval
– Drawback of the standard
– Ontology help required:
• Domain Specific Ontology (Football, Location)
• Automatically extract information
• Use for a better search result

25. Research Fields
• Design: still in research
• Queries: techniques need to be modified
• Rest:
– Modeling: complex objects, wide range of types
– Storage: representation, compression, buffering
during I/O, mapping
– Performance: physical limitations, parallel
processing

26. • Thank you!
• Questions?

27. References
• Wikipedia: Various Pages
• Computer Science and Engineering Department
Resources:
– University of Notre Dame
– Northumbria University
– Carnegie Mellon University
– Boston College
– Simon Fraser University
– Georgia Institute of Technology
• Interview with A. Anil Sinaci