Principles of multimedia - Introduction

1. Principles Of Multimedia
610905
MCA Semester III
Faculty : Swapna Kolhatkar
MCA Department

2. Unit 1
UNIT I: Introduction 8 Hrs
What is multimedia,
goals and objectives,
characteristics of multimedia presentation,
multimedia application,
multimedia building blocks,
multimedia and internet.

3. Unit 1 ...
Multimedia Architecture:
User interface: GTK +, Qt,
Windows multimedia support,
hardware support,
distributed multimedia application,
streaming technologies,
multimedia database systems,
multimedia authoring tools,
overview of multimedia software tools,
multimedia document architecture (MHEG, SGML,
ODA, OMF, etc.)

4. What is Multimedia
Multimedia means multiple intermediaries
between the source and sink of information
or multiple means by which information is
stored, transmitted, presented or perceived.
The multiple means by which we perceive
information are :
1.Text 3. Sound
2. Images 4. Video

5. Characteristics of Multimedia
Multimedia is any combination of text, graphic,
art, sound, animation and video delivered by
any electronic means.

Multiple media

non linearity

scope of interactivity

integrity

Digital Representation.

6. 1. Multiple Media
Pictures
1. images – captured by camera
2. graphics – hand drawn
Time
1. static – text, images, graphics
2. variable – sound, movies (motion pic
tures or animation)

7. 2. Non Linearity
It is the capability of jumping or navigating
from one point within a presentation to
another point without apparent delay.
Eg. Multimedia based presentation
Non linear eg. - TV shows, motion pictures.

8. 3. Interactivity
Interaction is made possible through a set of
interactivity elements embedded within the
presentation like buttons, menu items or
hyperlinks. It helps in taking user inputs.

9. 4. Digital Representation
Analog – audio and video cassette on
magnetic tapes. Also called sequential
access.
Digital – random access, hard disk, floppy disk
and compact disc.

10. 5. Integrity
Several media types present and playing
continuously, need to be integrated or be a
part of a single entity which is the
presentation.

11. Purpose of Multimedia
To present visualizations of concepts
To present animations
To require interactive participation of
learner

12. Multimedia Application
Home Entertainment
Educational Purpose
Industrial Training
Information kiosks
corporate Presentations
Business
Tourism and travel industry
Electronic shopping
communication and networks.
Medicine
engineering applications
content based storage and retrieval system (CBSR)

13. Multimedia Building Blocks
Text
Sound
Images
Graphics
Animation
Video

14. Multimedia and Internet
The Internet has been home to multimedia -- graphics,
animation, and sound files -- for years and years.
The intertwining of multimedia and the Internet (slowly
and irrevocably) makes sense. People like multimedia
on their computers. It's engaging, entertaining, and
often makes otherwise complex computers a little
easier to use. Certainly it works: Some multimedia
educational CD-ROMs are meant to be enjoyed by
children who are barely old enough to walk. The
Internet, once an ugly, text-only system, certainly
needed some pictures, sound, and point-and-click
simplicity to make it understandable to the masses.

15. This example is only the beginning of how
multimedia is changing the Internet.
Consumer products and services, such as
Mosaic in a Box and America Online, are
making the Net still easier for all to enjoy.
GoogleEarth

16. User interface: GTK +, Qt,
GTK (GIMP Toolkit) is a library for creating graphical user
interfaces. It is licensed using the LGPL (Lesser General
Public License) license, so you can develop open software,
free software, or even commercial non-free software using
GTK without having to spend anything for licenses or
royalties.

17. Hello World in GTK
Now for a program with a widget (a button).
#include <gtk/gtk.h>
static void hello( GtkWidget *widget, gpointer data )
{
g_print ("Hello Worldn");
}
static gboolean delete_event( GtkWidget *widget, GdkEvent
*event, gpointer data )
{
g_print ("delete event occurredn");
return TRUE;
}

18. GTK+ contd..
The GTK+ library itself contains widgets, that is, GUI
components such as #GtkButton or #GtkTextView.
The GtkButton widget can hold any valid child widget. That is
it can hold most any other standard GtkWidget. The most
commonly used child is the GtkLabel.
Syntax : GtkWidget *gtk_label_new( const char *str );

19. Qt+
Widgets are the basic building blocks for graphical user
interface (GUI) applications built with Qt. Each GUI
component (e.g. buttons, labels, text editor) is a widget that
is placed somewhere within a user interface window, or is
displayed as an independent window. Each type of widge is
provided by a subclass of QWidget, which is itself a
subclass of QObject.

20. Distributed Multimedia
Applications
In a true distributed system not only the
hardware but the software also remains
dispersed.
The reasons for distribution of multimedia are
those of traditional data processing objects
plus some new ones.
i.e. Moving and sharing.

21. Moving and sharing...
Moving data to the point of use
Sharing of data and resources.
Eg. An architectural blueprint can be worked
upon by different groups of construction
engineers, dispersed geographically.

22. More Examples

Companies can communicate product
information to customers in format and
location they prefer.

Advertising companies

Medical professionals can collaborate

23. Architecture

Creation of subsystem where the content
is created.

A storage subsystem where the created
content is stored.

A distribution subsystem using which the
content may be distributed to multiple
users.

A management subsystem where the
entire operations are managed.

24. Distributed Multimedia
Applications
Videophone, video conference and multimedia
conference
Video on Demand (VoD)
Movie on Demand (MoD)
Interactive Television (ITV)
Voice over IP (VoIP)

25. Streaming Technologies
Streaming media is media that is consumed
(read, heard, viewed) while it is being
delivered.
It is more a property of the delivery system
than the media itself.
2 Techniques
1. Progressive download
2. Real time streaming.

26. Progressive download
A media file containing audio or video is
downloaded and stored on the clients local
file system. As the file is being downloaded,
the client is able to play back the media file
without having to wait for the file to be
downloaded in its entirety.
Mostly used for short video clips.

27. Real time Streaming
The media file is streamed to the client but is
only played and not stored by the client.
Mostly used for long videos, internet radio and
TV broadcasts.

28. A streaming media system is made of many
interacting technologies. Video cameras and
audio recorders

29. Multimedia Database Systems
Multimedia database systems (MMDBS) can
be defined as collections of media elements
like text, image, audio and video that can be
stored, manipulated and retrieved.

30. 1. Storage and retrieval of
multimedia
a)Textual Notations
MMDB needs to be populated first before
it can be utilized.
For retrieval, text notations were
manually generated by individuals.
Disadvantages :
Subjective nature of manual desc.
More time and effort
All multimedia may not have text desc.

31. 1. Storage and retrieval of
multimedia
b) Low-Level Features
color
shape
loudness
pitch.
c)Semantic features
A Semantic Object (SO) is a collection of
image pixels or audio samples that
corresponds to the projection of a real object
or event in an image, audio, video sequence
eg a cluster of trees, a car moving on a
road.

32. 1. Storage and retrieval of
multimedia
d) Knowledge based Interpretation
Extraction and representation of low level
features and semantic objects is not
sufficient to appropriately handle user
queries because the system still cannot
interpret what the features or objects mean.
e) Queries and searching.
'display all images containing the color red'
'display all audio files containing a laugh'

33. General Architecture of MMDBS
A multimedia database consists of 2 levels of
abstractions
1. physical level consisting of the media
objects (Media DB)
2. logical level consisting of the features
characterizing the media objects.
(Features DB)

34. Basic overview of an MMDBS
Symbolic links
Media
file
Media
DB
Feature Extraction
Feature
Info
Feature
DB

35. Extended view of an MMDBS
Media DB Feature DB
Media
element
Result list
of query
Feature
Extraction
Query
Interface
Semantic Info
Query
Symbolic
Link

36. Prototype Implementations
1. MINOS , MULTOS
2. REDI
3. MIM
4. QBIC
5. OVID
6. CORE
7. VisualSEEk
8. SCORE
9. CHABOT
10. CANDID
11. VISION
12. JACOB
13. FACE etc

37. Multimedia Authoring Tools
Also known as authorware, a program that helps you
write hypertext or multimedia applications. Authoring
tools usually enable you to create a final application
merely by linking together objects, such as a
paragraph of text, an illustration, or a song. By defining
the objects' relationships to each other, and by
sequencing them in an appropriate order, authors
(those who use authoring tools) can produce attractive
and useful graphics applications. Most authoring
systems also support a scripting language for more
sophisticated applications.

38. Examples
Authorware
Formula Multimedia
HyperStudio
iShell 3
MaxMedia
MediaMixer
Opus
Platypus
Revolution

39. Overview of Multimedia Software
Tools
* Digital Audio
* Music Sequencing and Notation
* Image/Graphics Editing
* Image/Graphics Editing
* Animation
* Multimedia Authoring
Ref : http://www.cs.cf.ac.uk/Dave/
Multimedia/node40.html

40. Digital Audio
Macromedia Soundedit -- Edits a variety of
different format audio files, apply a variety of
effects
CoolEdit -- Edits a variety of different format
audio files

41. Music Sequencing and Notation
Cakewalk
Cubase
Logic Audio
Marc of the Unicorn Performer

42. Image/Graphics Editing
Adobe Photoshop
Adobe Premiere
Macromedia Freehand

43. Animation
Avid SoftImage
Animated Gif building packages e.g.
GifBuilder

44. Multimedia Authoring
Macromedia Director
Authorware
Microcosm
Flash : Macromedia
HyperCard

45. Multimedia Document Architecture
Document
A document consists of a set of structural information
using which some content is displayed.
A document is aimed at the perception of a human
and is accessible for computer processing.
A multimedia document is a document which
comprises of information coded in at least one
continuous (time dependent) medium and in one
discrete (time independent) medium.

46. Document Architecture
It specifies the content of a document, the physical
layout of document like pages and frames and
logical layout of the document like paragraph,
sections, headings etc.
For exchanging of documents between different
applications, the documents themselves must
adhere to a standard architecture.

47. Standardized Architectures
SGML (Standard Generalized Markup
Language)
ODA (Open Document Architecture)
specifically for multimedia ....
OMF (Open Media Framework)
MHEG (Multimedia and Hypermedia
Information Coding Expert Group)
DCA (Document Content Architecture)
MODCA (Mixed Object Document Content
Architecture)

48. SGML
It prescribes a standard format for embedding
descriptive markup within a document.
SGML allows you to set up hierarchical
models for each type of document
produced.
It forces each element in the structure (labeled
chapter, title, paragraph) to fit in the logical,
predictable structure of document.
A typical document has 3 layers – structure,
content and style.

49. SGML ...
1. Structure
DTD – (Document Type Definition) which
provides a framework and rules for the
definition of the elements that constitute
a document.
A DTD accompanies a document
wherever it goes.

50. SGML ...
2. Content
Content is the information itself. Creating an SGML document
involves inserting tags around content.
<topic><par>Multimedia is interesting !! </par></topic>
3. Style
2 standard based style sheets :
FOSI – Formatting Output Specification Instance
DSSSL – Document Style Semantics and Specification Language

51. SGML Structures
1. Elements – element names are always case – insensitive. It has 3
parts, start tag, content, end tag [<pre> example </pre>]
2. Attributes – attribute/value pair appear before the final '>'
[<h1 align = 'center'> ................. </h1>]
3. Entities – named part of a marked up document, irrespective of
any structural considerations. When parser encounters any entity
reference, it immediately substitutes the value declared for the
entity name.

52. SGML DTD
Rules are the first stage in the creation of a formal specification for the
structure of an SGML document ie SGML DTD
Minimization Rules determine whether or not start and end tags must be
present in every occurrence of the element concerned.
Content Model for specifying the content in terms of other elements or
using special reserved words.
Occurrence indicators, group connectors and entity definitions together
make the DTD.

53. Example
<! ELEMENT poem - - (title?, stanza+)>
<! ELEMENT title - 0 (#PCDATA)>
- : must be present (start tag is present for both)
0 : may be omitted. (for title, end tag may be omitted)
#PCDATA : valid character data
?, *, + : Occurrence indicators

54. ODA
The main property of ODA is the distinction among content, logical
structure and layout structure. This is in contrast to SGML where only
logical structure and contents are defined.
1. The ODA Architecture – enables compound documents to be
encoded and interchanged electronically between systems
conforming to the standards.
2. Logical, layout and content information – indept of page layout
3. Generic structures – reports as defining class of the document.
4. Style – Layout (figure + caption) and presentation(indentation of
first line)
5. The document profile – author name, create date, keywords etc

55. MHEG
An MHEG document can be distributed over the network or in
CD ROM format, but for presentation only, no editing is
possible.
MHEG is intended to produce electronic multimedia versions
of printed books.
It supports encoding of interactive multimedia presentations
defining a platform independent set of multimedia objects
like buttons, text fields etc. Facility of an embedded
scripting language for event handling is also provided.

56. MHEG ...
It is based on object hierarchy where each object has a set of
components and behaviours.
MH-Object
Behavior
Action
Link
Script
Component
content
Interaction
Descriptor

57. HyTIME (extension of SGML)
It is meta language similar to SGML. The structure allows for:
1. The specification of spatial interrelations between
document elements.
2. The specification of temporal interrelations between
document elements (synchronization and time
dependencies)
3. The specification of the overall logical structure
4. The specification of pointers between elements within or
outside the document.

58. OMF
It is a standard format for the interchange of digital media data
among heterogeneous platforms. The primary emphasis is video
production.
1. Source (analogue) material object represent videotape and film
so that the origin of the data is readily identified.
2. special track types store (SMPTE) time codes for segments of
data.
3. Transitions and effects for overlapping and sequences of
segments are predefined.

59. OMF ...
The OMF provides structures for 3 distinct elements :
1. Digital media data – audio, video and graphics for m only part of the
information.
2. Media Sources – describes the original physical sources of data.
3. Composition – describe the arrangement of sections of sources and how
they are played over time.
OMF interchange keeps sources separate from compositions. More than one
composition can reference sections of the same digital media data, reducing
storage requirements.

60. OMF ...
OMF uses objects called Mobs (media Objects) to uniquely
describe compositions and sources. Each Mob contains an
identifier called a MobID
It uses SMPTE time code for numbering frames.
It also has support for motion control.