from production format to distribution format

1. compression
from production format to distribution format
erik.luyten@dml.kuleuven.be

2. What's this ?
René Margritte: La Trahison des Images (1928-29)

3. Moving images !?

4. Temporal resolution
2 images/second 5 images/second
12,5 images/second 25 images/second

5. A very small image

6. Spatial resolution
7 x 10 = 70 14 x 20 = 280
28 x 40 = 1120 56 x 80 = 4480
112 x 160 = 17920 224 x 320 = 71680
448 x 640 = 309120

7. graylevel resolution
1 bit/ 2 2 bit/ 4 3 bit/ 8 4 bit/ 16
5 bit/ 32 6 bit/ 64 7 bit/128 8 bit/256

8. Creating colour: mixing light
RGB triangle Red channel Green channel Blue channel
3 x 8 bits 256 red steps 256 green steps 256 blue steps

9. Color resolution
24 bit color pallet 8 bit color pallet 8 bit optimized pallet 4 bit color pallet
4 bit color pallet
244 KB file 28 KB file 28 KB file No “dithering”
16 KB file

10. Video signal(uncompressed)
768 pixels
Each pixel has 3 values: (3 bytes)
25 images/second • Red: 256 steps (1 byte)
• Green: 256 steps (1 byte)
• Blue: 256 steps (1 byte)
Datarate for the RGB video signal:
768(pix) x 576(lines) x 3(byte) x 25(images)=32 MByte/sec(256 Mbit/sec)

11. Video signal (component)
720 pixels
Each pixel has 3 values: (2 bytes)
• Luminace(Y): 256 steps (1 byte)
25 images/second • Cr (R-Y): 128 steps (1/2 byte)
• Cb (B-Y): 128 steps (1/2 byte)
Datarate for the component video signal:
720(pix) x 576(lines) x 2(byte) x 25(images)=20 MByte/sec(160
Mbit/sec)

12. Audio signal
Sampling Rate: 44,1 kHz (CD)
Bit Depth: 16 bit (65.536 steps)
Stereo: x 2
16 bit x 2 x 44.100 = 1.411.200 bps
1,4 Mbit/sec
Production systems normally use a sampling rate of
48 kHz
16 bit x 2 x 48.000 = 1.536.000 bps

13. Video distribution ?
 1 second video: 32 Mbyte (256 Mbit/sec)
 1 minute video: 1,9 GByte
 1 hour video: 111 GByte
Disk Network
 floppy disk = 0,05 seconds  Ethernet 100 Mbit/sec (shared) LAN
1/2.5
 CD-rom = 20 seconds  Ethernet 10 Mbit/sec (shared !) LAN
1/25
 1 Gbyte hard disk = 32 seconds  ISDN lijn ( 2B + D) : 128Kbit/sec
1/2000
 16 Gbyte array = 8 min 30 sec  Modem connection : 56 Kbit/sec
1/4600

14. Compression techniques
Lossless
No information loss
Lossy
(Important) information is lost (f.i. medical applications)
Intraframe (spatial compression)
Compression within a single frame
Interframe (temporal compression)
Compression between successive frames
Symmetric
Coding and decoding equally processor intensive
Asymmetric
Coding is more processor intensive than decoding

15. Lower datastream ?
320x240, 25fps, 24 bit 160x120 pix
43,2 Mbit/sec 10,4 Mbit/sec
12,5fps 8 bit
21,6 Mbit/sec 14,4 Mbit/sec

16. Intraframe compression (in one frame)
Run Length Encoding
•without compression:
one line of 100 (white) pixels
100 x 3 bytes = 300 bytes
100 • RLE compression
pixels
100 identical white pixels in a row
2 bytes: total amount of pixels involved
3 bytes for the colour information
2 + 3 = 5 bytes of info for 100 pixels
• Compression factor of 60:1
without information loss

17. Other Intraframe techniques
Discrete Cosine Transformation
•JPEG, MPEG, DV,…
•Type of Fourier Transformation
•Blocks of 8 x 8 pixels and/or 16 x 16
Wavelet Compression
•MPEG-4 (still image), JPEG2000
•Improving image resolution as more data are received
Vector Quantization
Fractal Compression

18. Interframe compression
Image 124
Image 125

19. Interframe: MPEG (Moving Picture Expert Group)
GOP Group Of Pictures
I-pictures: (Intra pictures) reference image with complete information for one
frame (scene changes, ...)
P-pictures: (Predicted pictures) images are “predicted” using the information in
earlier I or P pictures.
B-pictures: (Bi-directional predicted) pictures are recomposed from information
in earlier or later I or P pictures. B-pictures are never used as reference for other
images.
I B B P B B P B B P B B I
Group Of Pictures (GOP)

20. Movie Formats
File Format:
Specifies how data are arranged in a file
(interleaving video and audio)
Codec:
Specifies how (image) data is compressed
Architecture:
Synchronisation of different media components
(video, audio, language tracks, text,…. tracks)
Control for interactivity, alternate tracks,…

21. Movie Architectures
 QuickTime
 AVI
 Windows Media Player
 Real Media
 Flash ?
 MPEG-4

22. Media platform
platform distribution cost alternates remarks
RTSP alternates QTVR, text, midi, SMIL,..
Player = free production and
Mac CD rom HTTP alternates
QuickTim Windows Download (HTTP)
Pro = €30 distribution
Mixed alternates
e (Linux) Streaming (RTSP)
Server= open
Skip prot.,instant - H.264
srce
on
Player = free RTSP alternates
Windows
Streaming (RTSP) Superpass=$13/ (Surestream only distribution and
Real Mac
Download (HTTP) m techn.) mainly for streaming
Linux Server=payable Dynamic
Windows Streaming ( ) Player=free Production anddistribution
Windows (Mac) Download (HTTP)
RTSP alternates
Server= Version 9 on windows
Media Dynamic
(Linux) CD rom Windows2003 platform
Flash
Windows CD rom
(Spark, Player=free
Mac Download (HTTP)
Server=€4759
? H.264
On2, Linux Streaming
h264)
Player=QT, Industrie standaard
Streaming (RTSP)
Windows Real,…
Download (HTTP) scalable (in the from handheld to HD
MPEG-4 Mac
CD rom
Server=open
future)
srce
Linux
DVD, GSM, …. H.264

23. File Formats
 DV
 MPEG-1
 MPEG-2
 MPEG-4

24. Codec’s
 Picture:
• Cinepak
• Indeo (Intel Indeo)
• H.261, H.263, H.264
• Sorenson
• M-JPEG
• MPEG-1, MPEG-2, MPEG-4
• DV
 Sound:
• IMA
• Qualcomm PureVoice, Qdesign Music,…
• MP3
• AAC (Advanced Audio Coding)
• WMA, RealAudio

25. MPEG-1, MPEG-2
MPEG-4 difference
MPEG-4 vs MPEG-1/2
display
and
video scaling
Decompression
compression
Composition
subtitles Rendering display
and transport and
background scaling
graphics
MPEG-4
display
Scaling, Rendering
Decompression
video Bitmap stream transport
Composition
Compression
subtitles ASCI code transport display
and
background Bitmap image transport
graphics Vector code transport

26. Codec’s and bitrate
Web ?
Distribution codec’s Production codec’s
56 kbit 200 kbit 1 Mbit 9 Mbit 25 Mbit 256 Mbit
uncompr.
MPEG-1
MPEG-2
H-261
H-263
Sorenson
DV
profile 0 profile 1
MPEG-4 MPEG-4
H.264
PIM, GSM, ... HD distribution Studio production

27. Comparing codecs
Cinepack : 4,2 Mbit/sec H-263 : 2 Mbit/sec MPEG 1: 1,8 Mbit/sec
Sorenson : 1,35 Mbit/sec MPEG-4 : 960 Kbit/sec H.264 : 502 Kbit/sec

28. Video to Computer
• Video recording
• Interlaced, 25 frames - 50 fields per second
• Restricted to 576 effective lines (PAL) and 3x4 picture
aspect ratio.
• Computer image
• Non-interlaced, 60 to 80 frames per second
• No restrictions on picture format and aspect ratio
• De-interlacing of video material is desirable.

29. De-interlacing
Field 1
313 Field 2
1
314
2
315
3
623
311
624
312
625
313

30. Delivery methods
 DVD, CD, USB stick,…
 Internet
• Progressive download (HTTP, FTP)
 “file transport” type (TCP protocol)
 No data loss: resent lost packages
 No real-time events
 Complete file must be downloaded to allow random access.
 No special server needed (normal web service over http)
• Streaming media (RTP)
 RTSP transport (UDP protocol)
 Lost data (probably) will not be recovered
 Real-time events possible
 “Random access” possibility (only needed fragments will be downloaded)
 Needs one or more(worldwide proxy delivery) streaming servers

31. (Progressive) download
Web server
http://
http://
www.avnet.kuleuven.be/
www.avnet.kuleuven.be/
staalkaart/2004/video04.mov
staalkaart/2004/video04.html
<object classid="clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B"
codebase="http://www.apple.com/qtactivex/qtplugin.cab"
width="320" height="195">
<param name="loop" value="false"> File : video04.mov
<param name="cache" value="true">
<param name="src" value="video04.mov"> TCP/IP
<param name="autoplay" value="true">
<param name="controller" value="true">
<embed width="320" height="195"
pluginspage="http://www.apple.com/quicktime/download/"
src="video04.mov" type="video/quicktime" controller="true"
autoplay="true" cache="true" loop="false">
</object>

32. (video) podcasting
 Audio or audio/video distribution via internet
 Download file to client computer
 Mostly used for handheld devices
 Broadcast channel ---> Feeds
 Programs ---> Episodes

33. Podcast delivery: URL link
Web server AVNet
•Request for web pqge
•HTML with itpc link
•Request XML data
•XML file + startup iTunes
•Request for episodes
•HTTP download
episodes to iTunes
Client computer

34. RTSP streaming (or RTMP)
Web server Streaming server
http:// Datastream
www.avnet.kuleuven.be/ RTP over UDP
Videotechniek/medatech.html
<object classid="clsid:02BF25D5-8C17-4B23-BC80-D3488ABDDC6B"
width="384" height="304" codebase="http://www.apple.com/qtactivex/qtplugin.cab">
<param name="autoplay" value="true">
<param name="controller" value="true">
<param name="pluginspage" value="http://www.apple.com/quicktime/download/indext.html">
<param name="target" value="myself">
<param name="type" value="video/quicktime">
<param name="src" value="jaarverslag0304_384k.mov">
rtsp://escher.avnet.kuleuven.be/
<param name="HREF"
value="rtsp://escher.avnet.kuleuven.be/mediatechnologie/mediatech_384k.mov">
<embed src="comtechn.mov" width="384" height="304" autoplay="true"
mediatechnologie/mediatech_384k.mov
controller="true" border="0"
pluginspage="http://www.apple.com/quicktime/download/indext.html"
target="myself" type="video/quicktime"
href="rtsp://escher.avnet.kuleuven.be/mediatechnologie/mediatech_384k.mov"></embed>
</object>

35. Streaming - unicast

36. Streaming - multicast

37. HTTP streaming (iPhone, iPad,…)
 Ordinary webserver (Apache 2.2)
• Apple HLS
 Segmented distribution of material. • Microsoft Smooth Streaming
 Material is chopped in 10 sec files • Adobe Flash HTTP Dynamic
 Index file with info on different segments Streaming
 Distributed to client using http protocol
 VOD via h.264 compressed material
 Live streaming possible
 Latency 20-30 sec
 Stream segmenter
#EXTM3U
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:10
#EXTINF:10,
http://media.example.com/segment1.ts
#EXTINF:10,
http://media.example.com/segment2.ts
#EXTINF:10,
http://media.example.com/segment3.ts
#EXT-X-ENDLIST