Algorithmen zum Verkleinern von digitalen Videos

Algorithmen zum Verkleinern von digitalen Videos

Wolfgang Effelsberg
Praktische Informatik IV
Universität Mannheim
Germany

Content

1. Motivation
2. Scaling und Cropping
3. Seam Carving for Still Images
4. An Improvement for Diagonal Lines
5. Seam Carving for Video
6. Conclusion and Outlook

Algorithmen zum Verkleinern Klagenfurt
Wolfgang Effelsberg 2
von digitalen Videos 23. Juli 2012

1. Motivation

• Mobile devices (smart phones, pad computers) are getting very popular for
showing videos.
• The representation of still images and videos on those devices requires an
adaptation to the screen size and the aspect ratio.
• Scaling and Cropping are easy to implement but they do not work very
well.
• Seam Carving is a promising technology for still images and videos.


2. Scaling and Cropping

Scaling
Scaling reduces the image linearly, without looking at its content.
„Letterboxing“ is used if the format does not fit on the screen.

Example:


Cropping
Cropping cuts the image parallel to the edges until the final format is reached,
without looking at the image content. Parts of the image can get lost.

Example:


3. Seam Carving for Still Images

Removing of specific pixels from the image, based on the importance of the
content
Based on an „energy function“ describing the importance of different parts of the
content


Example

Reduce the width by 40 %
Energy is maximal on edge pixels in the image.

original image energy image


Naive Approach

Remove from each line the pixels with minimal energy

original image 200 pixels removed


Remove Entire Columns

Remove the 200 columns with the smallest energy

original image 200 columns with minimal
energy removed


Definition of a Seam

• A vertical seam is an 8-connected path of pixels, running from the top to the bottom
of the image and containing exactly one pixel per row.
• Horizontal seams are defined analogously.


Advantage of Seams

Can be adapted better to the regions of interest (regions of high energy)


Seams for our Example

200 vertical seams with minimal energy removed

original image 200 vertical seams removed


Our Other Example

It os obvious that seam carving removes unimportant content first.


Energy Functions (1)

A simple energy function based on the difference in light intensity between two pixels

δ δ
e( I ( x, y )) = I ( x, y ) + I ( x, y )
δx δy

Better (but more complicated) energy functions are possible, for example, based on
important objects found in the image.


Energy Functions (2)

We use saliency and face recognition to identify regions of interest

δ δ
esal + face ( x, y ) = ws ⋅ saliency ( x, y ) + w f ⋅ face( x, y ) + I ( x, y ) + I ( x, y )
δx δy

original image saliency map face map seams with esal+face new
as energy function image


4. An Improvement for Diagonal Lines

A problem with traditional seam carving: diagonal lines

original image width reduced to 40% by
seam carving


The Problem with Diagonal Lines (1)

When a seam crosses a diagonal line visible artefacts appear.

a diagonal line seams cutting that line seams removed


The Problem with Diagonal Lines (2)

Especially critical if several seams cross the line at the same place.
Idea: distribute the seams more evenly over the line.

Neigboring seams line after removing distribute seams evenly line after removing
cutting a line the seams the seams


Solution for Diagonal Lines (1)

Solution: modify the energy function between the computation of the seams: increase
the energy at intersection points

seam cutting a line modified energy energy function third seam
function and new modified again
seam


Solution for Diagonal Lines (2)

Result

original image traditional seam carving improved seam carving


5. Seam Carving for Video
First idea: apply seam carving for each frame separately
 the video gets shaky.

original video seam carved for each frame separately


Seam Carving for Video: First Idea
The video frames define a 3D block over time.
We remove 2D “seam surfaces”. The seam pixels of one surface are connected in
the third dimension.
We use a graph algorithm (min-cut max-flow) to discover optimal “seam surfaces”.
Problem: takes considerable time to compute.

time


Seam Carving for Videos: Second Idea

Fast seam carving for video
• Accumulate all energy values of all frames in the first frame.
• Compute the optimal seams there.
• Map those seams back to the block of frames.


Example for the Second Idea

seams carved separately fast seam carving
for each frame


Another Example

scaling fast seam carving


6. Conclusions and Outlook

• Seam Carving is a useful technique to reduce image sizes without losing
relevant content.
• Finding a good energy function is a real challenge.
• Improvements are possible for diagonal lines.
• Flicker removal is of critical importance for videos. Fast Seam Carving is
a useful algorithm for that problem.
• Seam carving does not work well if most of the content is relevant for the
viewer.
• Seam Carving can be combined with other techniques to create pleasant
smal-screen versions of video.


Vielen Dank …

… für Ihre Aufmerksamkeit!

Meine Mailadresse ist
effelsberg@informatik.uni-mannheim.de


Algorithmen zum Verkleinern von digitalen Videos

Recommended

Recommended

More Related Content

More from Förderverein Technische Fakultät

More from Förderverein Technische Fakultät (20)

Recently uploaded

Recently uploaded (20)

Algorithmen zum Verkleinern von digitalen Videos