2. YOU
CAN'T
WATCH
A
MOVIE
BY
THE
POOL…CAN
YOU?
Amazon Kindle “Poolside” commercial Fall 2010
2
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
2
3. THE
CHALLENGE
OF
THE
MOBILE
VIEWING
ENVIRONMENT
Difficult
for
emissive
display
to
compete
with
sun’s
brightness!
O[en
need
to
reduce
display
brightness
to
conserve
power
3
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
3
4. PROBLEM
STATEMENT
Display
contrast
ra]o
is
affected
independently
from
the
bright
and
dark
ends
of
the
range
Display
contrast
ra]o
must
be
adapted
independently
Full
brightness
Dark
room
4
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
Reduced
brightness
Dark
room
Reduced
brightness
Bright
room
4
5. CONTRAST
RATIO
CHANGES
FROM
1000:1
TO
50:1
WITH
AMBIENT
LIGHT
Source:
hap://www.displaymate.com/Tablet_ShootOut_3.htm
5
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
5
6. SOME
CONTENT
IS
HDR
WHILE
DISPLAY
IS
VERY
LDR
6
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
6
7. THIS
ALSO
MEANS
THAT
DISPLAYS
NEED
TO
BE
BRIGHT
AND
BURN
POWER
Windows
8
recommended
Excessive
brightness:
wastes
power
Display
brightness
(nits)
160
140
120
100
Not
bright
enough
to
see
images/
movies/
games
80
60
40
20
0
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Ambient
light
(lux)
From
embedded
Ambient
Light
sensor
(ALS)
7
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
7
8. HOW
ABOUT
ADJUSTING
GAMMA?
8
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
8
9. YOU
CAN’T
CHANGE
GAMMA
WITHOUT
CHANGING
COLORS
Increasing
gamma
makes
image
less
colorful
and
reduces
contrast
–”wash
out”
9
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
9
10. ASSERTIVE
DISPLAY
Asser]ve
Display
Core
ALS
AD_CALC
VIDEO
PIPELINE
RGB
10
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
iridix_display
gamma
LUT
perceptual
dither
RGB
10
11. ASSERTIVE
DISPLAY
Content
adapted
to
display
contrast
ra]o
Same
power
consump]on
Display
at
max
brightness:
contrast
ra]o
too
low
for
mul]media
Sample
condi]ons:
Ambient
light
1400
lux
–indoors
in
daylight
11
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
11
12. PLAY
AN
IMMERSIVE
GAME
IN
ANY
CONDITIONS
Asser]ve
Display:
Content
adapted
to
display
contrast
ra]o
Same
power
consump]on
Display
at
max
brightness:
contrast
ra]o
too
low
for
mul]media
Sample
condi]ons:
Ambient
light
1400
lux
–indoors
in
daylight
12
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
12
13. THE
4
SCREENS
Asser]ve
Display
ON
(No
specific
calibra]on)
Asser]ve
Display
OFF
Ambient
light
200
lux.
Smartphone:
full
brightness.
TV:
Standard
Mode
13
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
13
14. WHY
IS
ASSERTIVE
DISPLAY
EFFECTIVE?
The
key
to
Asser]ve
Display
is:
Brightness
can
be
set
according
to
preference
and
power
saving
Contrast
raDo
is
always
correct
for
op]mal
mul]media
viewability
It
enables
independent
adjustment
of
bright
and
dark
regions
of
each
frame
So
it
decouples
brightness
from
contrast
raDo
14
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
15. ASSERTIVE
DISPLAY:
XBOX
CALL
OF
DUTY
15
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
15
16. ASSERTIVE
DISPLAY:
NOKIA
LUMIA
1520
16
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
16
17. IMPLEMENTATION
OF
ASSERTIVE
DISPLAY
ON
A
HETEROGENOUS
PROCESSOR
Headline
results
on
Exynos
5410:
Frame
size
Maximum
frame
rate
(single
core)
CPU
(Neon)
only
1080P
(2MP)
17
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
OpenGL
ES2.0
OpenCL
4fps
28fps
45fps
17
18. VIDEO
PROCESSING
ENGINE:
PIXEL-‐BY-‐PIXEL
TONE
MAPPING
18
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
18
19. VIDEO
PROCESSING
ENGINE:
PIXEL-‐BY-‐PIXEL
TONE
MAPPING
Contrast
in
highlights
enhanced
Contrast
in
midtones
preserved
Contrast
in
shadows
enhanced
iridix
calculates
a
unique
tone
curve
for
each
pixel
of
each
frame
19
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
19
20. ASSERTIVE
DISPLAY
INVOLVES
A
GPU-‐FRIENDLY
IMAGE
TRANSFORM
y Use
of
GPU
effec]ve
since
many
pixel
transforma]ons
can
be
done
in
parallel
y However,
transform
is
calculated
per-‐pixel
adap]vely
on-‐the-‐fly,
so
beyond
simple
texture
mapping
⇒ Requires
mul]ple
CPU
and
GPU
passes
with
intermediate
data
store
per
frame
⇒ Main
reason
for
going
from
GL
to
CL
20
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
20
22. BENEFITS
OF
OPENCL
1. Intermediate
memory
accesses
With
Open-‐CL
compliant
hardware,
CPU
and
GPU
memory
can
be
shared,
removing
mul]ple
memory
transfers
required
by
the
algorithms
2. Intermediate
data
precision
Even
if
input
data
is
8-‐bit,
intermediate
calcula]ons
require
at
least
12
bit
precision.
OpenGL
is
inefficient
for
such
opera]ons
when
mul]ple
processing
stages
are
needed
3. Transforma]on
of
different
color
planes
We
apply
a
different
transform
to
Y,U
and
V
planes.
In
OpenCL
these
can
be
done
by
a
single
kernel,
whereas
in
OpenGL
separate
kernels
are
required
22
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
22
23. PERFORMANCE
COMPARISONS
AD
v3
ImplementaDon
Maximum
pixel
rate
Power
consumpDon
at
1080P30
Full
HW
IP
core
(28nm
TSMC
LP)
4K2K
at
60fps
3mW
CPU
only
(Exynos
5410)
1080P
at
4fps
N/A
OpenGL
ES2.0
(Exynos
5410)
1080P
at
28fps
112mW
OpenCL
(Exynos
5410)
1080P
at
45fps
78mW
y GPU-‐based
processing
is
much
beaer
than
CPU,
but
dedicated
hardware
is
s]ll
best
‒ Partly
because
HW
core
does
not
require
system
memory
access
23
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL
23
24. CONCLUSIONS
y An
immersive
experience
can
be
achieved
on
normal
displays
in
all
condi]ons
by
means
of
advanced
image
processing
y Latest
processors
with
CPU/GPUs,
including
those
in
PS4
and
Xbox
One,
can
run
these
kind
of
algorithms
in
real-‐]me
y Extensions
of
this
approach
to
deliver
cinema-‐like
experiences
for
both
video
and
gaming
are
in
development
24
|
PRESENTATION
TITLE
|
NOVEMBER
14,
2013
|
CONFIDENTIAL