Holography is able to reconstruct a three dimensional structure of an object by recording full wave fields of light emitted from the object. This requires a huge amount of data to be encoded, stored, transmitted, and decoded for holographic content, making it a practical usage challenging, specifically for bandwidth-constrained networks and memory-limited devices. In the delivery of holographic content via the internet, bandwidth wastage should be avoided to tackle high bandwidth demands of holography streaming. For real-time applications, encoding time-complexity is also a major problem. In this paper, the concept of Dynamic Adaptive Streaming over HTTP (DASH) is extended to holography image streaming and view-aware adaptation techniques are studied. As each area of a hologram contains information of a specific view and instead of encoding and decoding the entire hologram, just the part required to render the selected view is encoded and transmitted via the network based on the users’ interactivity. Four different strategies, namely, (i) monolithic, (ii) single view, (iii) adaptive view, and (iv) non-real time streaming are explained and compared in terms of (a) bandwidth requirements, (b) encoding time-complexity, and (c) bitrate overhead. Experimental results show that the view-aware methods reduce the required bandwidth for holography streaming at the cost of a bitrate increase.
Powerful Google developer tools for immediate impact! (2023-24 C)
Towards View-Aware Adaptive Streaming of Holographic Content
1. Towards View-Aware Adaptive Streaming of Holographic Content
Hadi Amipour1, Christian Timmerer1,2, and Mohammad Ghanbari1,3
1Alpen-Adria-Universität Klagenfurt, Klagenfurt, Austria
2Bitmovin, Klagenfurt, Austria
3School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK
This research has been supported in part by the
Christian Doppler Laboratory ATHENA: https://athena.itec.aau.at/
2. Challenges for digital holographic video display systems
2
01
Displays
Very premature and heterogeneouse in design
No established standard how to supply holographic data to the display
02
Recording
Recording at high resolution is difficult
Require expertise to build and operate
03
CGH More calculation-intensive than classical image rendering
04
Coding New transform is needed for digital holograms
05
QoE Accurate model is required for modelling perceptual visual quality
3. Workflow for end-to-end hologram delivery
3Signal processing challenges for digital holographic video display systems
4. 4
Dataset
The dataset consists of diffuse holograms generated from 3D point clouds
Interfere-II
Resolution: 8192 x 8192
Pixel pitch: 1 um
Wavelength: 633 nm
Field of view: 370°
Full parallax
5. 5
Compression
Each hologram is stored as a matrix that
contains complex numbers.
In order to encode each hologram in the
hologram plane, each raw hologram is divided
into two parts, real and imaginary.
Both real and imaginary parts are encoded
using an ordinary image/video encoder.
6. 6
Viewports
Each hologram contains information of all views of an object
To render each requested view, the corresponding area of that
view in the hologram is extracted
8. 8
Monilithic
streaming
The entire hologram is sent. The delivery of out of viewport areas of a
holographic content leads to bandwidth wastage
Increased encoding/decoding time-complexity
Best user interactivity streaming
HTTPSeg1Seg1
Highest bitrateLowest bitrate
HTTP Server Client
Select viewDecoder
9. 9
Single view
streaming
One view is requested and the corresponding segment is transmitted
Highest possible bandwidth reduction
Reduces encoding/decoding time-complexity
It is impractical in user interactive hologram
10. 10
Single view
streaming
One view is requested and the corresponding segment is transmitted
Highest possible bandwidth reduction
Reduces encoding/decoding time-complexity
It is impractical in user interactive hologram
HTTPSeg_1Seg_1
Highest bitrateLowest bitrate
HTTP Server Client
Select view
Decoder
Seg_2
Seg_N
Seg_2
Seg_N
11. 11
Adaptive view
streaming
Each partition of holograms is extended to a larger partition
Increases the user experience
HTTPSeg_1Seg_1
Highest bitrateLowest bitrate
HTTP Server Client
Select view
Decoder
Seg_2
Seg_N
Seg_2
Seg_N
d
d = 256 512x512 views
12. 12
Adaptive view
streaming
Each partition of holograms is extended to a larger partition
Increases the user experience
d
d = 256 512x512 views
18. Conclusion
18
01
Monolithic
streaming
The entire hologram is encoded and transmitted
Requires the highest bandwidth and encoding/decoding time complexity
All views are available in the client side
02
Single view
streaming
Only one view is transmitted
Requires the lowest bandwidth and encoding/decoding time complexity
Impractical in user interactive display systems
03
Adaptive view
streaming
In addition to the requested viewport, its neighboring views are transmitted
Efficient in terms of bandwidth consumption
A trade-off is established between user interactivity and bandwidth
consumption
04
Non-real time
streaming
The overall bitrate increases compared to the monolithic streaming
The storage space is decreased compared to the single/adaptive view
streaming