Exploiting layer and spatial correlations to enhance SVC and tile based 360-degree video streaming

Abstract Streaming 360-degree videos over today’s Internet is challenging due to its high bandwidth and low Motion-to-Photon latency requirements. Viewport adaptive streaming is a promising technique to cope with these challenges by allocating more bitrates to a user’s constrained field of view and streaming appropriate representations based on the user’s estimated throughput and fixation point. A typical dynamic adaptive streaming system uses a Media Presentation Description (MPD) file to describe the sustainable bitrates of a period for different quality representations. However, video bitrates vary significantly along the temporal dimension for Variable Bitrate (VBR) videos. In addition to the throughput overestimate/underestimate problem, video bitrate overestimate/underestimate also affects the efficiency of Adaptive Bitrate (ABR) algorithms. In this paper, we first discovered that for SVC and tile-based encoding, strong video chunk size correlation can be observed on both the layer and spatial dimensions for 360-degree videos. We then proposed to enhance the 360-degree video scheduling framework with a chunk size predictor that predicts the size of: (1) a future enhancement layer video chunk based on the corresponding base layer chunk of the same tiled segment; (2) future base layer video chunks based on the horizontally adjacent base layer video chunks. Experiments showed that ABR algorithms augmented by a chunk size predictor can significantly improve a user’s perceived video playback quality.