Establishment of linkages across GOP boundaries for reverse playback on compressed video

Modern video coding standards are basically designed for forward playback only. Recently, we have developed a number of macroblock(MB)-based techniques to support reverse playback for compressed videos by exploiting the motion relationship between adjacent frames. Nevertheless, the MB-based techniques give no effect on traversing GOP boundaries reversely since no inter-frame prediction takes place between the last frame of one GOP and the first frame of its succeeded GOP. In this paper, we borrow ideas from the SP-picture concept in H.264 to establish linkages across GOP boundaries by encoding the last frame of each GOP as a primary SP-frame as well as its corresponding secondary SP-frame. During reverse playback across GOP boundaries, the secondary SP-frame is decoded by using the I-frame in its succeeded GOP as the motion-compensated frame. We believe that we are the first to consider SP-frames to build the linkage between two GOPs for reverse playback. Our scheme can also be used in any future coding standards that offer the similar coding mechanism of SP-frames. This approach can remarkably mitigate the required decoder complexity over 90% during reverse playback across GOP boundaries, but the straightforward implementation introduces serious drift errors. Therefore, as a further contribution of this paper, a novel scheme is designed to avoid the drift problem. Instead of arranging the primary SP-frame before the I-frame, the proposed scheme allocates various MBs within the GOP to be encoded as the SP-picture type. This completely new and unique arrangement for SP coding in which a primary SP MB and its corresponding secondary SP MB are no longer at the same time instant is specially designed for our MB-based techniques, and can be proved to eliminate the possible drift effect for reverse playback. With this allocation strategy, results from our experimental work show that the inherent GOP discontinuity problem can be avoided without introducing additional drift between forward and reverse playback.     

Dynamic bandwidth allocation and buffer dimensioning under equal segment scheme to support video‐on‐demand services

This paper focuses on the optimization of network bandwidth allocation and buffer dimensioning to transport pre‐stored MPEG video data from source to playback destination across ATM networks. This is one of the most important issues in the support of video‐on‐demand (VoD) service. This paper provides a novel scheme in the dynamic allocation of bandwidth to segments of video using ABR mode. The dynamic bandwidth allocation is based on a new concept, called playback tunnel which is obtained from the traffic characteristics of the pre‐stored MPEG video trace to determine the optimum of transmission bandwidth as well as the buffer capacity to ensure that the playback buffer neither underflows nor overflows. The proposed scheme is tested with real‐life MPEG video traces. The obtained results have shown its significant performance improvement in terms of the capacity of playback buffer, the start‐up playback delay, the size of video segment and the network multiplexing gain. Copyright © 2001 John Wiley & Sons, Ltd.     

Smoothing variable-bit-rate video in an internetwork

The burstiness of compressed video complicates the provisioning of network resources for emerging multimedia services. For stored video applications, the server can smooth the variable-bit-rate stream by transmitting frames into the client playback buffer in advance of each burst. Drawing on prior knowledge of the frame lengths and client buffer size, such bandwidth-smoothing techniques can minimize the peak and variability of the rate requirements while avoiding underflow and overflow of the playback buffer. However, in an internetworking environment, a single service provider typically does not control the entire path from the stored-video server to the client buffer. This paper presents efficient techniques for transmitting variable-bit-rate video across a portion of the route, from an ingress node to an egress node. We develop efficient techniques for minimizing the network bandwidth requirements by characterizing how the peak transmission rate varies as a function of the playback delay and the buffer allocation at the two nodes. We present an efficient algorithm for minimizing both the playback delay and the buffer allocation, subject to a constraint on the peak transmission rate. We then describe how to compute an optimal transmission schedule for a sequence of nodes by solving a collection of independent single-link problems, and show that the optimal resource allocation places all buffers at the ingress and egress nodes. Experiments with motion-JPEG and MPEG traces show the interplay between buffer space, playback delay, and bandwidth requirements for a collection of full-length video traces     

Design of frame dependency for VCR streaming videos

The structure of Group of pictures (GOP) is favorable for GOP-based random-access operations, while inside the GOP, the sequential coding dependency between frames is unfavorable for frame-based random-access or fast-scan operations. In a video streaming system, when some VCR operations such as fast-scan operations are performed, they may cause serious server load or network load due to the speed-up of transmitting frames. In this paper, we formulated the fast-scan and random-access operations of VCR functionality, and proposed a novel frame dependency to eliminate extra server load and minimize requirements of the network bandwidth when performing fast-scan and random-access operations. With the proposed structure, the server does not have to speed up the transmission rate of frames to fulfill the requested playback rate. The number of frames to be transmitted is also significantly reduced, and thus the network load can be decreased.     

A low-cost unrestricted fast playback scheme for video streaming

In this brief, we propose a low cost fast-forward and fast-backward playback scheme for video streaming applications. By reshaping the ordinary linear encoded group-of-pictures (GOP) structure into a hierarchical structure, the transmission overhead caused by frame dependencies can be reduced. The resulting video streaming system can provide all directions video playback with any speed-up factor. A set-top box architecture that supports the binary tree structured GOP playback is also described.     

Robust Video Transmission With Distributed Source Coded Auxiliary Channel

We propose a novel solution to the problem of robust, low-latency video transmission over lossy channels. Predictive video codecs, such as MPEG and H.26x, are very susceptible to prediction mismatch between encoder and decoder or “drift” when there are packet losses. These mismatches lead to a significant degradation in the decoded quality. To address this problem, we propose an auxiliary codec system that sends additional information alongside an MPEG or H.26x compressed video stream to correct for errors in decoded frames and mitigate drift. The proposed system is based on the principles of distributed source coding and uses the (possibly erroneous) MPEG/H.26x decoder reconstruction as side information at the auxiliary decoder. The distributed source coding framework depends upon knowing the statistical dependency (or correlation) between the source and the side information. We propose a recursive algorithm to analytically track the correlation between the original source frame and the erroneous MPEG/H.26x decoded frame. Finally, we propose a rate-distortion optimization scheme to allocate the rate used by the auxiliary encoder among the encoding blocks within a video frame. We implement the proposed system and present extensive simulation results that demonstrate significant gains in performance both visually and objectively (on the order of 2 dB in PSNR over forward error correction based solutions and 1.5 dB in PSNR over intrarefresh based solutions for typical scenarios) under tight latency constraints.      

A New Dual-bitstream Video Streaming System with VCR Functionalities Using SP/SI-frames

With the proliferation of digital video and the popularity of video streaming applications, it is highly desirable to find and access video segments of interest by searching through the content of video at a speed that is faster than a normal playback. The key functions that enable quick browsing of video are fast-forward and fast-reverse playbacks. However, motion-compensated prediction adopted in the current video coding standards drastically complicates these operations. One approach to implement the fast-forward/reverse playback is to store an additional reverse-encoded bitstream in the server. Once the client requests a fast-forward/reverse operation, the server can select an appropriate frame for the client from either the forward-encoded bitstream or the reverse-encoded bitstream by considering the cost of network bandwidth and decoder complexity. Unfortunately, these two bitstreams are encoded separately. The frame in one bitstream may not be exactly identical to the frame in another bitstream. If one of these frames is then used as the reference for the requested frame, which is in another bitstream, it induces mismatch errors. In this paper, a novel H.264 dual-bitstream system aiming at providing the fast-forward/reverse playback based on SP/SI-frames is proposed. The proposed system can completely eliminate mismatch errors when the frame in the reverse-encoded bitstream replaces the frame in the forward-encoded bitstream and vice versa. Experimental results confirm that the proposed system is effective in eliminating mismatch errors so as to enhance the performance of the dual-bitstream system.     

TRASS: A transmission rate‐adapted streaming server in a wireless environment

The video streaming quality in a wireless communication network environment is largely affected by various network characteristics, such as a limited channel bandwidth and a variant transmission rate. The playback quality of User Equipments (UEs) may not be smooth when the service is delivered via a wireless environment. From the viewpoints of most video receivers, a smooth playback with a lower video quality may be more significant than a lagged or distorted playback with a higher video quality as the transmission rate degrades. Based on the above, we sketch an adaptation agent—Transmission‐Rate Adapted Streaming Server (TRASS), which is located between the original video server and UEs, to adaptively transform the streaming video based on the real transmission rate. In our proposed scheme, UEs would feedback their network access statuses to TRASS and then TRASS would deliver adaptive quality of video streams to UEs according to their feedbacks. The theoretical analysis and simulations using different video tracks encoded in MPEG‐4 and H.264/AVC formats show that TRASS can help wireless streaming users to get a smooth playback quality with a lower packet failure rate. With a low probability of receiving a worse quality of video, users' Quality of Experience can subsequently be raised. Copyright © 2010 John Wiley & Sons, Ltd.     

Error-resilient coding of H.264 based on periodic macroblock

For the compressed video, since an inter-frame depends on the previously encoded frame, the error in one inter-frame may propagate to the following inter-frames. In this paper, we present a new error-resilient coding scheme to alleviate the effect of error propagation for the new coding standard H.264. In this new coding standard, multiple reference frame is adopted to improve the coding efficiency. By making use of the reference frame buffer in the encoder, we can reference some macroblocks in every n/sup th/ inter-frame to the frame that is n frames interval away, and these macroblocks are named as periodic macroblocks. The periodic macroblock can efficiently alleviate the error propagation between two frames that contain periodic macroblocks. We prove it in theory that encoding selected periodic macroblocks will reduce the loss probability of pixel. The selection of periodic macroblock is based on the distortion expectation of each macroblock in every n/sup th/ frame. The number of periodic macroblocks in every n/sup th/ frame can be adjusted according to the available transmission bandwidth, as the periodic macroblock will consume little more bits. The simulation results prove that the periodic macroblocks can obviously improve the quality of video at different macroblock loss rates. When the macroblock lost rate is 15% in every frame, the PSNR of video sequence can be improved about 3dB with 5% bitrate increase.     

Scalable On-Demand Streaming of Nonlinear Media

A conventional video file contains a single temporally-ordered sequence of video frames. Clients requesting on-demand streaming of such a file receive (all or intervals of) the same content. For popular files that receive many requests during a file playback time, scalable streaming protocols based on multicast or broadcast have been devised. Such protocols require server and network bandwidth that grow much slower than linearly with the file request rate. This paper considers ldquononlinearrdquo video content in which there are parallel sequences of frames. Clients dynamically select which branch of the video they wish to follow, sufficiently ahead of each branch point so as to allow the video to be delivered without jitter. An example might be ldquochoose-your-own-endingrdquo movies. With traditional scalable delivery architectures such as movie theaters or TV broadcasting, such personalization of the delivered video content is very difficult or impossible. It becomes feasible, in principle at least, when the video is streamed to individual clients over a network. For on-demand streaming of nonlinear media, this paper analyzes the minimal server bandwidth requirements, and proposes and evaluates practical scalable delivery protocols.     

Variable segmentation based on intrinsic video rate characteristics to transport pre‐stored video across networks

A variable segmentation scheme based on intrinsic video rate characteristics (VSIVRC) is proposed for efficient dynamic bandwidth allocation to transport pre‐stored variable bit rate (VBR) video across networks. The proposed scheme is evaluated with a set of real‐life MPEG video traces and also compared with two fixed segmentation schemes: equal data segmentation (EDS) and equal frame segmentation (EFS). The obtained results show that the proposed VSIVRC scheme significantly outperforms both EDS and EFS in terms of range of selective bandwidth, matchness between arrival stream and playback stream, buffer occupancy and bandwidth allocation as well. Copyright © 2003 John Wiley & Sons, Ltd.     

New video discarding policies for improving UDP performance over wired/wireless networks

Multimedia communication over wired and wireless networks becomes a compulsory need for many recent applications. To effectively react to the tremendous demand of video streaming over the Internet, videos are usually compressed by utilizing spatial and temporal redundancy. It is noteworthy to mention that compressing videos may degrade their quality if it is not investigated properly. In other words, as a consequence of exploiting redundancies, frame dependencies emanate, which make discarding frames, because of occupying the whole capacity of network elements, have severe implications on the video quality. Furthermore, transmitting videos over capacity‐limited links owing to error‐prone channels, power constraints and bandwidth variations will severely affect the video quality. Additionally, as the current coding schemes are characterized by being able to afford high compression efficiency, sensitivity to packet losses becomes untolerated. Therefore, insuring the perceived quality of the delivered videos to be always high in spite of aforementioned challenges is the primary focus of current researchers. In this paper, we propose efficient and novel video discarding policies that mainly aim to reduce the number of frames being lost through substitution of those frames that are very difficult or even impossible to decode at the receiver side. This is accomplished by controlling and maintaining the buffer occupancy of network elements. Our proposed policies are evaluated in terms of frameput, rate of non‐decodable frames, peak signal‐to‐noise ratio, structural similarity index and average buffer occupancy. Our proposed policies behave very well and achieve a remarkable enhancement over what is closely connected in the literature. Copyright © 2015 John Wiley & Sons, Ltd.     

基于GOP取帧与变帧率的VCR实现方法

 MPEG编码视频帧间存在解码依赖性,导致VCR操作尤其是快进快退,需要服务端传送大量解码依赖帧.目前的VCR实现方法在解决这种依赖传输现象时,系统资源消耗与机顶盒解码复杂度非常高.本文提出一种基于GOP取帧与变帧率的VCR实现方法,基于GOP取帧消除依赖传输从而降低快进快退码率,可降到正常播的22%.变帧率实现码率可定制的效果,满足不同链路需求.该方法无需预处理视频,可作为在线算法应用于实时视频系统.     

MPEG4视频流量预测自适应算法的研究

为了保证用户的服务质量(QOS),宽带分组网在传送视频信息时需要进行动态带宽分配,而视频流量预测在动态带宽分配中发挥着重要的作用。本文从自相关性、自相似性的Hurst参数两个方面,阐明GOP时间尺度上的流量能够体现原始帧序列的流量特性,并在固定步长的LMS自适应算法(FSSA)的基础上提出的一种新的可变步长自适应算法(VSSA),在GOP的大时间尺度上预测MPEG4视频流量,通过大量的仿真实验表明,VSSA算法可以明显地改善预测性能。     

QoE optimization for HTTP adaptive streaming: Performance evaluation of MEC-assisted and client-based methods

Seamless streaming of high quality video under unstable network condition is a big challenge. HTTP adaptive streaming (HAS) provides a solution that adapts the video quality according to the network conditions. Traditionally, HAS algorithm runs at the client side while the clients are unaware of bottlenecks in the radio channel and competing clients. The traditional adaptation strategies do not explicitly coordinate between the clients, servers, and cellular networks. The lack of coordination has been shown to lead to suboptimal user experience. As a response, multi-access edge computing (MEC)-assisted adaptation techniques emerged to take advantage of computing and content storage capabilities in mobile networks. In this study, we investigate the performance of both MEC-assisted and client-side adaptation methods in a multi-client cellular environment. Evaluation and comparison are performed in terms of not only the video rate and dynamics of the playback buffer but also the fairness and bandwidth utilization. We conduct extensive experiments to evaluate the algorithms under varying client, server, dataset, and network settings. Results demonstrate that the MEC-assisted algorithms improve fairness and bandwidth utilization compared to the client-based algorithms for most settings. They also reveal that the buffer-based algorithms achieve significant quality of experience; however, these algorithms perform poorly compared with throughput-based algorithms in protecting the playback buffer under rapidly varying bandwidth fluctuations. In addition, we observe that the preparation of the representation sets affects the performance of the algorithms, as does the playback buffer size and segment duration. Finally, we provide suggestions based on the behaviors of the algorithms in a multi-client environment.     

Guest Editors’ Introduction

In H.264/AVC, a deblocking filter improves visual quality by reducing the presence of blocking artifacts in decoded video frames. The deblocking filter accounts for one third of the computational complexity of the decoder. This paper exploits the scalability on the hardware and the algorithmic level to synergize the performance and to reduce the computational complexity. First, we propose a modular deblocking filter architecture which can be scaled to adapt to the required computing capability for various bit-rates, resolutions, and frame rate of video sequences. The scalable architecture is based on FPGA using dynamic partial reconfiguration. This desirable feature of FPGAs makes it possible for different hardware configurations to be implemented during run-time. The proposed design can be scaled to filter up to four different edges simultaneously, resulting in significant reduction of total processing time. Secondly, our experiments show that significant reduction in computational complexity can be achieved by the increased presence of skipped macroblocks at lower bit-rates, thus, avoiding redundant filtering operations. The implemented architecture is evaluated using the Xilinx Virtex-4 ML410 FPGA board. The design operates at a maximum frequency of 103 MHz. The reconfiguration is done through Internal Configuration Access Port (ICAP) to achieve maximum performance needed by real time applications.     

基于宏块类型信息的快速视频分段算法

提出了一种基于宏块类型信息的快速视频分段算法:对MPEG视频流进行两次分析,第一次(粗略分析)只分析P-帧中宏块统计信息,检测出可能存在的镜头边界,第二次(精确分析)再对粗略分析找出的边界邻近的B-帧和P-帧的宏块类型进行分析,从而对场景变换进行精确分析和定位。实验结果表明,粗略分析可以满足实时检测的速度要求,帧定位误差控制在10帧之内,精确分析可以进一步把帧定位误差控制在2帧之内。     

Bounded‐delay and loss‐free guarantees for real‐time video in integrated services packet networks

In this paper, we analyse upper bounds on the end‐to‐end delay and the required buffer size at the leaky bucket and packet switches within the network in the context of the deterministic bandwidth allocation method in integrated services packet networks. Based on that formulation, we then propose a CAC method suitable to ISPN to guarantee the bounded end‐to‐end delay and loss‐free packet transmissions. As an example application, the GOP–CBR MPEG‐2 is considered. In that case, we also show tighter bounds by slightly modifying the coding method of GOP–CBR MPEG‐2. Using the actual traced data of GOP–CBR MPEG‐2, we discuss the applicabilities of our analytical results and proposed CAC by comparing with simulation. Numerical results show that the loose upper bounds can also achieve more utilization even in the context of deterministic bandwidth allocation compared with the peak bandwidth allocation strategy. Copyright © 2000 John Wiley & Sons, Ltd.     

A heuristic approach of bandwidth management for video sources in ATM networks

One of the most important properties in the ATM network is that the resource of the network, including buffer and bandwidth, can be flexibly managed according to different demands of various applications. The network bandwidth can be effectively allocated and utilized if the data volume of the arrival traffic can be predicted precisely. In this paper, we study the bandwidth management schemes for variable bit rate (VBR) pre‐coded MPEG video sources. The proposed bandwidth allocation method, which predicts the bandwidth by the frame correlation, demonstrates a quite good performance when comparing with a previous scheme, especially for the video scenes with the combination of intraframes and interframes. Bandwidth allocation of a multiplexer connected to several video sources is also studied by using heuristic information. The experimental results show that the proposed method is much better than that of the fixed bandwidth allocation and is suitable for the application of MPEG video services. Copyright © 2000 John Wiley & Sons, Ltd.