Sliding‐window forward error correction using Reed‐Solomon code and unequal error protection for real‐time streaming video

Forward error correction (FEC) techniques are widely used to recover packet losses over unreliable networks in real‐time video streaming applications. Traditional frame‐level FEC encodes 1 video frame in each FEC coding window. By contrast, in the expanding‐window FEC scheme, high‐priority frames are included in the FEC processing of the following frames, so as to construct a larger coding window. In general, expanding‐window FEC improves the recovery performance of FEC, because the high‐priority frame can be protected by multiple windows and the use of a larger coding window increases the efficiency. However, the larger window size also increases the complexity of the coding and the memory space requirements. Consequently, expanding‐window FEC is limited in terms of practical applications. Sliding‐window FEC adopts a fixed window size in order to approximate the performance of the expanding‐window FEC method, but with a reduced complexity. Previous studies on sliding‐window FEC have generally adopted an equal error protection (EEP) mechanism to simplify the analysis. This paper considers the more practical case of an unequal error protection (UEP) strategy. An analytical model is derived for estimating the playable frame rate (PFR) of the proposed sliding‐window FEC scheme with a Reed‐Solomon erasure code for real‐time non‐scalable streaming applications. The analytical model is used to determine the optimal FEC configuration which maximizes the PFR value under given transmission rate constraints. The simulation results show that the proposed sliding‐window scheme achieves almost the same performance as the expanding‐window scheme, but with a significantly lower computational complexity.     

Motion adaptive video coding scheme for time‐varying network

Adaptive video coding algorithms can encode a video stream dynamically on the basis of the amount of bandwidth available in a network. In this paper, a novel approach for adaptive video coding based on look‐up tables has been proposed. On the basis of the network conditions, the proposed codec estimates quantization parameter (QP) and also the spatial resolution for a group of pictures from the look‐up table. Then, QP for each frame is estimated on the basis of the motion content in each frame. More motion leads to more burden of bits while encoding the motion vectors. Also, quantization noise is less prominent in high‐motioned frames than frames possessing low motion information because of the temporal masking phenomenon of the human eyes. So, the main trick of the proposed scheme is to assign higher QP to the frames having higher motion than those having less motion. This method also reduces the requirement of excess bits for encoding of motion vectors having higher motion content. Additionally, QP is adjusted on the basis of the buffer availability in order to avoid bit loss error. Experimental results show the efficacy of the proposed codec. Copyright © 2013 John Wiley & Sons, Ltd.     

分布式压缩视频编码的帧分类重构方法研究

在很多的应用场景中需要具有低复杂度的视频编码器,新兴的分布式视频编码和压缩感知技术正好适用于这些场景中,因而出现了一种新的视频编码方案——分布式压缩视频编码。在现有的一些分布式压缩视频编码方案中,视频帧在编码端是独立编码,在解码端进行联合解码,具体来说就是关键帧独立解码,非关键帧在由关键帧生成的边信息的帮助下进行解码,这就忽略了非关键帧之间的相关性。本文提出一个新的分布式视频编码方案,将非关键帧分为主非关键帧和次非关键帧,主非关键帧利用关键帧生成地边信息进行解码,而次非关键帧先利用相邻的主非关键帧进行观测值预测,然后再利用关键帧生成的边信息进行解码。实验结果表明,在本文提出的框架下,非关键帧的重构质量提高了有2dB~4dB。      

Perceptual-based distributed video coding

In this paper, we propose a perceptual-based distributed video coding (DVC) technique. Unlike traditional video codecs, DVC applies video prediction process at the decoder side using previously received frames. The predicted video frames (i.e., side information) contain prediction errors. The encoder then transmits error-correcting parity bits to the decoder to reconstruct the video frames from side information. However, channel codes based on i.i.d. noise models are not always efficient in correcting video prediction errors. In addition, some of the prediction errors do not cause perceptible visual distortions. From perceptual coding point of view, there is no need to correct such errors. This paper proposes a scheme for the decoder to perform perceptual quality analysis on the predicted side information. The decoder only requests parity bits to correct visually sensitive errors. More importantly, with the proposed technique, key frames can be encoded at higher rates while still maintaining consistent visual quality across the video sequence. As a result, even the objective PSNR measure of the decoded video sequence will increase too. Experimental results show that the proposed technique improves the R-D performance of a transform domain DVC codec both subjectively and objectively. Comparisons with a well-known DVC codec show that the proposed perceptual-based DVC coding scheme is very promising for distributed video coding framework.     

Joint Source/Channel Coding Based on Two‐Dimensional Optimization for Scalable H.264/AVC Video

The scalable extension of the H.264/AVC video coding standard (SVC) demonstrates superb adaptability in video communications. Joint source and channel coding (JSCC) has been shown to be very effective for such scalable video consisting of parts of different significance. In this paper, a new JSCC scheme for SVC transmission over packet loss channels is proposed which performs two‐dimensional optimization on the quality layers of each frame in a rate‐distortion (R‐D) sense as well as on the temporal hierarchical structure of frames under dependency constraints. To compute the end‐to‐end R‐D points of a frame, a novel reduced trellis algorithm is developed with a significant reduction of complexity from the existing Viterbi‐based algorithm. The R‐D points of frames are sorted under the hierarchical dependency constraints and optimal JSCC solution is obtained in terms of the best R‐D performance. Experimental results show that our scheme outperforms the existing scheme of [13] with average quality gains of 0.26 dB and 0.22 dB for progressive and non‐progressive modes respectively.     

Low‐Complexity MPEG‐4 Shape Encoding towards Realtime Object‐Based Applications

Although frame‐based MPEG‐4 video services have been successfully deployed since 2000, MPEG‐4 video coding is now facing great competition in becoming a dominant player in the market. Object‐based coding is one of the key functionalities of MPEG‐4 video coding. Realtime object‐based video encoding is also important for multimedia broadcasting for the near future. Object‐based video services using MPEG‐4 have not yet made a successful debut due to several reasons. One of the critical problems is the coding complexity of object‐based video coding over frame‐based video coding. Since a video object is described with an arbitrary shape, the bitstream contains not only motion and texture data but also shape data. This has introduced additional complexity to the decoder side as well as to the encoder side. In this paper, we have analyzed the current MPEG‐4 video encoding tools and proposed efficient coding technologies that reduce the complexity of the encoder. Using the proposed coding schemes, we have obtained a 56 percent reduction in shape‐coding complexity over the MPEG‐4 video reference software (Microsoft version, 2000 edition).     

Object-based stereo video compression using fractals and shape-adaptive DCT

Based on the classical fractal video compression method, an improved object-based stereo video compression scheme with Shape-Adaptive DCT is proposed in this paper. Firstly, we use more effective macroblock partition scheme instead of classical quadtree partition scheme; thus reducing the block searching strategy. The stereo fractal video coding is proposed which matches the macroblock with two reference frames in left and right view results in increasing compression ratio and reducing bit rate when transmitting compressed stereo data. The stereo codec combines the Motion Compensation Prediction (MCP) and Disparity Compensation Prediction (DCP). Fractal coding is adopted and each object is encoded independently by a prior video segmentation alpha plane, which is defined exactly as in MPEG-4. The testing results with the nature monocular and stereo video sequences provide promising performances at low bit rate coding. We believe it will be a powerful and efficient technique for the object-based monocular and stereo video sequences coding.     

Adaptive Pre‐/Post‐Filters for NRT‐Based Stereoscopic Video Coding

Non‐real‐time delivery of stereoscopic video has been considered as a service scenario for 3DTV to overcome the limited bandwidth in the terrestrial digital television system. A hybrid codec combining MPEG‐2 and H.264/AVC has been suggested for the compression of stereoscopic video for 3DTV. In this paper, we propose a stereoscopic video coding scheme using adaptive pre‐/post‐filters (APPF) to improve the quality of 3D video while retaining compatibility with legacy video coding standards. The APPF are applied adaptively to blocks of various sizes determined by the macroblock coding mode and reference frame index. Experiment results show that the proposed method achieves up to 24.86% bit rate savings relative to a hybrid codec of MPEG‐2 and H.264/AVC including the inter‐view prediction.     

Dynamic Retry Adaptation Scheme to Improve Transmission of H.264 HD Video over 802.11 Peer‐to‐Peer Networks

This paper presents a dynamic retry adaptation scheme for H.264 HD video, called DRAS.264, which dynamically adjusts the retry limits of frames at the medium access control (MAC) layer according to the impact those frames have on the streamed H.264 HD video. DRAS.264 is further improved with a bandwidth estimation technique, better prediction of packet delays, and expanded results covering multi‐slice video. Our study is performed using the Open Evaluation Framework for Video Over Networks as a simulation environment for various congestion scenarios. Results show improvements in average peak signal‐to‐noise ratios of up to 4.45 dB for DRAS.264 in comparison to the default MAC layer operation. Furthermore, the ability of DRAS.264 to prioritize data of H.264 bitstreams reduces error propagation during video playback, leading to noticeable visual improvements.     

Performance modeling of FEC‐based unequal error protection for H.264/AVC video streaming over burst‐loss channels

Unequal error protection systems are a popular technique for video streaming. Forward error correction (FEC) is one of error control techniques to improve the quality of video streaming over lossy channels. Moreover, frame‐level FEC techniques have been proposed for video streaming because of different priority video frames within the transmission rate constraint on a Bernoulli channel. However, various communication and storage systems are likely corrupted by bursts of noise in the current wireless behavior. If the burst losses go beyond the protection capacity of FEC, the efficacy of FEC can be degraded. Therefore, our proposed model allows an assessment of the perceived quality of H.264/AVC video streaming over bursty channels, and is validated by simulation experiments on the NS‐2 network simulator at a given estimate of the packet loss ratio and average burst length. The results suggest a useful reference in designing the FEC scheme for video applications, and as the video coding and channel parameters are given, the proposed model can provide a more accurate evaluation tool for video streaming over bursty channels and help to evaluate the impact of FEC performance on different burst‐loss parameters. Copyright © 2014 John Wiley & Sons, Ltd.     

Improved Method for the Macroblock‐Level Deblocking Scheme

This paper presents a deblocking method for video compression in which the blocking artifacts are effectively extracted and eliminated based on both spatial and frequency domain operations. Firstly, we use a probabilistic approach to analyze the performance of the conventional macroblock‐level deblocking scheme. Then, based on the results of the analysis, an algorithm to reduce the computational complexity is introduced. Experimental results show that the proposed algorithm outperforms the conventional video coding methods in terms of computation complexity while coding efficiency is maintained.     

An improved side information generation for distributed video coding

This paper presents a side information (SI) scheme for distributed video coding based on multilayer perceptron. The suggested scheme predicts a Wyner–Ziv (WZ) frame from two decoded key frames. The network is trained offline using patterns from different standard video sequences with varied motion characteristics to achieve generalization. The proposed scheme is simulated along with other standard video coding schemes. Performance comparisons have been made with respect to training convergence, rate distortion (RD), peak signal to noise ratio (PSNR), number of requests per SI frame, decoding time requirement, etc. In general, it is observed that the proposed scheme has a superior SI frame generation capability as compared to its competent schemes.     

基于边信息的分布式视频压缩感知的残差重构

压缩感知(Compressed Sensing,CS)结合了视频信号的变换和信息压缩过程,为简化编码算法提供了一种新的解决思路.把分布式视频编码(DVC)和CS结合在一起,构建简单的视频编码框架,并利用原始视频帧与边信息之间的相关性进行残差重构,提出了一种基于边信息的分布式视频压缩感知编解码方案.此方法对关键帧采用传统的帧内编、解码;对非关键帧CS进行随机观测提取观测向量,解码端利用优化的边信息和传输的CS观测向量进行联合重构.实验结果表明,该方法在运动较平滑的序列中比参考方案的恢复质量提高了4 ～6 dB.     

A retransmission‐based scheme for video streaming over wireless channels

Efficient resource allocation is a key factor to improve the efficiency of video transmission over wireless channels. To increase the number of correctly received video frames at the decoder, it is desirable to reduce the video source rate while increasing error protection when the wireless channel is anticipated to be bad or when the receiver buffer is approaching starvation. In this study, we introduce a retransmission‐based adaptive source‐channel rate control scheme for video transmission over wireless packet networks. In this scheme, the level of adaptiveness is optimized to reduce the bandwidth requirement while guaranteeing delay and loss bounds. The proposed scheme has the advantage of providing closed‐form expressions of the near‐optimum parameters of the proposed model, which are then fed back to the transmitter to scale both the source and channel rates adaptively. Simulation and numerical investigations are carried out to verify the adequacy of the analysis and study the impact of the adaptive process on the continuity of the video playback process. Copyright © 2009 John Wiley & Sons, Ltd.     

Downlink media streaming with wireless fountain coding in wireline‐cum‐WiFi networks

This paper addresses the problem of streaming packetized media data in a combined wireline/802.11 network. Since the wireless channel is normally the bottleneck for media streaming in such a network, we propose that wireless fountain coding (WFC) be used over the wireless downlink in order to efficiently utilize the wireless bandwidth and exploit the broadcast nature of the channel. Forward error correction (FEC) is also used to combat errors at the application‐layer. We analytically obtain the moment generating function (MGF) for the wireless link‐layer delay incurred by WFC. With the MGF, the expected value of this wireless link‐layer delay is found and used by the access point (AP), who has no knowledge of the buffer contents of wireless receivers, to make a coding‐based decision. We then derive the end‐to‐end packet loss/late probability based on the MGF. We develop an integrated ns‐3/EvalVid simulator to evaluate our proposed system and compare it with the traditional 802.11e scheme which is without WFC capability but equipped with application‐ and link‐layer retransmission mechanisms. Through extensive simulations of video streaming, we show that streaming with WFC is able to support more concurrent video flows compared to the traditional scheme. When the deadlines imposed on video packets are relatively stringent, streaming with WFC also shows superior performance in terms of packet loss/late probability, video distortion, and video frame delay, over the traditional scheme. Copyright © 2011 John Wiley & Sons, Ltd.     

Very low bit-rate video coding with DFD segmentation

A video coding system for applications requiring very low bit-rate is presented. This coding scheme uses an intraframe coder for the initial frame in the video sequence and subsequent frames are coded using an interframe coding method. A wavelet-based technique is used for intraframe coding. For interframe coding, displaced frame differences (DFD) are computed and coded using a segmentation-based method wherein the displaced frame difference is segmented into active and inactive regions using morphological operators. To meet the very low bit-rate requirements, the motion vectors are processed so as to reduce their contribution to the overall bit-rate. To reduce coding artifacts, a post-processing technique is developed for use at the decoder. Coding performance of the proposed coding scheme is evaluated at 16 kbit/s and 32 kbit/s using luminance component of several typical test sequences at QCIF resolution with a frame rate 8.3 frame/s.     

Hardware‐Software Implementation of MPEG‐4 Video Codec

This paper presents an MPEG‐4 video codec, called MoVa, for video coding applications that adopts 3G‐324M. We designed MoVa to be optimal by embedding a cost‐effective ARM7TDMI core and partitioning it into hardwired blocks and firmware blocks to provide a reasonable tradeoff between computational requirements, power consumption, and programmability. Typical hardwired blocks are motion estimation and motion compensation, discrete cosine transform and quantization, and variable length coding and decoding, while intra refresh, rate control, error resilience, error concealment, etc. are implemented by software. MoVa has a pipeline structure and its operation is performed in four stages at encoding and in three stages at decoding. It meets the requirements of MPEG‐4 SP@L2 and can perform either 30 frames/s (fps) of QCIF or SQCIF, or 7.5 fps (in codec mode) to 15 fps (in encode/decode mode) of CIF at a maximum clock rate of 27 MHz for 128 kbps or 144 kbps. MoVa can be applied to many video systems requiring a high bit rate and various video formats, such as videophone, videoconferencing, surveillance, news, and entertainment.     

A Hierarchical Scheme for Video-Based Person Re-identification Using Lightweight PCANet and Handcrafted LOMO Features

A two-level hierarchical scheme for video-based person re-identification (re-id) is presented, with the aim of learning a pedestrian appearance model through more complete walking cycle extraction. Specifically, given a video with consecutive frames, the objective of the first level is to detect the key frame with lightweight Convolutional neural network (CNN) of PCANet to reflect the summary of the video content. At the second level, on the basis of the detected key frame, the pedestrian walking cycle is extracted from the long video sequence. Moreover, local features of Local maximal occurrence (LOMO) of the walking cycle are extracted to represent the pedestrian' s appearance information. In contrast to the existing walking-cycle-based person re-id approaches, the proposed scheme relaxes the limit on step number for a walking cycle, thus making it flexible and less affected by noisy frames. Experiments are conducted on two benchmark datasets: PRID 2011 and iLIDS-VID. The experimental results demonstrate that our proposed scheme outperforms the six state-of-art video-based re-id methods, and is more robust to the severe video noises and variations in pose, lighting, and camera viewpoint.     

Using selective discard to improve real‐time video quality on an ethernet local area network

To support real‐time MPEG video on Ethernet, a selective discard scheme can be implemented to trade‐off the loss of more important MPEG “I” frames for less important “B” frames. Such a scheme results in better video quality. Copyright © 1999 John Wiley & Sons, Ltd.     

Computation-aware fast motion estimation for H.264/AVC using image indexing

The key to designing a real-time video coding system is efficient motion estimation, which reduces temporal redundancies. The motion estimation of the H.264/AVC coding standard can use multiple references and multiple block sizes to improve rate-distortion performance. The computational complexity of H.264 is linearly dependent on the number of allowed reference frames and block sizes using a full exhaustive search. Many fast block-matching algorithms reduce the computational complexity of motion estimation by carefully designing search patterns with different shapes or sizes, which have a significant impact on the search speed and distortion performance. However, the search speed and the distortion performance often conflict with each other in these methods, and their high computational complexity incurs a large amount of memory access. This paper presents a novel block-matching scheme with image indexing, which sets a proper priority list of search points, to encode a H.264 video sequence. This study also proposes a computation-aware motion estimation method for the H.264/AVC. Experimental results show that the proposed method achieves good performance and offers a new way to design a cost-effective real-time video coding system.