Motion-compensated DCT temporal filters for efficient spatio-temporal scalable video coding

Although most of the proposals for implementing motion-compensated temporal filtering (MCTF) schemes are based on the wavelet transform, in this paper, we propose an MCTF framework based on the discrete cosine transform (DCT). Using DCT decimation and interpolation, several temporal decomposition structures named motion-compensated DCT temporal filters (MCDCT-TF) are introduced. These structures are able to employ filters of any length with particular emphasis on 5/3 DCT and 7/4 DCT. The proposed MCDCT-TF and the two-dimensional (2D) DCT decimation technique are incorporated into H.264/AVC to provide spatio-temporal scalability. Compared with the current MCTF-based lifting schemes such as Haar, and 5/3 wavelet filters, simulation results show that the proposed MCDCT-TF utilizing longer tap DCT filters achieves a significant improvement in coding gain. The impact of odd/even group of frames, the decimation/interpolation ratios, and motion-compensated connectivity on the MCDCT-TF performance are also analyzed. Moreover, simulation results show that the performance of the presented scalable video coding is close to the single layer H.264/AVC and is slightly inferior to the temporal scalability supported in JSVM, the state-of-the-art scalable video coding standard, that gets its gain from Hierarchical B-pictures. However, our spatio-temporal coding scheme outperforms the spatio-temporal supported in JSVM even if it uses hierarchical B-pictures to improve its gain.     

Joint Source/Channel Coding Based on Two‐Dimensional Optimization for Scalable H.264/AVC Video

The scalable extension of the H.264/AVC video coding standard (SVC) demonstrates superb adaptability in video communications. Joint source and channel coding (JSCC) has been shown to be very effective for such scalable video consisting of parts of different significance. In this paper, a new JSCC scheme for SVC transmission over packet loss channels is proposed which performs two‐dimensional optimization on the quality layers of each frame in a rate‐distortion (R‐D) sense as well as on the temporal hierarchical structure of frames under dependency constraints. To compute the end‐to‐end R‐D points of a frame, a novel reduced trellis algorithm is developed with a significant reduction of complexity from the existing Viterbi‐based algorithm. The R‐D points of frames are sorted under the hierarchical dependency constraints and optimal JSCC solution is obtained in terms of the best R‐D performance. Experimental results show that our scheme outperforms the existing scheme of [13] with average quality gains of 0.26 dB and 0.22 dB for progressive and non‐progressive modes respectively.     

Multiview coding and error correction coding for 3D video over noisy channels

We consider the joint source–channel coding problem of stereo video transmitted over AWGN and flat Rayleigh fading channels. Multiview coding (MVC) is used to encode the source, as well as a type of spatial scalable MVC. Our goal is to minimize the total number of bits, which is the sum of the number of source bits and the number of forward error correction bits, under the constraints that the quality of the left and right views must each be greater than predetermined PSNR thresholds at the receiver. We first consider symmetric coding, for which the quality thresholds are equal. Following binocular suppression theory, we also consider asymmetric coding, for which the quality thresholds are unequal. The optimization problem is solved using both equal error protection (EEP) and a proposed unequal error protection (UEP) scheme. An estimate of the expected end-to-end distortion of the two views is formulated for a packetized MVC bitstream over a noisy channel. The UEP algorithm uses these estimates for packet rate allocation. Results for various scenarios, including non-scalable/scalable MVC, symmetric/asymmetric coding, and UEP/EEP, are provided for both AWGN and flat Rayleigh fading channels. The UEP bit savings compared to EEP are given, and the performances of different scenarios are compared for a set of stereo video sequences.     

Comparison of video protection methods for wireless networks

An experimental comparison of video protection methods targeted for wireless networks is presented. Basic methods are the data partitioning, reversible variable length coding, and macroblock row interleaving as well as macroblock scattering for packet loss protection. An implementation is described, in which scalable video is protected unequally with forward error correcting codes and retransmissions. Comparisons are performed for simulated wideband code division multiple access channel, and measurements are carried out with wireless local area network, Bluetooth as well as with GSM high speed circuit switched data. For the measurements, point-to-point connections are used. The achieved video quality is examined in our real-time wireless video demonstrator. The performance is measured with peak-signal-to-noise-ratio of received video, data overhead, communication delay, number of lost video frames, and decoding frame rate. Results show that the quality of decoded video can be improved by 1 dB with transparent connections compared to connections designed for general packet data. As a conclusion, a video coding subsystem must have access to the error control in a wireless link for the best quality in varying conditions.     

基于H.264 SVC无线视频传输的自适应差错保护方法

提出了一种针对H.264可分级编码(H.264 SVC)的自适应前向纠错编码保护方案.通过比较不同的纠错方案,提出了划分丢包率区间的概念,并根据不同区间的丢包率自适应地选择最佳的纠错方案.仿真结果表明,与单一保护方法相比,所提自适应方法能够取得更好的保护效果,更适于在无线信道中进行视频传输.     

Transparent encryption techniques for H.264/AVC and H.264/SVC compressed video

Transparent encryption of video content requires to provide a video preview that is left in plaintext, while the enhancement information is encrypted. In this paper we propose three algorithms that provide transparent encryption. The first two ones are based on the idea of generating controlled drift in such a way as to obtain the desired quality level, while the third algorithm employs scalable video coding. We provide experimental results on several video sequences, as well as a security analysis, showing that the proposed algorithms provide an effective framework to perform transparent encryption.     

SSIM-based error-resilient rate-distortion optimization of H.264/AVC video coding for wireless streaming

The SSIM-based rate-distortion optimization (RDO) has been verified to be an effective tool for H.264/AVC to promote the perceptual video coding performance. However, the current SSIM-based RDO is not efficient for improving the perceptual quality of the video streaming application over the error-prone network, because it does not consider the transmission induced distortion in the encoding process. In this paper, a SSIM-based error-resilient RDO scheme for H.264/AVC is proposed to improve the wireless video streaming performance. Firstly, with the help of the SSE-based RDO, we present a low-complexity Lagrange multiplier decision method for the SSIM-based RDO video coding in the error-free environment. Then, the SSIM-based decoding distortion of the user end is estimated at the encoder and is correspondingly introduced into the RDO to involve the transmission induced distortion into the encoding process. Further, the Lagrange multiplier is theoretically derived to optimize the encoding mode selection in the error-resilient RDO process. Experimental results show that the proposed SSIM-based error-resilient RDO can obtain superior perceptual video quality (more structural information) to the traditional SSE-based error-resilient RDO for wireless video streaming at the same bit rate condition.     

一种基于缩减栅格算法的SVC联合信源/信道编码方法

 针对H.264/AVC可分级编码扩充标准(SVC),本文提出了一种联合信源/信道编码(JSCC)的视频传输方法.该方法采用一种新型的缩减栅格算法与拉格朗日优化技术对SVC数据与差错控制保护级别进行最优分配.本文提出的缩减栅格算法采用疏散式栅格结构,实行依层计算与缩减队列的策略,计算效率比同类高效算法提高了约20~50倍.大量仿真数据表明,采用本文高效算法的JSCC方法在各种信道条件下达到同类方法相同的优化性能.     

Mode dependent down-sampling and interpolation scheme for high efficiency video coding

In this paper, a mode dependent down-sampling and interpolation scheme is proposed to improve the coding efficiency of the intra prediction module. In the proposed method, we elaborately design the down-sampling structures and interpolation schemes for each directional intra prediction mode by minimizing the spatial prediction distance. The sampled pixels are predicted with a traditional directional intra prediction scheme, and the non-sampled pixels are predicted from the interpolation of their neighboring reconstructed sampling pixels. Both the residuals of the sampled and non-sampled pixels are encoded at last. Experimental results show that the proposed method achieves an average 7.52% bitrate reduction relative to KTA reference software. Since the down-sampling structure and interpolation method is only related to the intra mode, there is no additional overhead at the encoder.     

Lagrange multiplier selection in wavelet-based scalable video coding for quality scalability

In this paper, a method for Lagrange multiplier selection is proposed in the context of rate-distortion optimisation for wavelet-based scalable video coding targeting quality scalability. Despite the prevalence of the conventional method for Lagrange multiplier selection in hybrid video coding, the underlying formulation is not applicable to wavelet-based scalable video coding. To address the inherent challenges, a thorough analysis of the rate-distortion models for transform video coding is provided with regard to low and middle-to-high bit-rates, respectively. Based on the analysis, the models are consolidated according to experimental observations and the consolidated rate-distortion models serve as the basis for the derivation of the Lagrange multiplier. Considering the influence of the open-loop prediction structure on the rate-distortion performance, the Lagrange multiplier is initially derived for a single-targeted bit-rate. Moreover, the method for Lagrange multiplier selection in scalable video coding aiming at multiple-targeted bit-rates is proposed in a general sense of bit-rate range, varying from low to high bit-rates, building on the initially derived Lagrange multiplier for a single-targeted bit-rate. The proposed Lagrange multiplier is content adaptive and well suited for wavelet-based scalable video coding where quantisation steps are unavailable. Detailed performance evaluation of the proposed method for wavelet-based scalable video coding is provided with regard to a given targeted bit-rate and multiple-targeted bit-rates, respectively. The experimental results have demonstrated the effectiveness of the proposed Lagrange multiplier for rate-distortion optimisation considering quality scalability in wavelet-based scalable video coding.     

Distributed video coding based on lossy syndromes generated in hybrid pixel/transform domain

A recently proposed class of distributed source coding based video coders enables low-complexity compression and robust transmission over unreliable channels. These architectures process the video signal either in the pixel or in the transform domain generating some side information that permits a correct decoding of the coded image from a set of possible correlated sources. The approach proposed in this paper processes the video sequence both in the pixel and in the transform domain exploiting the advantages of both schemes and generating a set of lossy syndromes. The resulting video coding scheme requires a lower computational complexity at the decoder with respect to their transform-domain counterparts (like DISCOVER or PRISM) and provides a high compression gain and an increased robustness against channel losses.     

Dynamic macroblock wavefront parallelism for parallel video coding

Wavefront parallelism is effective for parallel video encoding thanks to its merits of low latency, no quality loss and high degree of parallelism. In traditional video encoders, macroblock row wavefront (MRW) parallelism was widely adopted. However the performance of MRW is limited by workload unbalance and computing resource unbalance among multiple work threads. This paper proposes a new dynamic macroblock wavefront (DMW) parallelism to alleviate the limitations of MRW. In DMW, the available macroblocks (MBs) are scheduled to work threads MB by MB other than MB row by MB row; and after one MB is encoded by a work thread, the MB on its right (if available) has the highest priority to be scheduled to avoid synchronization delay. Experimental results demonstrate that video encoder with traditional MRW can be accelerated by more than 10% with our proposed DMW. When hyper-threading is used, the advantage of DMW is more prominent.     

视频感兴趣区域快速提取与编码算法

如何在复杂度和时延受限的条件下,减少时域、空域和统计冗余,获得率失真性能的最优化是视频编码设计的核心问题。作为最新的视频编码标准,H.264在取得更高压缩率的同时,编码复杂度也随之剧增,限制了其在实时环境中的应用。本文在传统视频编码框架基础上,结合人类视觉系统感知特征,提出一种基于编码信息的视频感兴趣区域快速提取算法和优化编码方案。仿真实验结果表明,本文提出的编码策略在重建视频质量基本无损失和维持原有码率的提前下,将H.264编码时间平均节省52.55%,并能够与其它快速编码技术相结合,进一步提高H.264编码速度,以更好的适应异构网络及多样性终端环境,满足日益丰富的多媒体与通信业务需求。     

An efficient scalable intra coding algorithm for spatial scalability in enhancement layer

Scalable video coding (SVC) is attractive due to the capability of reconstructing lower resolution or lower quality signals from partial bit streams, which allows for simple solutions adaptted to network and terminal capabilities. This article addresses the spatial scalability of SVC and proposes an efficient H.264-based scalable intra coding algorithm. In comparison with precious single layer intra prediction (SLIP) method, the proposed algorithm aims to improve the intra coding performance of the enhancement layer by a new inter layer intra prediction (ILIP) method. The main idea of ILIP is that up-sampled and reconstructed pixels of the base layer are very useful to predict and encode those pixels of the enhancement layer, especially when those neighbouring pixels are not available. Experimental results show that the peak signal to noise ratio (PSNR) data of luminance component of encoded frames are improved, and both bit-rates and computation complexity are maintained very well. For sequence Football, the average increase of PSNR is up to 0.21?dB, while for Foreman and Bus, they are 0.14?dB and 0.17?dB, respectively.     

Content-adaptive bitstream-layer model for coding distortion assessment of H.264/AVC networked video

Bitstream-layer models are designed to use the information extracted from both packet headers and payload for real-time and non-intrusive quality monitoring of networked video. This paper proposes a content-adaptive bitstream-layer (CABL) model for coding distortion assessment of H.264/AVC networked video. Firstly, the fundamental relationship between perceived coding distortion and quantization parameter (QP) is established. Then, considering the fact that the perceived coding distortion of a networked video significantly relies on both the spatial and temporal characteristics of video content, spatial and temporal complexities are incorporated in the proposed model. Assuming that the residuals before Discrete Cosine Transform (DCT) keep to the Laplace distribution, the scale parameters of the Laplace distribution are estimated utilizing QP and quantized coefficients on the basis of the Parseval theorem firstly. Then the spatial complexity is evaluated using QP and the scale parameters. Meanwhile, the temporal complexity is obtained using the weighted motion vectors (MV) considering the variations in temporal masking extent for high motion regions and low motion regions, respectively. Both the two characteristics of video content are extracted from the compressed bitstream without resorting to a complete decoding. Using content related information, the proposed model is able to adapt to different video contents. Experimental results show that the overall performance of CABL model significantly outperforms that of the P.1202.1 model and other coding distortion assessment models in terms of widely used performance criteria, including the Pearson Correlation Coefficient (PCC), the Spearman Rank Order Correlation Coefficient (SROCC), the Root-Mean-Squared Error (RMSE) and the Outlier Ratio (OR).     

Robust spatio-temporal error concealment for packet- lossy H.264 video transmission

1 Introduction With the ubiquitous use of Internet and the deployment of next generation of networks, video communications are increa- singly becoming the major services in demand. Unlike data transmission, video communication is essentially time-sensitiv…     

Efficient rate control scheme for low bit rate H.264/AVC video coding

This article presents an efficient rate control scheme for H.264/AVC video coding in low bit rate environment. In the proposed scheme, an improved rate-distortion (RD) model by both analytical and empirical approaches is developed. It involves an enhanced mean absolute difference estimating method and a more rate-robust distortion model. Based on this RD model, an efficient macroblock-layer rate control scheme for H.264/AVC video coding is proposed. Experimental results show that this model encodes video sequences with higher peak signal-to-noise ratio gains and generates bit stream closer to the target rate.