Perceptually driven video error protection using a distributed source coding approach

In video communication systems, the video signals are typically compressed and sent to the decoder through an error-prone transmission channel that may corrupt the compressed signal, causing the degradation of the final decoded video quality. In this context, it is possible to enhance the error resilience of typical predictive video coding schemes using as inspiration principles and tools from an alternative video coding approach, the so-called Distributed Video Coding (DVC), based on the Distributed Source Coding (DSC) theory. Further improvements in the decoded video quality after error-prone transmission may also be obtained by considering the perceptual relevance of the video content, as distortions occurring in different regions of a picture have a different impact on the user's final experience. In this context, this paper proposes a Perceptually Driven Error Protection (PDEP) video coding solution that enhances the error resilience of a state-of-the-art H.264/AVC predictive video codec using DSC principles and perceptual considerations. To increase the H.264/AVC error resilience performance, the main technical novelties brought by the proposed video coding solution are: (i) design of an improved compressed domain perceptual classification mechanism; (ii) design of an improved transcoding tool for the DSC-based protection mechanism; and (iii) integration of a perceptual classification mechanism in an H.264/AVC compliant codec with a DSC-based error protection mechanism. The performance results obtained show that the proposed PDEP video codec provides a better performing alternative to traditional error protection video coding schemes, notably Forward Error Correction (FEC)-based schemes.     

Motion-compensated DCT temporal filters for efficient spatio-temporal scalable video coding

Although most of the proposals for implementing motion-compensated temporal filtering (MCTF) schemes are based on the wavelet transform, in this paper, we propose an MCTF framework based on the discrete cosine transform (DCT). Using DCT decimation and interpolation, several temporal decomposition structures named motion-compensated DCT temporal filters (MCDCT-TF) are introduced. These structures are able to employ filters of any length with particular emphasis on 5/3 DCT and 7/4 DCT. The proposed MCDCT-TF and the two-dimensional (2D) DCT decimation technique are incorporated into H.264/AVC to provide spatio-temporal scalability. Compared with the current MCTF-based lifting schemes such as Haar, and 5/3 wavelet filters, simulation results show that the proposed MCDCT-TF utilizing longer tap DCT filters achieves a significant improvement in coding gain. The impact of odd/even group of frames, the decimation/interpolation ratios, and motion-compensated connectivity on the MCDCT-TF performance are also analyzed. Moreover, simulation results show that the performance of the presented scalable video coding is close to the single layer H.264/AVC and is slightly inferior to the temporal scalability supported in JSVM, the state-of-the-art scalable video coding standard, that gets its gain from Hierarchical B-pictures. However, our spatio-temporal coding scheme outperforms the spatio-temporal supported in JSVM even if it uses hierarchical B-pictures to improve its gain.     

Joint Source/Channel Coding Based on Two‐Dimensional Optimization for Scalable H.264/AVC Video

The scalable extension of the H.264/AVC video coding standard (SVC) demonstrates superb adaptability in video communications. Joint source and channel coding (JSCC) has been shown to be very effective for such scalable video consisting of parts of different significance. In this paper, a new JSCC scheme for SVC transmission over packet loss channels is proposed which performs two‐dimensional optimization on the quality layers of each frame in a rate‐distortion (R‐D) sense as well as on the temporal hierarchical structure of frames under dependency constraints. To compute the end‐to‐end R‐D points of a frame, a novel reduced trellis algorithm is developed with a significant reduction of complexity from the existing Viterbi‐based algorithm. The R‐D points of frames are sorted under the hierarchical dependency constraints and optimal JSCC solution is obtained in terms of the best R‐D performance. Experimental results show that our scheme outperforms the existing scheme of [13] with average quality gains of 0.26 dB and 0.22 dB for progressive and non‐progressive modes respectively.     

A new distortion measure for motion estimation in motion-compensated hybrid video coding

In the video coding standards MPEG-x and H.26x, a motion-compensated prediction technique is used for enhancing the coding performance of bitrate reduction or peak signal to noise ratio (PSNR) improvement. This technique takes advantage of the correlation between consecutive frames in the time domain, which is relatively higher than that between adjacent blocks in the spatial domain. In order to utilize the correlation between consecutive frames, the conventional video coding standards have used the motion estimation (ME) and compensation technique, where the Sum of the Absolute Differences (SAD) is usually used as the distortion measure. The ME estimates the reference block that could minimize the residual signal between the current and reference blocks. However, the SAD is not appropriate to the specific sequences that have global or local illumination changes. In addition, the high-resolution video sequences have higher spatial correlation than the low-resolution video sequences in general. Therefore, a new distortion measure that can consider spatial and temporal correlation simultaneously may be helpful to enhance the coding performance. The proposed distortion measure searches for a reference block that minimizes the motion-compensated residual signal when the DC-component is predicted. In our proposed algorithm, the maximum BD-rate improvement is up to 13.6% for illumination-changed video sequences, and the average BD-rate improvement is 6.6% for various high-resolution video sequences in the baseline profile.     

Multiview coding and error correction coding for 3D video over noisy channels

We consider the joint source–channel coding problem of stereo video transmitted over AWGN and flat Rayleigh fading channels. Multiview coding (MVC) is used to encode the source, as well as a type of spatial scalable MVC. Our goal is to minimize the total number of bits, which is the sum of the number of source bits and the number of forward error correction bits, under the constraints that the quality of the left and right views must each be greater than predetermined PSNR thresholds at the receiver. We first consider symmetric coding, for which the quality thresholds are equal. Following binocular suppression theory, we also consider asymmetric coding, for which the quality thresholds are unequal. The optimization problem is solved using both equal error protection (EEP) and a proposed unequal error protection (UEP) scheme. An estimate of the expected end-to-end distortion of the two views is formulated for a packetized MVC bitstream over a noisy channel. The UEP algorithm uses these estimates for packet rate allocation. Results for various scenarios, including non-scalable/scalable MVC, symmetric/asymmetric coding, and UEP/EEP, are provided for both AWGN and flat Rayleigh fading channels. The UEP bit savings compared to EEP are given, and the performances of different scenarios are compared for a set of stereo video sequences.     

Comparison of video protection methods for wireless networks

An experimental comparison of video protection methods targeted for wireless networks is presented. Basic methods are the data partitioning, reversible variable length coding, and macroblock row interleaving as well as macroblock scattering for packet loss protection. An implementation is described, in which scalable video is protected unequally with forward error correcting codes and retransmissions. Comparisons are performed for simulated wideband code division multiple access channel, and measurements are carried out with wireless local area network, Bluetooth as well as with GSM high speed circuit switched data. For the measurements, point-to-point connections are used. The achieved video quality is examined in our real-time wireless video demonstrator. The performance is measured with peak-signal-to-noise-ratio of received video, data overhead, communication delay, number of lost video frames, and decoding frame rate. Results show that the quality of decoded video can be improved by 1 dB with transparent connections compared to connections designed for general packet data. As a conclusion, a video coding subsystem must have access to the error control in a wireless link for the best quality in varying conditions.     

Performance modeling of FEC‐based unequal error protection for H.264/AVC video streaming over burst‐loss channels

Unequal error protection systems are a popular technique for video streaming. Forward error correction (FEC) is one of error control techniques to improve the quality of video streaming over lossy channels. Moreover, frame‐level FEC techniques have been proposed for video streaming because of different priority video frames within the transmission rate constraint on a Bernoulli channel. However, various communication and storage systems are likely corrupted by bursts of noise in the current wireless behavior. If the burst losses go beyond the protection capacity of FEC, the efficacy of FEC can be degraded. Therefore, our proposed model allows an assessment of the perceived quality of H.264/AVC video streaming over bursty channels, and is validated by simulation experiments on the NS‐2 network simulator at a given estimate of the packet loss ratio and average burst length. The results suggest a useful reference in designing the FEC scheme for video applications, and as the video coding and channel parameters are given, the proposed model can provide a more accurate evaluation tool for video streaming over bursty channels and help to evaluate the impact of FEC performance on different burst‐loss parameters. Copyright © 2014 John Wiley & Sons, Ltd.     

基于H.264 SVC无线视频传输的自适应差错保护方法

提出了一种针对H.264可分级编码(H.264 SVC)的自适应前向纠错编码保护方案.通过比较不同的纠错方案,提出了划分丢包率区间的概念,并根据不同区间的丢包率自适应地选择最佳的纠错方案.仿真结果表明,与单一保护方法相比,所提自适应方法能够取得更好的保护效果,更适于在无线信道中进行视频传输.     

Transparent encryption techniques for H.264/AVC and H.264/SVC compressed video

Transparent encryption of video content requires to provide a video preview that is left in plaintext, while the enhancement information is encrypted. In this paper we propose three algorithms that provide transparent encryption. The first two ones are based on the idea of generating controlled drift in such a way as to obtain the desired quality level, while the third algorithm employs scalable video coding. We provide experimental results on several video sequences, as well as a security analysis, showing that the proposed algorithms provide an effective framework to perform transparent encryption.     

一种低编码复杂度可分级的视频编码系统

在可分级视频编码（SVC,scalable video coding）的框架下,利用分布式视频编码（DVC,distributed video coding）技术,设计了一种低编码复杂度的SVC方案。该系统具有空间可分级的特性,各分层中仅用到了传统的帧内编码技术和DVC技术,最大限度的减小了SVC系统的编码复杂度。在该...     

Efficient in-network adaptation of encrypted H.264/SVC content

This paper addresses the efficient adaptation of encrypted scalable video content (H.264/SVC). RTP-based in-network adaptation schemes on a media aware network element (MANE) in an IPTV and VoD scenario are considered.Two basic alternatives to implement encryption and adaptation of H.264/SVC content are investigated: (i) full, format-independent encryption making use of Secure RTP (SRTP); (ii) SVC-specific encryption that leaves the metadata relevant for adaptation (NAL unit headers) unencrypted.The SRTP-based scheme (i) is straightforward to deploy, but requires the MANE to be in the security context of the delivery, i.e., to be a trusted node. For adaptation, the content needs to be decrypted, scaled, and re-encrypted. The SVC-specific approach (ii) enables both full and selective encryption, e.g., of the base layer only. SVC-specific encryption is based on own previous work, which is substantially extended and detailed in this paper. The adaptation MANE can now be an untrusted node; adaptation becomes a low-complexity process, avoiding full decryption and re-encryption of the content.This paper presents the first experimental comparison of these two approaches and evaluates whether multimedia-specific encryption can lead to performance and application benefits. Potential security threats and security properties of the two approaches in the IPTV and VoD scenario are elementarily analyzed. In terms of runtime performance on the MANE our SVC-specific encryption scheme significantly outperforms the SRTP-based approach. SVC-specific encryption is also superior in terms of induced end-to-end delays. The performance can even be improved by selective application of the SVC-specific encryption scheme. The results indicate that the efficient adaptation of SVC-encrypted content on low-end, untrusted network devices is feasible.     

SSIM-based error-resilient rate-distortion optimization of H.264/AVC video coding for wireless streaming

The SSIM-based rate-distortion optimization (RDO) has been verified to be an effective tool for H.264/AVC to promote the perceptual video coding performance. However, the current SSIM-based RDO is not efficient for improving the perceptual quality of the video streaming application over the error-prone network, because it does not consider the transmission induced distortion in the encoding process. In this paper, a SSIM-based error-resilient RDO scheme for H.264/AVC is proposed to improve the wireless video streaming performance. Firstly, with the help of the SSE-based RDO, we present a low-complexity Lagrange multiplier decision method for the SSIM-based RDO video coding in the error-free environment. Then, the SSIM-based decoding distortion of the user end is estimated at the encoder and is correspondingly introduced into the RDO to involve the transmission induced distortion into the encoding process. Further, the Lagrange multiplier is theoretically derived to optimize the encoding mode selection in the error-resilient RDO process. Experimental results show that the proposed SSIM-based error-resilient RDO can obtain superior perceptual video quality (more structural information) to the traditional SSE-based error-resilient RDO for wireless video streaming at the same bit rate condition.     

一种基于缩减栅格算法的SVC联合信源/信道编码方法

 针对H.264/AVC可分级编码扩充标准(SVC),本文提出了一种联合信源/信道编码(JSCC)的视频传输方法.该方法采用一种新型的缩减栅格算法与拉格朗日优化技术对SVC数据与差错控制保护级别进行最优分配.本文提出的缩减栅格算法采用疏散式栅格结构,实行依层计算与缩减队列的策略,计算效率比同类高效算法提高了约20~50倍.大量仿真数据表明,采用本文高效算法的JSCC方法在各种信道条件下达到同类方法相同的优化性能.     

H.264/AVC基本档次编码器实现时域可伸缩编码的方法

提出一种在H.264/AVC基本档次编码器中实现时域可伸缩编码的方案,该方案通过H.264/AVC标准所提供的多参考帧和内存管理控制操作机制来实现。对于现有的H.264/AVC解码器,不需任何修改,即可直接解码由本方案生成的时域可伸缩码流。     

Motion adaptive video coding scheme for time‐varying network

Adaptive video coding algorithms can encode a video stream dynamically on the basis of the amount of bandwidth available in a network. In this paper, a novel approach for adaptive video coding based on look‐up tables has been proposed. On the basis of the network conditions, the proposed codec estimates quantization parameter (QP) and also the spatial resolution for a group of pictures from the look‐up table. Then, QP for each frame is estimated on the basis of the motion content in each frame. More motion leads to more burden of bits while encoding the motion vectors. Also, quantization noise is less prominent in high‐motioned frames than frames possessing low motion information because of the temporal masking phenomenon of the human eyes. So, the main trick of the proposed scheme is to assign higher QP to the frames having higher motion than those having less motion. This method also reduces the requirement of excess bits for encoding of motion vectors having higher motion content. Additionally, QP is adjusted on the basis of the buffer availability in order to avoid bit loss error. Experimental results show the efficacy of the proposed codec. Copyright © 2013 John Wiley & Sons, Ltd.     

Mode dependent down-sampling and interpolation scheme for high efficiency video coding

In this paper, a mode dependent down-sampling and interpolation scheme is proposed to improve the coding efficiency of the intra prediction module. In the proposed method, we elaborately design the down-sampling structures and interpolation schemes for each directional intra prediction mode by minimizing the spatial prediction distance. The sampled pixels are predicted with a traditional directional intra prediction scheme, and the non-sampled pixels are predicted from the interpolation of their neighboring reconstructed sampling pixels. Both the residuals of the sampled and non-sampled pixels are encoded at last. Experimental results show that the proposed method achieves an average 7.52% bitrate reduction relative to KTA reference software. Since the down-sampling structure and interpolation method is only related to the intra mode, there is no additional overhead at the encoder.     

Lagrange multiplier selection in wavelet-based scalable video coding for quality scalability

In this paper, a method for Lagrange multiplier selection is proposed in the context of rate-distortion optimisation for wavelet-based scalable video coding targeting quality scalability. Despite the prevalence of the conventional method for Lagrange multiplier selection in hybrid video coding, the underlying formulation is not applicable to wavelet-based scalable video coding. To address the inherent challenges, a thorough analysis of the rate-distortion models for transform video coding is provided with regard to low and middle-to-high bit-rates, respectively. Based on the analysis, the models are consolidated according to experimental observations and the consolidated rate-distortion models serve as the basis for the derivation of the Lagrange multiplier. Considering the influence of the open-loop prediction structure on the rate-distortion performance, the Lagrange multiplier is initially derived for a single-targeted bit-rate. Moreover, the method for Lagrange multiplier selection in scalable video coding aiming at multiple-targeted bit-rates is proposed in a general sense of bit-rate range, varying from low to high bit-rates, building on the initially derived Lagrange multiplier for a single-targeted bit-rate. The proposed Lagrange multiplier is content adaptive and well suited for wavelet-based scalable video coding where quantisation steps are unavailable. Detailed performance evaluation of the proposed method for wavelet-based scalable video coding is provided with regard to a given targeted bit-rate and multiple-targeted bit-rates, respectively. The experimental results have demonstrated the effectiveness of the proposed Lagrange multiplier for rate-distortion optimisation considering quality scalability in wavelet-based scalable video coding.     

Distributed video coding based on lossy syndromes generated in hybrid pixel/transform domain

A recently proposed class of distributed source coding based video coders enables low-complexity compression and robust transmission over unreliable channels. These architectures process the video signal either in the pixel or in the transform domain generating some side information that permits a correct decoding of the coded image from a set of possible correlated sources. The approach proposed in this paper processes the video sequence both in the pixel and in the transform domain exploiting the advantages of both schemes and generating a set of lossy syndromes. The resulting video coding scheme requires a lower computational complexity at the decoder with respect to their transform-domain counterparts (like DISCOVER or PRISM) and provides a high compression gain and an increased robustness against channel losses.