Invertible temporal subband/wavelet filter banks with half-pixel-accurate motion compensation

Three-dimensional (3-D) subband/wavelet coding with motion compensation has been demonstrated to be an efficient technique for video coding applications in some recent research works. When motion compensation is performed with half-pixel accuracy, images need to be interpolated in both temporal subband analysis and synthesis stages. The resulting subband filter banks developed in these former algorithms were not invertible due to image interpolation. In this paper, an invertible temporal analysis/synthesis system with half-pixel-accurate motion compensation is presented. We look at temporal decomposition of image sequences as a kind of down-conversion of the sampling lattices. The earlier motion-compensated (MC) interlaced/progressive scan conversion scheme is extended for temporal subband analysis/synthesis. The proposed subband/wavelet filter banks allow perfect reconstruction of the decomposed video signal while retaining high energy compaction of subband transforms. The invertible filter banks are then utilized in our 3-D subband video coder. This video coding system does not contain the temporal DPCM loop employed in the conventional hybrid coder and the earlier MC 3-D subband coders. The experimental results show a significant PSNR improvement by the proposed method. The generalization of our algorithm for MC temporal filtering at arbitrary subpixel accuracy is also discussed.     

New temporal filtering scheme to reduce delay in wavelet-based video coding.

Scalability is an important desirable property of video codecs. Wavelet-based motion-compensated temporal filtering provides the most powerful scheme for scalable video coding and provides high-compression efficiency that competes with the current state of art codecs. However, the delay introduced by the temporal filtering schemes is sometimes very high, which makes them unsuitable for many real-time applications. In this paper, ue propose a new temporal filter set to minimize delay in 3-D wavelet-based video coding. The new filter set gives a performance at par with existing longer filters. The length of the filter can vary from two to any number of frames depending on delay requirements. If the frames are processed as separate groups of frames (GOFs), the proposed filter set will not have any boundary effects at the GOF. Experimental results are presented and conclusions are drawn.     

Low-complexity video coding via power-rate-distortion optimization

Wireless multimedia sensor networks (WMSNs) have been potentially applicable for several emerging applications. The resources, i.e., power and bandwidth available to visual sensors in a WMSN are, however, very limited. Hence, it is important but challenging to achieve efficient resource allocation and optimal video data compression while maximizing the overall network lifetime. In this paper, a power-rate-distortion (PRD) optimized resource-scalable low-complexity multiview video encoding scheme is proposed. In our video encoder, both the temporal and interview information can be exploited based on the comparisons of extracted media hashes without performing motion and disparity estimations, which are known to be time-consuming. We present a PRD model to characterize the relationship between the available resources and the RD performance of our encoder. More specifically, an RD function in terms of the percentages for different coding modes of blocks and the target bit rate under the available resource constraints is derived for optimal coding mode decision. The major goal here is to design a PRD model to optimize a “motion estimation-free” low-complexity video encoder for applications with resource-limited devices, instead of designing a general-purpose video codec to compete compression performance against current compression standards (e.g., H.264/AVC). Analytic results verify the accuracy of our PRD model, which can provide a theoretical guideline for performance optimization under limited resource constraints. Simulation results on joint RD performance and power consumption (measured in terms of encoding time) demonstrate the applicability of our video coding scheme for WMSNs.     

A low-complexity image compression approach with single spatial prediction mode and transform

The well-known low-complexity JPEG and the newer JPEG-XR systems are based on block-based transform and simple transform-domain coefficient prediction algorithms. Higher complexity image compression algorithms, obtainable from intra-frame coding tools of video coders H.264 or HEVC, are based on multiple block-based spatial-domain prediction modes and transforms. This paper explores an alternative low-complexity image compression approach based on a single spatial-domain prediction mode and transform, which are designed based on a global image model. In our experiments, the proposed single-mode approach uses an average 20.5 % lower bit-rate than a standard low-complexity single-mode image coder that uses only conventional DC spatial prediction and 2-D DCT. It also does not suffer from blocking effects at low bit-rates.     

Efficient video watermarking based on singular value decomposition in the discrete wavelet transform domain

This paper presents an efficient, robust, and imperceptible video watermarking technique based on singular value decomposition (SVD) performed in the Discrete Wavelet Transform (DWT) domain. In the proposed DWT-based SVD video watermarking method, the video frames are transformed with the DWT using two resolution levels. The high frequency band HH and the middle frequency bands LH and HL are SVD transformed and the watermark is hidden in them. The proposed DWT-based SVD video watermarking method is characterized by two improvements: (1) a cascade of two powerful mathematical transforms; the Discrete Wavelet Transform (DWT)-based SVD using additive method, and (2) an error correction code is applied and embeds the watermark with spatial and temporal redundancy. The aim of these improvements is to increase robustness against attacks based on video characteristics and the robustness against image processing attacks, realize high security level, protect the watermark against bit errors and obtain good perceptual quality. The proposed DWT-based SVD video watermarking method has been tested in the presence of video and image processing attacks and experimental results proved that the proposed DWT-based SVD video watermarking method survives attacks based on video characteristics and image processing techniques.     

On the adaptive three-dimensional transform coding techniques for moving images

In this paper, an adaptive three-dimensional transform coding technique based on the 3-D discrete cosine transform (DCT) for removing the temporal correlation is proposed. Because of the nonstationary nature of the image data, the energy distribution in a 3-D DCT block varies along the vertical, horizontal and temporal directions. Thus, adaptive schemes, such as the 3-D classification, the classified linear scanning technique and the VLC table selection scheme, are used to take local variations into account. Also, in our approach, a hybrid technique, which adaptively combines relatively simple inter-frame coding with intra-frame coding, is presented. Through intensive computer simulations, the performance of the proposed 3-D transform coding technique is evaluated on several well-known moving sequences. The results show that, especially for moving sequences containing slow or moderate motion, the proposed technique provides an improved performance over the scheme with motion compensation (CCITT, 1989) at rates above 0.5 b/pixel (bpp), and a good visual quality of the reconstructed images is also obtained. Thus, the proposed 3-D transform coding technique is believed to be a good candidate for the digital VCR, since motion compensation is not required in the proposed 3-D coding technique.     

Video coding scheme using DCT-pyramid vector quantization

A new and effective video coding scheme for contribution quality is proposed. The CMTT/2, a joint committee of CCIR and CCITT, has proposed a video coding scheme (already approved at European level by ETS) working at 34-45 Mbit/s. Basically this proposal includes a DCT transform for spatial correlation removal and motion compensation for temporal correlation removal. The individual transform coefficients are then scalar quantized with a non uniform bit assignment. Starting from the CMTT/2 proposal, the study presents a new video coding scheme designed using a vector quantizer solution instead of the scalar one. Specifically, the pyramid vector quantization (PVQ) has been chosen as the vector quantization method as it is able to reduce the DCT coefficients Laplacian distribution. Simulation results show that the proposed video coding scheme gives the same contribution quality at 22 Mbit/s as the one obtained with the CMTT/2 proposal at 45 Mbit/s.     

Distributed video coding based on lossy syndromes generated in hybrid pixel/transform domain

A recently proposed class of distributed source coding based video coders enables low-complexity compression and robust transmission over unreliable channels. These architectures process the video signal either in the pixel or in the transform domain generating some side information that permits a correct decoding of the coded image from a set of possible correlated sources. The approach proposed in this paper processes the video sequence both in the pixel and in the transform domain exploiting the advantages of both schemes and generating a set of lossy syndromes. The resulting video coding scheme requires a lower computational complexity at the decoder with respect to their transform-domain counterparts (like DISCOVER or PRISM) and provides a high compression gain and an increased robustness against channel losses.     

Motion-compensated 3-D subband coding of video

This paper describes a video coding system based on motion-compensated three-dimensional (3-D) subband/wavelet coding (MC-3DSBC), which can overcome the limits of both 3-D SBC and MC prediction-based coding. In this new system, spatio-temporal subbands are generated by MC temporal analysis and a spatial wavelet transform, and then encoded by 3-D subband-finite state scalar quantization (3DSB-FSSQ). The rate allocation from the GOP level to each class of subbands is optimized by utilizing the structural property of MC-3DSBC that additive superposition approximately holds for both rate and distortion. The proposed video coding system is applied to several test video clips. Its performance exceeds that of both a known MPEG-1 implementation and a similar subband MC predictive coder while maintaining modest computational complexity and memory size.     

Joint Source/Channel Coding Based on Two‐Dimensional Optimization for Scalable H.264/AVC Video

The scalable extension of the H.264/AVC video coding standard (SVC) demonstrates superb adaptability in video communications. Joint source and channel coding (JSCC) has been shown to be very effective for such scalable video consisting of parts of different significance. In this paper, a new JSCC scheme for SVC transmission over packet loss channels is proposed which performs two‐dimensional optimization on the quality layers of each frame in a rate‐distortion (R‐D) sense as well as on the temporal hierarchical structure of frames under dependency constraints. To compute the end‐to‐end R‐D points of a frame, a novel reduced trellis algorithm is developed with a significant reduction of complexity from the existing Viterbi‐based algorithm. The R‐D points of frames are sorted under the hierarchical dependency constraints and optimal JSCC solution is obtained in terms of the best R‐D performance. Experimental results show that our scheme outperforms the existing scheme of [13] with average quality gains of 0.26 dB and 0.22 dB for progressive and non‐progressive modes respectively.     

SNR and temporal scalable coding of 3-D mesh sequences using singular value decomposition

A signal-to-noise ratio (SNR) and temporal scalable coding algorithm for 3-D mesh sequences using singular value decomposition (SVD) is proposed in this work. The proposed algorithm employs SVD to represent a mesh sequence with a small number of basis vectors, and encodes those basis vectors with a bit plane coder. We analytically derive the contribution of each bit plane to the reconstructed mesh quality, and transmit the bit planes in the decreasing order of their amounts of contribution. As the decoder receives more bit planes, it reconstructs higher quality mesh sequences progressively. Moreover, we develop a temporal prediction mode to improve the rate–distortion (R–D) performance further, which also supports temporal scalability. Simulation results demonstrate that the proposed algorithm yields significantly better R–D performance than conventional SVD-based coders.     

Two-Dimensional DOA Estimation by?Cross-Correlation Matrix Stacking

In this paper, we present a novel scheme to improve the two-dimensional (2-D) direction-of-arrival (DOA) estimation performance for narrowband signals impinging on two orthogonal uniform linear arrays (ULAs). The proposed scheme exploits the cross-correlation matrix information between subarray data to construct a stacking matrix and derive an expanded signal subspace representation through the singular value decomposition (SVD). This method enables the alleviation of the effects of additive noise. In particular, 2-D DOA estimation can be achieved by computing two rotation matrices with the same set of eigenvectors obtained by partitioning the expanded signal subspace. The pair matching procedure for elevation and azimuth angles is implemented by permutation test. Simulation results demonstrate that the proposed method performs better than the existing techniques in DOA estimation as well as the detection of successful pair matching.     

Estimation of depth fields suitable for video compression based on3-D structure and motion of objects

Intensity prediction along motion trajectories removes temporal redundancy considerably in video compression algorithms. In three-dimensional (3-D) object-based video coding, both 3-D motion and depth values are required for temporal prediction. The required 3-D motion parameters for each object are found by the correspondence-based E-matrix method. The estimation of the correspondences-two-dimensional (2-D) motion field-between the frames and segmentation of the scene into objects are achieved simultaneously by minimizing a Gibbs energy. The depth field is estimated by jointly minimizing a defined distortion and bit-rate criterion using the 3-D motion parameters. The resulting depth field is efficient in the rate-distortion sense. Bit-rate values corresponding to the lossless encoding of the resultant depth fields are obtained using predictive coding; prediction errors are encoded by a Lempel-Ziv algorithm. The results are satisfactory for real-life video scenes.     

Restricted H.264/AVC video coding for privacy protected video scrambling

The issue of personal privacy has garnered significant attention with the extensive application of video surveillance systems. Privacy region scrambling is an effective method to protect privacy in video. To ensure that nonprivacy regions are not affected by scrambling, particular methods must be taken to prevent drift error in privacy protected video scrambling. However, existing methods have significantly reduced the coding efficiency. In this paper, we focus on improving coding efficiency while preventing drift error in privacy protected H.264/AVC video scrambling, which is the state-of-the-art coding standard. A restricted video coding scheme is proposed, which involves three parts of Mode Restricted Intra Prediction (MRIP), Search Window Restricted Motion Estimation (SWRME) and Boundary Strength Restricted Deblocking Filtering (BSRDF). Experimental results show that the proposed restricted video coding scheme prevents drift error with higher coding efficiency than others.     

适合于渐进传输的嵌入式三维小波视频构架

描述了怎样有效地将基本层视频编码H.264嵌入到3D小波视频编码的构架,在3D可扩展视频编码方案中提取出基本视频层可以获得诸如闭环短时预测、环路滤波和率失真最优化等特点,还能与现有的视频标准相适应,相对于3D小波视频编码可改善在低码率条件下的信噪比.     

SSIM-based error-resilient rate-distortion optimization of H.264/AVC video coding for wireless streaming

The SSIM-based rate-distortion optimization (RDO) has been verified to be an effective tool for H.264/AVC to promote the perceptual video coding performance. However, the current SSIM-based RDO is not efficient for improving the perceptual quality of the video streaming application over the error-prone network, because it does not consider the transmission induced distortion in the encoding process. In this paper, a SSIM-based error-resilient RDO scheme for H.264/AVC is proposed to improve the wireless video streaming performance. Firstly, with the help of the SSE-based RDO, we present a low-complexity Lagrange multiplier decision method for the SSIM-based RDO video coding in the error-free environment. Then, the SSIM-based decoding distortion of the user end is estimated at the encoder and is correspondingly introduced into the RDO to involve the transmission induced distortion into the encoding process. Further, the Lagrange multiplier is theoretically derived to optimize the encoding mode selection in the error-resilient RDO process. Experimental results show that the proposed SSIM-based error-resilient RDO can obtain superior perceptual video quality (more structural information) to the traditional SSE-based error-resilient RDO for wireless video streaming at the same bit rate condition.     

Unconstrained motion compensated temporal filtering (UMCTF) for efficient and flexible interframe wavelet video coding

We introduce an efficient and flexible framework for temporal filtering in wavelet-based scalable video codecs called unconstrained motion compensated temporal filtering (UMCTF). UMCTF allows for the use of different filters and temporal decomposition structures through a set of controlling parameters that may be easily modified during the coding process, at different granularities and levels. The proposed framework enables the adaptation of the coding process to the video content, network and end-device characteristics, allows for enhanced scalability, content-adaptivity and reduced delay, while improving the coding efficiency as compared to state-of-the-art motion-compensated wavelet video coders. Additionally, a mechanism for the control of the distortion variation in video coding based on UMCTF employing only the predict step is proposed. The control mechanism is formulated by expressing the distortion in an arbitrary decoded frame, at any temporal level in the pyramid, as a function of the distortions in the reference frames at the same temporal level. All the different scenarios proposed in the paper are experimentally validated through a coding scheme that incorporates advanced features (such as rate-distortion optimized variable block-size multihypothesis prediction and overlapped block motion compensation). Experiments are carried out to determine the relative efficiency of different UMCTF instantiations, as well as to compare against the current state-of-the-art in video coding.     

Robust Digital Terrestrial TV Broadcasting System with Error-Resilient Schemes

In this paper, error-resilient schemes are proposed to support robust video transmission for digital terrestrial TV broadcasting (DTTB). In particular, a temporal error concealment incorporated with a low-complexity block-matching is developed, achieving an effective reception of predictive pictures in harsh terrestrial environment. Special algorithms are also designed for isolated I-pictures. Moreover, combined with an intra/inter case prediction, an adaptive error concealment scheme is further contrived to fit for different error conditions. Extensive simulations have been conducted under various DTTB channel conditions, even with a very high packet error rate, to verify the effectiveness of the proposed schemes.     

Motion estimation in low-delay hierarchical p-frame coding using motion vector composition

Low-delay hierarchical prediction structure is currently adopted in various new video coding standards. The only hurdle of this structure is the need of motion estimation in distant reference frames. To maintain high coding efficiency, a large search range for motion estimation can improve the coding efficiency in distant reference pictures. Computational complexity will thus be increased dramatically. In this paper, a fast motion estimation scheme for a low-delay hierarchical P-frame structure is proposed. The proposed scheme adopts a motion vector composition strategy to expedite the motion estimation process for distant reference frames in the hierarchical P structure. In addition, a motion vector composition algorithm is tailor-made with the proposed hierarchical P coding scheme to further improve the coding efficiency. Simulation results show that the proposed scheme can deliver a remarkable complexity savings and coding efficiency improvement on coding a frame in low temporal layers of the hierarchical P structure.