Block motion compensated coding of interlaced sequences using adaptively deinterlaced fields

Video coding algorithms using block motion compensation were first developed for progressively scanned sequences and as such, are not entirely suitable for interlaced sequences In this paper we present a new approach for block-based coding of interlaced sequences. This proposed algorithm processes the interlaced sequence as a sequence of even and odd fields by using the last decoded field, adaptively deinterlaced, for the motion compensated prediction of the current field. The deinterlacing is performed at the decoder and no extra information has to be sent to guide the adaptation. The algorithm is a simple and efficient alternative to algorithms using the last two decoded fields for the motion compensated prediction of the current field. The new approach can easily incorporate the use of fast search algorithms and allows the use of true half-pixel accuracy in the estimates of the vertical component of the motion vectors. In HDTV sequences tested, this algorithm achieves superior performance due to this half-pixel accuracy.     

Invertible temporal subband/wavelet filter banks with half-pixel-accurate motion compensation

Three-dimensional (3-D) subband/wavelet coding with motion compensation has been demonstrated to be an efficient technique for video coding applications in some recent research works. When motion compensation is performed with half-pixel accuracy, images need to be interpolated in both temporal subband analysis and synthesis stages. The resulting subband filter banks developed in these former algorithms were not invertible due to image interpolation. In this paper, an invertible temporal analysis/synthesis system with half-pixel-accurate motion compensation is presented. We look at temporal decomposition of image sequences as a kind of down-conversion of the sampling lattices. The earlier motion-compensated (MC) interlaced/progressive scan conversion scheme is extended for temporal subband analysis/synthesis. The proposed subband/wavelet filter banks allow perfect reconstruction of the decomposed video signal while retaining high energy compaction of subband transforms. The invertible filter banks are then utilized in our 3-D subband video coder. This video coding system does not contain the temporal DPCM loop employed in the conventional hybrid coder and the earlier MC 3-D subband coders. The experimental results show a significant PSNR improvement by the proposed method. The generalization of our algorithm for MC temporal filtering at arbitrary subpixel accuracy is also discussed.     

一种基于立体视邻接帧时空相关性的最小代价函数帧估计算法

为了在立体视频序列编码中获得高的压缩率,需要对立体视频序列中一个视的序列按传统方法进行独立编码;另一个视的序列中,只对其中一些参考帧(I帧或P帧)按视差补偿预测的方法进行编码,其余帧不进行编码和传输,而在解码端用立体视帧估计的方法得到重建.本文提出了一种基于立体视中邻接帧在图像、视差场和运动矢量场之间高度相关性的方法.对于因遮挡而缺乏估计的区域,则结合了图像强度的连续性和运动,视差矢量的分布特性,构造了代价方程并估计出该部分的运动矢量及强度值.实验证明,重建出来的帧图像在视觉和信噪比意义上均具有较好的效果.     

一种基于硬件实现的快速运动估计半像素级搜索算法

研究了在绝对差和准则下的整像素级块匹配和半像素级块匹配的联系，对绝对差和进行了合理的数学曲线拟合，通过数学曲线来预测最小绝对差和所在半像素位置，从而得到半像素级最佳匹配矢量。分析了三种不同的凹函数预测模型，提出了一种适合硬件实现的运动估计快速半像素级搜索算法。该算法直接根据整像素级运动估计的结果来推算半像素级运动估计结果，在很大程度上降低了半像素级运动估计的运算复杂度，从而利用低码率视频编码的实时实现。试验结果表明该算法可获得较好的重建图像质量。该算法利于硬件实现，可以方便地集成到现有的视频编码器中，具有较好的实用价值。     

H.264标准半像素精度运动估计的硬件结构设计

设计了一种实时的基于可变块的半像素精度运动估计模块,包括半像素插值模块和半像素搜索模块,插值模块采用6阶FIR滤波器进行插值,搜索模块采用分级搜索算法.此模块应用在H.264标准便携视频设备的编码部分,用Verilog语言编写,采用Xilinx公司XC4VSX25的FPGA芯片作为硬件实现的运算核心.     

Temporal image sequence prediction using motion field interpolation

A new method for motion-compensated temporal prediction of image sequences is proposed. Motion vector fields in natural scenes should possess two basic properties. First, the field should be smoothly varying within moving objects to compensate for nonrigid or rotational motion, and scaling of objects. Second, the field should be discontinuous along the boundaries of the objects. In the proposed method the motion vector field is modelled using finite element methods and interpolated using adaptive interpolators to satisfy the above-stated requirements. This is particularly important when only very sparse estimates of motion vector fields are available in the decoder due to bit-rate constraints limiting the amount of overhead information that can be transmitted. The proposed prediction method can be applied for low-bit-rate video coding in conventional codecs based on motion-compensated prediction and transform coding, as well as in model-based codecs. The performance of the proposed method is compared with standard motion-compensated prediction based on block matching. It is shown that for simple video telephony scenes a reduction of more than 30% in the energy of the prediction error can be achieved with an unchanged number of transmitted motion vectors and with only a modest increase in computational complexity. When implemented in an H.261 codec the new prediction method can improve the peak SNR 1–2 dB producing a significant visual improvement.     

Recovery of motion vectors for error concealment based on an edge-detection approach

A new approach to error concealment is presented which exploits both spatial and temporal information from the current and previous frames. The technique consists of two stages, motion vector estimation and enhancement of the estimated motion vector. In the first stage the proposed method estimates a replacement for the corrupted motion vector by applying dynamic weights to related motion vectors from the top, bottom, left and right sub-macroblocks. The estimated motion vectors are then enhanced using a new approach based on edge detection in the second stage. The experimental results for several test video sequences are compared with conventional error concealment methods and higher performance is achieved in both objective peak signal-to-noise ratio measurements and subjective visual quality.     

Wavelet-based coding of time-varying vector fields of ocean-surface winds

Geoscience applications often produce sizable datasets that are vector-valued and increasingly in need of compression algorithms to reduce storage and transmission burdens, particularly when the data are time-varying. In this paper, several advanced interframe-compression techniques are extended from the traditional realm of natural video to the coding of time-varying vector fields. Although similar to natural video in some respects, time-varying vector-field sequences often possess complex temporal evolution of vector-valued features that are important to the analytic quality of the data yet defy the simple motion models widely employed for natural video. To improve coding performance, motion compensation with reduced resolution is proposed such that motion compensation is applied only at low spatial resolution, while high-resolution information, for which the motion model fails, is intraframe coded with no temporal decorrelation. In empirical results on datasets of ocean-surface winds, this reduced-resolution motion-compensation technique results in significant performance improvement and greater feature preservation.     

Half-pixel filter of MC-DCT compressed video

A novel half-pixel filter is proposed to extract one block with half-pixel precision motion vector in the DCT domain. The proposed filter reduces the computational complexity by integrating the interpolation and translation into a single step, while improves the video quality compared with the existing half-pixel filter.     

一种基于H.264/AVC的高效块匹配搜索算法

本文针对H.264/AVC的编码特点,提出了一种利用时空域运动相关性的快速块匹配搜索算法.该算法充分利用了视频序列的运动程度与宏块编码模式间的关联特性以及运动矢量的统计特征,明显减少了运动估计的搜索复杂度.实验表明,本文方法的搜索速度分别比FS和DS算法平均提高了77.96%和32.19%;重建图像的PSNR比DS算法平均提高了0.06dB,更接近FS算法的编码质量.     

Human visual weighted quantization for transform/subband imagecoding revisited for interlaced pictures

This article addresses the problem of human visual weighted quantization for transform/subband coding of interlaced pictures. Merged field coding is assumed. The criterion proposed for the optimization of the quantizer is the distortion measured on the deinterlaced and decoded image. This criterion and the associated weighting factors are shown to be dependent on the motion between two successive fields.     

Joint depth-motion dense estimation for multiview video coding

The multiview video coding (MVC) extension of H.264/MPEG-4 AVC [1] is one of the most promising visual encoders for three-dimensional television and free viewpoint video applications. In this paper, we propose a joint dense motion/disparity estimation algorithm, designed to replace the classical temporal/inter-view unit within MVC, which uses a block-based motion/disparity estimation. The motion vector fields and the disparity vector fields are therefore simultaneously derived using the stereo-motion consistency constraint in a set theoretic convex optimization framework. The obtained displacement vector fields are then jointly segmented by minimizing a rate-distortion cost function, in line with the multiple reference frame strategy used in H.264/MPEG-4 AVC. Experimental results demonstrate the benefits of the proposed method compared to the separated dense estimation scheme or the block-based estimation technique.     

On the performance bounds of motion-compensated deinterlacing

The terms ‘proscan up-conversion’ or ‘deinterlacing’ stand for algorithms for the conversion of interlaced image sequences into progressive format by interpolating the lines which are missing due to interlaced scanning. Although deinterlacing is already known to improve TV image quality, such techniques will become even more important in future. Firstly, standard interlaced image sequences will be increasingly processed and displayed on computer platforms and thus usually require progressive scanning and an increased temporal resolution. Secondly, the emerging market of light valve projection equipment demands high-quality proscan up-conversion since the image quality of these display techniques relies heavily on the light efficiency of non-interlaced scanning. By describing deinterlacing as a problem of linear prediction theory, this paper derives upper MSE performance bounds for these algorithms. Thus, a theoretical framework is provided which allows to analyse the effects of various parameters like motion vector inaccuracy, temporal integration and memory constraints on the efficiency of motion-compensated deinterlacing. It is, for example, shown that these algorithms can be very sensitive to motion vector inaccuracies especially if the image sequence is recorded with very short exposure time. Finally, it is briefly demonstrated that the presented analysis can easily be extended to the case of field- and frame-rate up-conversion of interlaced (‘100 Hz’) and non-interlaced image sequences.     

Neighbourhood-blocks motion vector estimation technique using pyramidal data structure

A pyramidal motion estimation technique that makes use of the motion correlation within a pyramidal level is proposed. In the proposed technique, motion vectors from neighbouring motion blocks are taken into consideration as possible candidates. This is done in lieu of scaling the motion vectors from the corresponding positions at the adjacent lower pyramidal level as the prediction motion vectors for the current pyramidal level (as performed in the conventional technique). Each of these candidate motion vectors is used as the prediction motion vector and refined, and the one that has the least matching distortion is chosen as the motion vector at the current pyramidal level. Compared to the conventional pyramidal motion estimation technique, the proposed method effectively overcomes the problem of propagation of false motion vectors. Simulation studies show that a substantial improvement is achieved in the performance, both in terms of the prediction mean square error and the number of coding bits for the motion vectors.     

Motion estimation in low-delay hierarchical p-frame coding using motion vector composition

Low-delay hierarchical prediction structure is currently adopted in various new video coding standards. The only hurdle of this structure is the need of motion estimation in distant reference frames. To maintain high coding efficiency, a large search range for motion estimation can improve the coding efficiency in distant reference pictures. Computational complexity will thus be increased dramatically. In this paper, a fast motion estimation scheme for a low-delay hierarchical P-frame structure is proposed. The proposed scheme adopts a motion vector composition strategy to expedite the motion estimation process for distant reference frames in the hierarchical P structure. In addition, a motion vector composition algorithm is tailor-made with the proposed hierarchical P coding scheme to further improve the coding efficiency. Simulation results show that the proposed scheme can deliver a remarkable complexity savings and coding efficiency improvement on coding a frame in low temporal layers of the hierarchical P structure.     

Optimization of quadtree segmentation and hybrid two-dimensionaland three-dimensional motion estimation in a rate-distortion framework

A rate-distortion framework is used to define a very low bit-rate coding scheme based on quadtree segmentation and optimized selection of motion estimators. This technique achieves maximum reconstructed image quality under the constraint of a target bit rate for the coding of the vector field and segmentation information. First, a complete scheme is proposed for hybrid two-dimensional (2-D) and three-dimensional (3-D) motion estimation and compensation. The quadtree object segmentation is optimized for hybrid motion estimation in the rate-distortion sense. This scheme adapts to the depth of the quadtree and the technique used for motion estimation for each leaf of the tree. A more sophisticated technique, adapted to the requirements of a very low bit-rate coder, is also proposed which also considers the transmission of the prediction error corresponding to the particular choice of the motion estimator. Based on these coding schemes, two versions of a very low bit-rate image sequence coder are developed. Experimental results illustrating the performance of the proposed techniques in very low bit-rate image sequence coding application areas are presented and evaluated     

Statistical properties of coded interlaced and progressive imagesequences

We investigate the statistical properties of interlaced and progressive image sequences in order to determine what is the most efficient format from a coding point of view. Intra- and inter-coding modes are both considered. The main result is that, for both intra- and inter-images, the coding of progressive images outperforms the coding of interlaced ones for low bit rates and large motion vectors. Particular attention is paid to inter-coding with subpel accuracy, where the effects of nonideal interpolation filters and finite precision of the motion vector are taken into account.     

基于运动估计的模糊图像盲复原

针对由于物体在不同方向移动速度不同导致退化的CCD图像空间移变特性的运动模糊,提出一种基于隔行扫描空间移变特性的CCD图像恢复方案。方法主要依据隔行扫描图像分为上下两场的特性,借助插值算法将单幅图像按场拆分为两幅存在时间差的图像,然后引入视频处理中基于图像序列确定运动向量的块匹配算法,估计模糊参数,从而实现应用图像序列算法估计单幅图像点扩散函数,最后利用最小二成方算法进行复原,其中引入了图像定向信息测量操作保持边缘。仿真实验结果表明,本文算法比以往算法具有大的信噪比(SNR)和图像保真度(IMFID),以及小的平均绝对差(ABSDIFF)和均方误差(MSE)。     

H.263: video coding for low-bit-rate communication

This article provides an overview of H.263, the new ITU-T Recommendation for low-bit-rate video communication. H.263 specifies a coded representation for compressing the moving picture component of audio-visual signals at low bit rates. The basic structure of the video source coding algorithm is taken from ITU-T Recommendation H.261 and is a hybrid of interpicture prediction to reduce temporal redundancy and transform coding of the prediction residual to reduce spatial redundancy. The source coder can operate on five standardized picture formats: sub-QCIF, QCIF, CIF, 4CIF, and 16CIF. The decoder has motion compensation capability with half-pixel precision, in contrast to H.261 which uses full-pixel precision and employs a loop filter. H.263 includes four negotiable coding options which provide improved coding efficiency: unrestricted motion vectors, syntax-based arithmetic coding, advanced prediction, and PB-frames     

An efficient motion vector coding scheme based on minimum bitrateprediction

Motion vector coding efficiency is becoming an important issue in low bitrate video coding because of its increasing relative bit portion. This work presents a new motion vector coding technique based on minimum bitrate prediction. In the proposed scheme, a predicted motion vector is chosen from the three causal neighboring motion vectors so that it can produce a minimum bitrate in motion vector difference coding. Then the prediction error, or motion vector difference (MVD), and the mode information (MODE) for determining the predicted motion vector at a decoder are coded and transmitted in order. Sending bits for the MVD ahead of bits for the MODE, the scheme can minimize the bit amount for the MODE by taking advantage of the fact that the minimum bitrate predictor is used for motion vector prediction. Adaptively combining this minimum bitrate prediction scheme with the conventional model-based prediction scheme, more efficient motion vector coding can be achieved. The proposed scheme improves the coding efficiency noticeably for various video sequences.