Hardware design focusing in the tradeoff cost versus quality for the H.264/AVC fractional motion estimation targeting high definition videos

This article presents an architecture for the fractional motion estimation (FME) of the H.264/AVC video coding standard focusing in a good tradeoff between the hardware cost and the video quality. The support to FME guarantees a high quality in the motion estimation process. The applied algorithmic simplifications together with the multiplierless implementation and with a well balanced pipeline allow a low cost and a high throughput solution. The architecture was also designed to avoid redundant external memory accesses when computing the FME. The design was divided in two main modules: integer motion estimation (with diamond search algorithm) and fractional refinement (half-pixel and quarter-pixel interpolation and search). The designed architecture was described in VHDL and synthesized to an Altera Stratix III FPGA. The architecture is able to reach 260 MHz when running in the target FPGA. In worst case scenario, this operation frequency allows a processing rate of 43 HD 1080p (1,920 × 1,080 pixels) frames per second, surpassing the requirements for real time processing. In comparison to related works, the developed architecture was able to achieve a good tradeoff among hardware costs, video quality and processing rate.     

Fast multi-frame motion estimation for H264/AVC system

Advanced video compression standard, H264/AVC, with multi-frame motion estimation, can offer better motion-compensation than the previous coding standards. However, the implementation of real-time multi-frame estimation for an H264/AVC system is difficult due to heavy computations. In this paper, a fast algorithm is proposed in an effort to reduce the searching computation for motion estimation with five reference frames. The fast multi-frame motion estimation consists of the adaptive full-search, three-step search, and diamond search methods using the content adaptive control process. Efficient control flow is proposed to select the searching algorithm dependent on video features. The adaptive algorithm can achieve better rate-distortion and lower computation for H264/AVC coding. The experiments indicate that the speed-up is 6–15 times compared with the full search method, while the image quality slightly degrades.     

基于H.264的内容自适应分数像素运动估计算法

分数像素精确运动估计的改进是整个运动估计模块优化的关键,本文提出了基于H.264的内容自适应分数像素运动估计算法。首先,提出基于平坦区域宏块预测的无效分数像素运动矢量(MV)搜索省略算法(SMBP);然后,改进H.264采用的基于中心的分数像素搜索算法(CBFPS),提出基于预测矢量的增强型菱形模板(EDSP)搜索算法。实验结果表明,内容自适应分数像素运动估计算法比分数像素全搜索算法(FFPS)在峰值信噪比(PSNR)有微小降低(0.095～0.209 dB)的情况下,平均减少了75.6%的分数像素搜索点,整个运动估计模块平均节省了38.5%的计算量。     

结合空时域下采样与重建的H.264/AVC压缩性能优化

为提高H.264/AVC标准在带宽资源严重受限时的压缩效率,采用空时域相结合的编码思路,提出了一种基于运动检测的自适应抽帧方法,并结合空域下采样与重建研究了一种改进的H.264/AVC压缩性能优化框架。在编码端,原视频先空域下采样以减少空间分辨率,然后根据视频运动特征,采用不同抽帧模式自适应地降低帧率,再经H.264/AVC编码,有效降低了编码码率。在解码端,解码视频则采用与抽帧模式相对应的运动估计与补偿插帧方法重建出抽取帧,再利用超分辨率重建技术将视频恢复到原空间分辨率。实验结果表明,所提方法在低码率段的视频压缩性能优于H.264/AVC标准编解码及相关文献方法。     

用于H.264/AVC的D级数据重用整数运动估计VLSI结构

本文提出了一种用于H.264/AVC的D级数据重用整数运动估计VLSI结构.提出的结构是在一种固定块尺寸运动估计VLSI结构基础上,利用交叉网络实现变块尺寸的计算,使用多bank的存储器组织方式,使片上存储器的读写规则简单,易于处理不同搜索范围和不同尺寸的视频的运动估计.提出的运动估计结构用Verilog HDL描述,使用HJTC 0.18μm工艺,用Synopsys DC做了逻辑综合.相比现有结构,该结构由于增加片上存储器,因此数据重用率高,大大降低了存储带宽需求;另外数据吞吐率高,能够满足高性能视频编码需求.     

一种高效的基于H.264/AVC压缩域信息的全局运动估计方法

本文提出了一种基于H.264/AVC压缩域的高效全局运动估计算法.由于H.264采用了多种新的视频压缩编码技术,使得其压缩码流的运动矢量(MV)场中包含大量噪声运动矢量,可参与全局运动估计的运动矢量相对较少.噪声运动矢量这里指的是与全局运动不相符的运动矢量.为了降低噪声运动矢量的影响、提高全局运动估计的精度和效率,在全...     

H.264/AVC中基于全零块检测的运动估计快速算法

全零块检测是面向低比特率的视频编码器常用优化方法之一.特别是与运动估计相结合,可以有效的减少编码器的计算复杂性.本文根据H.264/AVC中整数变换的特点,给出了相应的全零块检测门限,提出了一种基于全零块检测的运动搜索提前中止准则.针对H.264/AVC多编码模式的特点,进一步将全零块检测用于H.264/AVC中多种编码模式的选择,有效的提高了运动估计的效率.利用这种方法,在有效减少编码器的计算复杂性,提高H.264/AVC软件编码器编码效率的同时,可以保持比特率和图像质量基本不变.     

Bit-by-Bit Pipelined and Hybrid-Grained 2D Architecture for Motion Estimation of H.264/AVC

In H.264/AVC, the motion estimation (ME) routine supports variable block size and involves highly parallel sum of absolute difference (SAD) computations. In this study, we introduce a bit serial hybrid-grained processing element (PE) based 2D architecture that has both early termination and intensive data reuse capabilities. PEs operate on most significant bit-first arithmetic for early termination and the 2D architecture enables on-chip data reuse between neighboring PEs in a bit-by-bit pipelined fashion. Hybrid-grained PEs reduce the hardware overhead of conventional adder tree structures used for implementing the variable block size ME. Our design reduces the gate count by 7x compared to its ASIC counterpart, operates at a comparable frequency while sustaining 30 fps and 60 fps; and outperforms bit parallel and bit serial architectures in terms of throughput and performance per gate for various video formats.     

General Architecture for MPEG-2/H.263/H.264/AVC to H.264/AVC Intra Frame Transcoding

The latest international video-coding standard H.264/AVC significantly achieves better coding performance compared to prior video coding standards such as MPEG-2 and H.263, which have been widely used in today’s digital video applications. To provide the interoperability between different coding standards, this paper proposes an efficient architecture for MPEG-2/H.263/H.264/AVC to H.264/AVC intra frame transcoding, using the original information such as discrete cosine transform (DCT) coefficients and coded mode type. Low-frequency components of DCT coefficients and a novel rate distortion cost function are used to select a set of candidate modes for rate distortion optimization (RDO) decision. For H.263 and H.264/AVC, a mode refinement scheme is utilized to eliminate unlikely modes before RDO mode decision, based on coded mode information. The experimental results, conducted on JM12.2 with fast C8MB mode decision, reveal that average 58%, 59% and 60% of computation (re-encoding) time can be saved for MPEG-2, H.263, H.264/AVC to H.264/AVC intra frame transcodings respectively, while preserving good coding performance when compared with complex cascaded pixel domain transcoding (CCPDT); or average 88% (a speed up factor of 8) when compared with CCPDT without considering fast C8MB. The proposed algorithm for H.264/AVC homogeneous transcoding is also compared to the simple cascaded pixel domain transcoding (with original mode reuse). The results of this comparison indicate that the proposed algorithm significantly outperforms the mode reuse algorithm in coding performance, with only slightly higher computation.     

Efficient coding of motion vector and mode information for H.264/AVC

This paper aims to reduce the size of header in H.264; therefore we only focus on compressing the larger header data into small one. H.264/AVC is the newest video coding standard, which adopts some new techniques to improve the compression efficiency. However, H.264 gets larger header than those of previous standards because the techniques exploit in detail the content of input image. The most parts of header are composed of macroblock type (mb_type) and motion vector difference (mvd), and there are redundancies in the header of H.264. Especially, when mode 8 is selected during mode decision at macroblock level, all four sub-macroblock data are transferred into bitstream. Building the idea above, we propose the scheme using the type code and quadtree to solve the problems. Experimental results show that our algorithm achieved lower total number of encoded bits over JM14.2 up to 31.87% bit reduction at QP 40 on average.     

Gravity direction-based ultra-fast intraprediction algorithm for H.264/AVC video coding

The H.264/AVC video coding standard uses in intraprediction, 9 directional modes for 4 × 4 and 8 × 8 luma blocks, and 4 directional modes for 16 × 16 luma macroblocks, and 8 × 8 chroma blocks. The use of the variable block size and multiple modes in intraprediction makes the intracoding of H.264/AVC very efficient compared with other compression standards; however, computational complexity is increased significantly. In this paper, we propose a fast mode selection algorithm for intracoding. This algorithm is based on the vector of the block’s gravity center whose direction is used to select the best candidate prediction mode for intracoding. On this basis, only a small number of intraprediction modes are chosen for rate distortion optimization (RDO) calculation. Different video sequences are used to test the performance of proposed method. The simulation results show that the proposed algorithm increases significantly the speed of intracoding with negligible loss of peak signal-to-noise ratio quality.     

Geometrically determining the leaky bucket parameters for video streaming over constant bit-rate channels

For streaming of pre-encoded bitstreams over constant bit rate (CBR) channels, the channel bandwidth, the receiver buffer capacity as well as the latency requirement vary greatly from application to application. In this paper, we attempt to determine the minimum buffer size and the minimum start-up delay required for streaming a pre-encoded bitstream over CBR channels at any specific bit rate. The proposed method employs geometric operations to derive the optimal determination for low or high bit rates and sub-optimal determination for medium bit rates. The algorithm developed requires little extra information from the encoder and is easy to implement. Our algorithm is implemented in a H.264/AVC video encoder and its performance is compared with that of H.264/AVC hypothetical reference decoder. Our approach provides new theoretical insight and an excellent solution for determining the leaky bucket parameters for video streaming over CBR channels.     

Efficient Level and Zero Coding Methods for H.264/AVC Lossless Intra Coding

Since H.264/AVC was designed mainly for lossy video coding, the entropy coding methods in H.264/AVC are not appropriate for lossless video coding. Based on statistical differences of residual data in lossy and lossless coding, we develop efficient level and zero coding methods. Therefore, we design an improved context-based adaptive variable length coding (CAVLC) scheme for lossless intra coding by modifying the relative entropy coding parts in H.264/AVC. Experimental results show that the proposed method provides approximately 6.8% bit saving, compared with the H.264/AVC FRExt high profile.      

Joint depth-motion dense estimation for multiview video coding

The multiview video coding (MVC) extension of H.264/MPEG-4 AVC [1] is one of the most promising visual encoders for three-dimensional television and free viewpoint video applications. In this paper, we propose a joint dense motion/disparity estimation algorithm, designed to replace the classical temporal/inter-view unit within MVC, which uses a block-based motion/disparity estimation. The motion vector fields and the disparity vector fields are therefore simultaneously derived using the stereo-motion consistency constraint in a set theoretic convex optimization framework. The obtained displacement vector fields are then jointly segmented by minimizing a rate-distortion cost function, in line with the multiple reference frame strategy used in H.264/MPEG-4 AVC. Experimental results demonstrate the benefits of the proposed method compared to the separated dense estimation scheme or the block-based estimation technique.     

一种基于H.264/AVC的高效块匹配搜索算法

本文针对H.264/AVC的编码特点,提出了一种利用时空域运动相关性的快速块匹配搜索算法.该算法充分利用了视频序列的运动程度与宏块编码模式间的关联特性以及运动矢量的统计特征,明显减少了运动估计的搜索复杂度.实验表明,本文方法的搜索速度分别比FS和DS算法平均提高了77.96%和32.19%;重建图像的PSNR比DS算法平均提高了0.06dB,更接近FS算法的编码质量.     

Mode-Dependent Templates and Scan Order for H.264/AVC-Based Intra Lossless Coding

In H.264/advanced video coding (AVC), lossless coding and lossy coding share the same entropy coding module. However, the entropy coders in the H.264/AVC standard were original designed for lossy video coding and do not yield adequate performance for lossless video coding. In this paper, we analyze the problem with the current lossless coding scheme and propose a mode-dependent template (MD-template) based method for intra lossless coding. By exploring the statistical redundancy of the prediction residual in the H.264/AVC intra prediction modes, more zero coefficients are generated. By designing a new scan order for each MD-template, the scanned coefficients sequence fits the H.264/AVC entropy coders better. A fast implementation algorithm is also designed. With little computation increase, experimental results confirm that the proposed fast algorithm achieves about 7.2% bit saving compared with the current H.264/AVC fidelity range extensions high profile.     

An efficient VLSI processor chip for variable block size integer motion estimation in H.264/AVC

Motion estimation (ME) is the most critical component of a video coding standard. H.264/AVC adopts the variable block size motion estimation (VBSME) to obtain excellent coding efficiency, but the high computational complexity makes design difficult. This paper presents an effective processor chip for integer motion estimation (IME) in H264/AVC based on the full-search block-matching algorithm (FSBMA). It uses architecture with a configurable 2D systolic array to obtain a high data reuse of search area. This systolic array supports a three-direction scan format in which only one row of pixels is changed between the two adjacent subblocks, thus reducing the memory accesses and saving clock cycles. A computing array of 64 PEs calculates the SAD of basic 4×4 subblocks and a modified Lagrangian cost is used as matching criterion to find the best 41 variable-size blocks by means of a tree pipeline parallel architecture. Finally, a mode decision module uses serial data flow to find the best mode by comparing the total minimum Lagrangian costs. The IME processor chip was designed in UMC 0.18 μm technology resulting in a circuit with only 32.3 k gates and 6 RAMs (total 59kBits on-chip memory). In typical working conditions (25 °C, 1.8 V), a clock frequency of 300 MHz can be estimated with a processing capacity for HDTV (1920×1088 @ 30 fps) and a search range of 32×32.     

Fast mode decision algorithm for intra prediction in H.264/AVC with integer transform and adaptive threshold

Compared with other existing video coding standards, H.264/AVC can achieve a significant improvement in compression performances. A robust criterion named the rate distortion optimization (RDO) is employed to select the optimal coding modes and motion vectors for each macroblock (MB), which achieves a high compression ratio while leading to a great increase in the complexity and computational load unfortunately. In this paper, a fast mode decision algorithm for H.264/AVC intra prediction based on integer transform and adaptive threshold is proposed. Before the intra prediction, integer transform operations on the original image are executed to find the directions of local textures. According to this direction, only a small part of the possible intra prediction modes are tested for RDO calculation at the first step. If the minimum mean absolute error (MMAE) of the reconstructed block corresponding to the best mode is smaller than an adaptive threshold which depends on the quantization parameter (QP), the RDO calculation is terminated. Otherwise, more possible modes need to be tested. The adaptive threshold aims to balance the compression performance and the computational load. Simulation results with various video sequences show that the fast mode decision algorithm proposed in this paper can accelerate the encoding speed significantly only with negligible PSNR loss or bit rate increment. This work is supported in part by China National Natural Science Foundation (CNSF) under Project No.60572045, the Ministry of Education of China Ph.D. Program Foundation under Project No.20050698033, and by a Cooperation Project (2005.7– 2007.7) with Microsoft Research Asia.     

基于H.264编码的多屏共享系统研究

H.264/AVC是一种由ITU-T视频编码专家组合ISO/IEC JTC1动态图像专家组联合提出的高度压缩视频编码器标准。然而H.264/AVC编码器较高的运算复杂度提高了多屏共享系统的延迟时间。H.264/AVC由多种开源的实现,其中X264因简单高效而得到广泛的应用。在此对多频共享系统的关键技术进行实现,分析X264编码器提供的运动估计算法并且提出一种优化的算法。实验表明,新的算法提高了编码的速度、减少了系统延迟时间,同时视频质量几乎没有产生损失。     

H.264编码器中1/4像素精度插值算法的VLSI实现

H.264视频编码标准中引入了1/4像素精度插值算法,大大提高了压缩效率,但同时使运算复杂度增加、存储带宽增大。针对以上问题,从运动估计的角度出发,采用一步插值法和数据复用技术,可使带宽减少26%,处理周期可减少45%;设计了相应的硬件结构:采用了5级流水线实现一步插值算法,通过输入缓冲单元实现了参考数据的复用;针对插值过程中产生的大量数据,采用乒乓操作结构,保证数据及时传递。该结构可以显著降低带宽,提高吞吐率,完全可以应用于实时编码器中。