VIDEO OBJECT SEGMENTATION BY 2-D MESH-BASED MOTION ANALYSIS

Video object extraction is a key technology in content-based video coding.A novel video object extracting algorithm by two Dimensional (2-D) mesh-based motion analysis is proposed in this paper.Firstly,a 2-D mesh fitting the original frame image is obtained via feature detection algorithm. Then,higher order statistics motion analysis is applied on the 2-D mesh representation to get an initial motion detection mask.After post-processing,the final segmenting mask is quickly obtained.And hence the video object is effectively extracted.Experimental results show that the proposed algorithm combines the merits of mesh-based segmenting algorithms and pixel-based segmenting algorithms,and hereby achieves satisfactory subjective and objective performance while dramatically increasing the segmenting speed.     

A Low Power Architecture for HASM Motion Tracking

This paper proposes low power VLSI architecture for motion tracking that can be used in online video applications such as in MPEG and VRML. The proposed architecture uses a hierarchical adaptive structured mesh (HASM) concept that generates a content-based video representation. The developed architecture shows the significant reducing of power consumption that is inherited in the HASM concept. The proposed architecture consists of two units: a motion estimation and motion compensation units.The motion estimation (ME) architecture generates a progressive mesh code that represents a mesh topology and its motion vectors. ME reduces the power consumption since it (1) implements a successive splitting strategy to generate the mesh topology. The successive split allows the pipelined implementation of the processing elements. (2) It approximates the mesh nodes motion vector by using the three step search algorithm. (3) and it uses parallel units that reduce the power consumption at a fixed throughput.The motion compensation (MC) architecture processes a reference frame, mesh nodes and motion vectors to predict a video frame using affine transformation to warp the texture with different mesh patches. The MC reduces the power consumption since it uses (1) a multiplication-free algorithm for affine transformation. (2) It uses parallel threads in which each thread implements a pipelined chain of scalable affine units to compute the affine transformation of each patch.The architecture has been prototyped using top-down low-power design methodology. The performance of the architecture has been analyzed in terms of video construction quality, power and delay.     

Overlapped patch elimination algorithm for deformable mesh videocoding

Updating mesh nodal points is one of the major tasks in deformable mesh video coding. Owing to object movement and inexact motion estimation, nodal point tracking often results in overlapped patches. A noniterative nodal point processing algorithm is proposed to solve this problem. Simulation results indicate that with this overlapped patch elimination algorithm, the updated mesh can maintain a rather uniform distribution and the moving object can be traced quite well     

An effective mesh-based motion compensation technique for video coding

The block-matching algorithm is the most popular motion compensation technique in video coding. However, it cannot provide acceptable quality at very low bit rate. In this paper, a new mesh-based motion compensation method is proposed to attack the problem. First, a regular non-uniform mesh, which has regular structure with variable patch size, is presented. The patch size is varied according to motion activity of a video sequence. Next, a weighted interpolation block matching is developed to improve the estimate accuracy of displacements of grid points. It utilizes the motion correlation between a grid point and its associated patches. Finally, based on the new mesh and motion estimation scheme, an efficient motion compensation algorithm is developed. When compared to the conventional motion compensation techniques, the proposed method improves performance significantly with lower computational complexity and overhead information bits.     

MPEG-4中基于DT网格编码算法的实现

本文按照MPEG-4标准对DT网格对象编码的要求,设计了一套完整的基于DT网格的编码算法.首先提出了网格对象编码的系统框架.在此基础上,着重研究了网格运动编码,提出了一种新的三角形运动连续预测方法及相应的残差图像处理方法.最后,对整个数据结构进行转换,使之完全按照标准规定的顺序进行编码.本文算法同H.263进行了全面的比较.结果表明,DT作为MPEG-4标准中的一种编码方法,有一定的实用价值.     

基于网格模型的运动估计技术

网格模型的运动估计和跟踪在基于模型的编码方法中占有重要的地位,在现有的网格运动估计技术中,六边形匹配(Hexagonal Matching)是一种有效的网格模型运动估计算法,它采用局部最优和迭代策略相结合的方法得到较好的全局运动估计结果.六边形匹配算法由块搜索和迭代六边形匹配两步组成,同所有的基于模型的运动估计技术一样,六边形匹配算法也存在运算量大的问题难于满足实时编码需要.本文对六边形匹配算法从三个方面进行改进:用四步搜索替代全搜索、快速的纹理映射技术和有效的节点排序减少迭代次数.改进后的算法在基本上保证原算法的运动估计效率的情况下,算法的复杂度有明显的降低,使得基于模型的编码技术向实用化前进了一步.     

Semi-regular representation and progressive compression of 3-D dynamic mesh sequences.

We propose an algorithm that represents three-dimensional dynamic objects with a semi-regular mesh sequence and compresses the sequence using the spatiotemporal wavelet transform. Given an irregular mesh sequence, we construct a semi-regular mesh structure for the first frame and then map it to subsequent frames based on the hierarchical motion estimation. The regular structure of the resulting mesh sequence facilitates the application of advanced coding schemes and other signal processing techniques. To encode the mesh sequence compactly, we develop an embedded coding scheme, which supports signal-to-noise ratio and temporal scalability modes. Simulation results demonstrate that the proposed algorithm provides significantly better compression performance than the static mesh coder, which encodes each frame independently.     

网格运动分析与形态学滤波相结合的视频对象分割

提出了一种基于二维网格运动分析与改进形态学滤波空域自动分割策略相结合的视频对象时空分割算法。该算法首先利用高阶统计方法对视频图像的二维网格表示进行运动分析，快速得到前景对象区域，通过后处理有效获得前景对象运动检测掩膜。然后，用一种结合交变序列重建滤波算法和自适应阈值判别算法的改进分水岭分割策略有效获得前景对象的精确边缘。最后，用区域基时空融合算法将时域分割结果和空域分割结果结合起来提取出边缘精细的视频对象。实验结果表明，本算法综合了多种算法的优点，主客观分割效果理想。     

一种适用于遮挡问题的运动补偿帧插值方法

针对二维运动估计中常出现的遮挡问题,提出一种基于网格模型的运动补偿自适应帧插值技术,将图像帧中分为3种不同区域,根据各自的特点分别进行运动估计和运动补偿插值。在准确检测遮挡区域前提下,单独对遮挡区域处理有效地减小了运动估计误差。提出采用特征窗口运动补偿匹配的方法进行网格节点运动矢量估计,消除了传统块匹配方法导致的块效应,得到了亚像素的匹配精度。实验证明,该方法简单易行,有效地解决了遮挡区域的运动估计问题,插值图像具有较好的视觉效果。     

One-dimensional dense disparity estimation for three-dimensional reconstruction

We present a method for fully automatic three-dimensional (3D) reconstruction from a pair of weakly calibrated images in order to deal with the modeling of complex rigid scenes. A two-dimensional (2D) triangular mesh model of the scene is calculated using a two-step algorithm mixing sparse matching and dense motion estimation approaches. The 2D mesh is iteratively refined to fit any arbitrary 3D surface. At convergence, each triangular patch corresponds to the projection of a 3D plane. The proposed algorithm relies first on a dense disparity field. The dense field estimation modelized within a robust framework is constrained by the epipolar geometry. The resulting field is then segmented according to homographic models using iterative Delaunay triangulation. In association with a weak calibration and camera motion estimation algorithm, this 2D planar model is used to obtain a VRML-compatible 3D model of the scene.     

Two-dimensional mesh-based mosaic representation for manipulationof video objects with occlusion

We present a two-dimensional (2-D) mesh-based mosaic representation, consisting of an object mesh and a mosaic mesh for each frame and a final mosaic image, for video objects with mildly deformable motion in the presence of self and/or object-to-object (external) occlusion. Unlike classical mosaic representations where successive frames are registered using global motion models, we map the uncovered regions in the successive frames onto the mosaic reference frame using local affine models, i.e., those of the neighboring mesh patches. The proposed method to compute this mosaic representation is tightly coupled with an occlusion adaptive 2-D mesh tracking procedure, which consist of propagating the object mesh frame to frame, and updating of both object and mosaic meshes to optimize texture mapping from the mosaic to each instance of the object. The proposed representation has been applied to video object rendering and editing, including self transfiguration, synthetic transfiguration, and 2-D augmented reality in the presence of self and/or external occlusion. We also provide an algorithm to determine the minimum number of still views needed to reconstruct a replacement mosaic which is needed for synthetic transfiguration. Experimental results are provided to demonstrate both the 2-D mesh-based mosaic synthesis and two different video object editing applications on real video sequences.     

Affine motion compensation using a content-based mesh

A content-based approach to the design of a triangular mesh is presented, and its application to affine motion compensation is investigated. An image is first segmented into moving objects, which are then approximated with polygons. Then, a triangular mesh is generated within each polygon, thus ensuring that no triangle straddles multiple regions. Translation and affine motion parameters are determined for each triangle, using bidirectional motion estimation. Results for three test sequences demonstrate the advantages offered by the proposed mesh design method, and by the use of affine motion compensation.     

Motion Estimation Algorithms Using the Deformation of Planar Hierarchical Mesh Grid for Videoconferencing Applications at Low Bit-rate Transmission

This paper studies the issue of reducing the temporal redundancy between consecutive frames of a videoconferencing sequence at low bit-rate transmission. To overcome the drawbacks of the traditional block matching algorithm implemented in the most current video coding standards, we propose to better describe the motion of objects through the deformation of planar rectangular mesh grid adapted to the edges of the moving objects in the scene. The traditional inter coding modes are then replaced by two new classes of encoding algorithms. The first one concerns the B-frames where the problem of motion estimation is solved by a bidirectional prediction algorithm which reconstructs the quadrilateral mesh grids without any coding cost. The second class of algorithm much more complex than the first one is specific to the P-frames based on the principle of merging two hierarchical grids of reference. This algorithm addresses not only the motion estimation problem based on the adaptive quadrilateral mesh grid but also the issue of the relevant information (e.g. the positions of the nodes, the connectivity of each quadrilateral mesh of the grid and the motion compensation) to efficiently encode. The implementation of these algorithms in a complete coding scheme offers good performance compared to the H.264/AVC video coder at low bit-rate transmission.     

A fast approach for accurate content-adaptive mesh generation

Mesh modeling is an important problem with many applications in image processing. A key issue in mesh modeling is how to generate a mesh structure that well represents an image by adapting to its content. We propose a new approach to mesh generation, which is based on a theoretical result derived on the error bound of a mesh representation. In the proposed method, the classical Floyd-Steinberg error-diffusion algorithm is employed to place mesh nodes in the image domain so that their spatial density varies according to the local image content. Delaunay triangulation is next applied to connect the mesh nodes. The result of this approach is that fine mesh elements are placed automatically in regions of the image containing high-frequency features while coarse mesh elements are used to represent smooth areas. The proposed algorithm is noniterative, fast, and easy to implement. Numerical results demonstrate that, at very low computational cost, the proposed approach can produce mesh representations that are more accurate than those produced by several existing methods. Moreover, it is demonstrated that the proposed algorithm performs well with images of various kinds, even in the presence of noise.     

基于区域灰度极小值的网孔织物图像分割算法

针对网孔织物图像的对比度低和噪声点多而导致分割结果中存在网孔连在一起和残缺等问题,提出一种基于区域灰度极小值的分割算法以期提高网孔的分割精度。首先利用高斯金字塔缩放和直方图均衡化算法处理图像以增强图像的纹理轮廓和明暗对比度。然后采用一种基于区域灰度极小值的分割算法以解决仅仅依靠灰度值大小而无法正确分割网孔的问题。最后采用一种多图像融合算法以解决基于局部灰度极小值的分割算法中阈值选择困难的问题。选择多种不同光照程度的网孔织物图像进行实验,实验结果表明所提算法的分割效果良好,能够有效地解决分割结果中网孔连在一起和残缺等问题,且网孔织物的分割错误率为0.24%。     

Occlusion-adaptive, content-based mesh design and forward tracking

Two-dimensional (2-D) mesh-based motion compensation preserves neighboring relations (through connectivity of the mesh) as well as allowing warping transformations between pairs of frames; thus, it effectively eliminates blocking artifacts that are common in motion compensation by block matching. However, available 2-D mesh models, whether uniform or non-uniform, enforce connectivity everywhere within a frame, which is clearly not suitable across occlusion boundaries. To this effect, we hereby propose an occlusion-adaptive forward-tracking mesh model, where connectivity of the mesh elements (patches) across covered and uncovered region boundaries are broken. This is achieved by allowing no node points within the background to be covered (BTBC) and refining the mesh structure within the model failure (MF) region(s) at each frame. The proposed content-based mesh structure enables better rendition of the motion (compared to a uniform or a hierarchical mesh), while tracking is necessary to avoid transmission of all node locations at each frame. Experimental results show successful motion compensation and tracking.     

基于Delaunay三角形网格的彩色视频帧间编码方法

充分利用彩色视频亮度和色度分量之间的相关性,提出了一种基于DT网格的彩色视频帧间编码方案。该方案在运动估计时仅对亮度分量Y进行DT(Delaunay triangulation)描述,对亮度分量的网格节点进行连续运动估计,利用6参数方法得到三角形内像素点的运动矢量,经过相似变换得到色度分量的运动矢量。另外对残差图像进行特殊处理和编码。实验结果表明,相对于适合低码率传输的H.263编码方法,在相同的压缩比下该方案解码图像有更好的主客观质量。     

Segmentation-based compression scheme for 3D animated models

This paper presents an efficient compression algorithm for animated three-dimensional (3D) meshes. First, a segmentation approach is applied to achieve the motion estimation. The main idea is to exploit the temporal coherence of the geometry component by using the heat diffusion properties. The motion of the resulting regions is accurately described by 3D affine transforms. These transforms are computed at the first frame to match the subsequent ones. Second, in order to achieve a good compression performance, an efficient rate control mechanism is proposed to quantize the temporal prediction errors. At this stage, a rate-distortion model is used for quantizing the residual information. Comparative coding tests, for irregular 3D mesh sequences, were conducted to evaluate the coding efficiency of the proposed compression scheme. Simulation results show very promising performances.     

SNR and temporal scalable coding of 3-D mesh sequences using singular value decomposition

A signal-to-noise ratio (SNR) and temporal scalable coding algorithm for 3-D mesh sequences using singular value decomposition (SVD) is proposed in this work. The proposed algorithm employs SVD to represent a mesh sequence with a small number of basis vectors, and encodes those basis vectors with a bit plane coder. We analytically derive the contribution of each bit plane to the reconstructed mesh quality, and transmit the bit planes in the decreasing order of their amounts of contribution. As the decoder receives more bit planes, it reconstructs higher quality mesh sequences progressively. Moreover, we develop a temporal prediction mode to improve the rate–distortion (R–D) performance further, which also supports temporal scalability. Simulation results demonstrate that the proposed algorithm yields significantly better R–D performance than conventional SVD-based coders.