Closed-form connectivity-preserving solutions for motioncompensation using 2-D meshes

Motion compensation using two-dimensional (2-D) mesh models requires computation of the parameters of a spatial transformation for each mesh element (patch). It is well known that the parameters of an affine (bilinear or perspective) mapping can be uniquely estimated from three (four) point correspondences (at the vertices of a triangular or quadrilateral mesh element). On the other hand, overdetermined solutions using more than the required minimum number of point correspondences provide increased robustness against correspondence-estimation errors, however, this necessitates special consideration to preserve mesh-connectivity. This paper presents closed-form, overdetermined solutions for least squares estimation of affine motion parameters for a triangular mesh, which preserve mesh-connectivity using patch-based or node-based connectivity constraints. In particular, four new algorithms are presented: patch-constrained methods using point correspondences or spatio-temporal intensity gradients, and node-constrained methods using point correspondences or spatio-temporal intensity gradients. The methods using point correspondences can be viewed as postprocessing of a dense motion field for best representation in terms of a set of irregularly spaced samples. The methods that are based on spatio-temporal intensity gradients offer closed-form solutions for direct estimation of the best node-point motion vectors (equivalently the best transformation parameters). We show that the performance of the proposed closed-form solutions are comparable to those of the alternative search-based solutions at a fraction of the computational cost.     

Incompressible deformation estimation algorithm (IDEA) from tagged MR images

Measuring the 3D motion of muscular tissues, e.g., the heart or the tongue, using magnetic resonance (MR) tagging is typically carried out by interpolating the 2D motion information measured on orthogonal stacks of images. The incompressibility of muscle tissue is an important constraint on the reconstructed motion field and can significantly help to counter the sparsity and incompleteness of the available motion information. Previous methods utilizing this fact produced incompressible motions with limited accuracy. In this paper, we present an incompressible deformation estimation algorithm (IDEA) that reconstructs a dense representation of the 3D displacement field from tagged MR images and the estimated motion field is incompressible to high precision. At each imaged time frame, the tagged images are first processed to determine components of the displacement vector at each pixel relative to the reference time. IDEA then applies a smoothing, divergence-free, vector spline to interpolate velocity fields at intermediate discrete times such that the collection of velocity fields integrate over time to match the observed displacement components. Through this process, IDEA yields a dense estimate of a 3D displacement field that matches our observations and also corresponds to an incompressible motion. The method was validated with both numerical simulation and in vivo human experiments on the heart and the tongue.     

基于27邻域网格的医疗图像三维重建

为实现医疗图像三维重建中的网格简化,提出一种基于27邻域网格建立三角形网格拓扑的新方法。首先利用移动立方体(MC)算法从一组计算机层析(CT)图像序列中提取三维重建体数据,将二维图像转化为由一组三角形面片组成的三维模型网;然后利用27邻域网格算法对此三角形网进行空间扫描,构建出三维模型的网格拓扑;最后利用二次误差测度(QEM)算法对所得三维模型拓扑进行简化,实现医疗图像三维模型动态可视化操作。通过使用Visual C++软件平台和OPENGL库对医疗图像进行三维重建,重建效果表明,本文方法的计算效率要优于传统的循环迭代方法;与传统的建立网格拓扑方法相比,本文方法具有算法简单、速度快、运算复杂度与数据量呈线性增长的优点。     

基于STL转换为X3D格式的研究与实现

通过对STL三角网格模型和X3D语法规范的分析与研究,根据X3D对三角形的相关定义和描述,提出了一种将STL三角网格模型转换为X3D格式的算法。该算法在实现将STL转化为X3D的同时,又完成了对STL文件中冗余数据的处理,使得转换后的X3D文件大小相当于原来STL文件大小的1/7左右,非常有利于三角网格模型在网络上的传输和操作。最后,以Java作为开发系统平台,运用Java3D及其装载器组件实现了转换后的X3D的装载和浏览,从而使STL文件可以在Internet环境下实现产品信息共享和可视化的操作。     

基于主动网格的运动补偿

提出了基于图像内容特征的主动网格生成方法和网格节点运动估计方法，网格的边界与运动场景的边界有很好的一致性，避免了相同三角形小片内存在不同的运动物体，克服这种情况下运动估计失效的问题。块匹配的运动补偿方法的缺陷在于块效应，基于主动网格的运动补偿消除了块效应和边界不连续现象，运动场的估计与实际运动场有很好的近似，对局部运动也有更好的逼近。     

Conditional Random Field Tracking Model Based on a Visual Long Short Term Memory Network

In dense pedestrian tracking, frequent object occlusions and close distances between objects cause difficulty when accurately estimating object trajectories. In this study, a conditional random field tracking model is established by using a visual long short term memory network in the three-dimensional (3D) space and the motion estimations jointly performed on object trajectory segments. Object visual field information is added to the long short term memory network to improve the accuracy of the motion related object pair selection and motion estimation. To address the uncertainty of the length and interval of trajectory segments, a multimode long short term memory network is proposed for the object motion estimation. The tracking performance is evaluated using the PETS2009 dataset. The experimental results show that the proposed method achieves better performance than the tracking methods based on the independent motion estimation.     

Affine motion compensation using a content-based mesh

A content-based approach to the design of a triangular mesh is presented, and its application to affine motion compensation is investigated. An image is first segmented into moving objects, which are then approximated with polygons. Then, a triangular mesh is generated within each polygon, thus ensuring that no triangle straddles multiple regions. Translation and affine motion parameters are determined for each triangle, using bidirectional motion estimation. Results for three test sequences demonstrate the advantages offered by the proposed mesh design method, and by the use of affine motion compensation.     

Estimation of depth fields suitable for video compression based on3-D structure and motion of objects

Intensity prediction along motion trajectories removes temporal redundancy considerably in video compression algorithms. In three-dimensional (3-D) object-based video coding, both 3-D motion and depth values are required for temporal prediction. The required 3-D motion parameters for each object are found by the correspondence-based E-matrix method. The estimation of the correspondences-two-dimensional (2-D) motion field-between the frames and segmentation of the scene into objects are achieved simultaneously by minimizing a Gibbs energy. The depth field is estimated by jointly minimizing a defined distortion and bit-rate criterion using the 3-D motion parameters. The resulting depth field is efficient in the rate-distortion sense. Bit-rate values corresponding to the lossless encoding of the resultant depth fields are obtained using predictive coding; prediction errors are encoded by a Lempel-Ziv algorithm. The results are satisfactory for real-life video scenes.     

基于多分辨率格网的三维物体识别方法

本文首先提出了一种改进的三维物体表达方法,它将一个三维物体表面网格与其它表面网格的几何关系表示为一个二维矩阵,称为距离角度图.这种表达能够描述任意形态物体,抑制杂乱背景和遮挡,几何意义直观,且适应不同分辨率、非规则的三角格网.然后,以这种表达方法为基础,本文阐述了一种基于多分辨率格网的,由粗到精的三维物体识别方法.它先在场景和模型的低分辨率格网上进行粗匹配以得到模型候选集合,之后在已匹配网格的高分辨率格网邻域上筛选模型候选集合,最后综合考虑多个网格对应的模型候选以得到最终模型候选的确认和验证.这种识别方法具有运算量小,准确可靠等优点,实验证明该方法正确有效.     

Segmentation-based compression scheme for 3D animated models

This paper presents an efficient compression algorithm for animated three-dimensional (3D) meshes. First, a segmentation approach is applied to achieve the motion estimation. The main idea is to exploit the temporal coherence of the geometry component by using the heat diffusion properties. The motion of the resulting regions is accurately described by 3D affine transforms. These transforms are computed at the first frame to match the subsequent ones. Second, in order to achieve a good compression performance, an efficient rate control mechanism is proposed to quantize the temporal prediction errors. At this stage, a rate-distortion model is used for quantizing the residual information. Comparative coding tests, for irregular 3D mesh sequences, were conducted to evaluate the coding efficiency of the proposed compression scheme. Simulation results show very promising performances.     

Estimation of dense image flow fields in fluids

The estimation of flow fields from time sequences of satellite imagery has a number of important applications. For visualization of cloud or sea ice movements in sequences of crude temporal sampling, a satisfactory nonblurred temporal interpolation can be performed only when the flow field or an estimate thereof is known. Estimated flow fields in weather satellite imagery might also be used on an operational basis as inputs to short-term weather prediction. The authors describe a method for the estimation of dense flow fields. Local measurements of motion are obtained by analysis of the local energy distribution, which is sampled by using a set of three-dimensional (3D) spatio-temporal filters. The estimated local energy distribution also allows the authors to compute a confidence measure of the estimated local normal flow. The algorithm, furthermore, utilizes Markovian random fields in order to integrate the local estimates of normal flows into a dense flow field by using measures of spatial smoothness. To obtain smoothness, the authors will constrain first-order derivatives of the flow field. The performance of the algorithm is illustrated by the estimation of the flow fields corresponding to a sequence of Meteosat thermal images. The estimated flow fields are used in a temporal interpolation scheme     

基于模糊量化和2 bit深度像素的运动估计算法

提出了一种2 bit深度像素的运动估计算法。首先,将像素深度的降采样过程形式化为区间分划和区间映射2个步骤,其中前者为多对一映射,决定着运动估计性能,后者为一一映射;其次,提出一种非均匀量化方法求解区间分划的3个初始阈值,并利用隶属度函数对初始阈值细化,从而克服信号噪声等因素导致的初始阈值周围像素值的误匹配;再次,讨论了适用于2 bit深度像素运动估计的误差度量准则,进而提出了基于模糊量化和2 bit深度像素的运动估计算法;最后,借助信号自相关函数,建立比特深度转换误差—运动向量精度模型来估计该算法所能达到的预测精度。实验结果证明,对于多种类型的视频序列,尤其是场景细节和物体运动比较复杂者,该算法始终能保持较高的估计精度,运动补偿的平均峰值信噪比较之传统2 bit深度像素的运动估计提高0.27 dB。     

Precision of direction of arrival (DOA) estimation using novel three dimensional array geometries

Numerous methods for direction of arrival (DOA) estimation, used in smart antennas have been already reported in previous studies. The precision of DOA estimation depends on the choice of the algorithm and the geometrical configuration of the antenna array. In this work, the performance of new geometrical configurations, i.e. 2D with equal area and 3D with equal volume including circular, square, triangular, hexagonal and star geometries, with equal number of antenna elements, are examined and compared to each other to find the most proper geometry. Monte-Carlo simulations are performed to evaluate the DOA precision of the proposed arrays using the MUSIC algorithm. It is shown that in three cases of comparison including 2D geometries, 3D geometries and 3D rotated geometries, with the star and triangular configurations one achieves better resolution in DOA estimation. It is also revealed that the rotated configurations show lower estimation error compared to normal configurations.     

Motion Estimation Algorithms Using the Deformation of Planar Hierarchical Mesh Grid for Videoconferencing Applications at Low Bit-rate Transmission

This paper studies the issue of reducing the temporal redundancy between consecutive frames of a videoconferencing sequence at low bit-rate transmission. To overcome the drawbacks of the traditional block matching algorithm implemented in the most current video coding standards, we propose to better describe the motion of objects through the deformation of planar rectangular mesh grid adapted to the edges of the moving objects in the scene. The traditional inter coding modes are then replaced by two new classes of encoding algorithms. The first one concerns the B-frames where the problem of motion estimation is solved by a bidirectional prediction algorithm which reconstructs the quadrilateral mesh grids without any coding cost. The second class of algorithm much more complex than the first one is specific to the P-frames based on the principle of merging two hierarchical grids of reference. This algorithm addresses not only the motion estimation problem based on the adaptive quadrilateral mesh grid but also the issue of the relevant information (e.g. the positions of the nodes, the connectivity of each quadrilateral mesh of the grid and the motion compensation) to efficiently encode. The implementation of these algorithms in a complete coding scheme offers good performance compared to the H.264/AVC video coder at low bit-rate transmission.     

非刚性稠密匹配大位移运动光流估计

光流场是目标检测,无人机定位等众多计算机视觉任务的重要基础.本文针对非刚性大位移运动等困难运动类型图像序列光流计算的准确性与鲁棒性问题,提出一种基于非刚性稠密匹配的TV-L1（Total Variational with L1 norm,TV-L1）大位移光流计算方法.首先,使用非刚性稠密块匹配计算图像序列初始最近邻域场,其次根据图像相邻块区域的相似性消除初始最近邻域场中的非一致性区域以得到准确的图像最近邻域场.然后,在图像金字塔分层计算框架下,将图像最近邻域场引入基于非局部约束的TV-L1光流估计模型,通过Quadratic Pseudo-Boolean Optimization（QPBO）融合算法在金字塔分层图像光流计算时对TV-L1模型光流估计进行大位移运动补偿.最后,采用标准测试图像序列对本文方法和当前代表性的变分方法LDOF（Large Displacement Optical Flow,LDOF）、Classic+NL、NNF（Nearest Neighbor Fields,NNF）以及深度学习方法FlowNet2.0进行对比分析.实验结果表明,本文方法能有效提高非刚性运动、大位移运动以及运动遮挡等困难运动类型光流估计的精度与鲁棒性.     

A Low Power Architecture for HASM Motion Tracking

This paper proposes low power VLSI architecture for motion tracking that can be used in online video applications such as in MPEG and VRML. The proposed architecture uses a hierarchical adaptive structured mesh (HASM) concept that generates a content-based video representation. The developed architecture shows the significant reducing of power consumption that is inherited in the HASM concept. The proposed architecture consists of two units: a motion estimation and motion compensation units.The motion estimation (ME) architecture generates a progressive mesh code that represents a mesh topology and its motion vectors. ME reduces the power consumption since it (1) implements a successive splitting strategy to generate the mesh topology. The successive split allows the pipelined implementation of the processing elements. (2) It approximates the mesh nodes motion vector by using the three step search algorithm. (3) and it uses parallel units that reduce the power consumption at a fixed throughput.The motion compensation (MC) architecture processes a reference frame, mesh nodes and motion vectors to predict a video frame using affine transformation to warp the texture with different mesh patches. The MC reduces the power consumption since it uses (1) a multiplication-free algorithm for affine transformation. (2) It uses parallel threads in which each thread implements a pipelined chain of scalable affine units to compute the affine transformation of each patch.The architecture has been prototyped using top-down low-power design methodology. The performance of the architecture has been analyzed in terms of video construction quality, power and delay.     

Motion-compensated compression of 3D animation models

A new algorithm is proposed to compress animated 3D mesh models. First, an input mesh model is partitioned into segments, and each segment is motion compensated from that of the previous time instance. Then, the motion residuals are effectively compressed by using a transform coding method. It is shown that the proposed algorithm yields a much higher compression ratio than the MPEG-4 codec     

Rigid and non-rigid 3D motion estimation from multiview image sequences

Multiview image sequence processing has been the focus of considerable attention in recent literature. This paper presents an efficient technique for object-based rigid and non-rigid 3D motion estimation, applicable to problems occurring in multiview image sequence coding applications. More specifically, a neural network is formed for the estimation of the rigid 3D motion of each object in the scene, using initially estimated 2D motion vectors corresponding to each camera view. Non-linear error minimization techniques are adopted for neural network weight update. Furthermore, a novel technique is also proposed for the estimation of the local non-rigid deformations, based on the multiview camera geometry. Experimental results using both stereoscopic and trinocular camera setups illustrate and evaluate the proposed scheme.