3D anatomical shape atlas construction using mesh quality preserved deformable models

3D anatomical shape atlas construction has been extensively studied in medical image analysis research, owing to its importance in model-based image segmentation, longitudinal studies and populational statistical analysis, etc. Among multiple steps of 3D shape atlas construction, establishing anatomical correspondences across subjects, i.e., surface registration, is probably the most critical but challenging one. Adaptive focus deformable model (AFDM) [1] was proposed to tackle this problem by exploiting cross-scale geometry characteristics of 3D anatomy surfaces. Although the effectiveness of AFDM has been proved in various studies, its performance is highly dependent on the quality of 3D surface meshes, which often degrades along with the iterations of deformable surface registration (the process of correspondence matching). In this paper, we propose a new framework for 3D anatomical shape atlas construction. Our method aims to robustly establish correspondences across different subjects and simultaneously generate high-quality surface meshes without removing shape details. Mathematically, a new energy term is embedded into the original energy function of AFDM to preserve surface mesh qualities during deformable surface matching. More specifically, we employ the Laplacian representation to encode shape details and smoothness constraints. An expectation–maximization style algorithm is designed to optimize multiple energy terms alternatively until convergence. We demonstrate the performance of our method via a set of diverse applications, including a population of sparse cardiac MRI slices with 2D labels, 3D high resolution CT cardiac images and rodent brain MRIs with multiple structures. The constructed shape atlases exhibit good mesh qualities and preserve fine shape details. The constructed shape atlases can further benefit other research topics such as segmentation and statistical analysis.     

基于特征的自适应正则化配准算法*

Andriy Myronenko提出了一种自适应正则化的方法并将其应用于非刚性图像的配准,该方法在配准速度和配准精确度方面都取得了比较好的效果。但该方法对变形场初始值比较敏感,选择不当则会陷入局部极小值而不能得到理想的配准结果。为了使原始算法得到更广泛的应用,本文引入了基于特征点的粗配准方法,得到了与真实变形场更加接近的初始变形场,从而摆脱了局部极小值的困扰,得到了正确的配准结果。实验证明,改进后的算法在应用范围和配准精度上都有了提高。     

基于深度卷积特征光流的形变医学图像配准算法

光流法是一种基于光流场模型的重要而有效的形变配准算法。针对现有光流法所用特征质量不高使得配准结果不够准确的问题，将深度卷积神经网络特征和光流法相结合，提出了基于深度卷积特征光流（DCFOF）的形变医学图像配准算法。首先利用深度卷积神经网络稠密地提取图像中每个像素所在图像块的深度卷积特征，然后基于固定图像和浮动图像间的深度卷积特征差异求解光流场。通过提取图像的更为精确和鲁棒的深度学习特征，使求得的光流场更接近真实形变场，提升了配准精度。实验结果表明，所提算法能够更有效地解决形变医学图像配准问题，其配准精度优于Demons算法、尺度不变特征变换(SIFT) Flow算法以及医学图像专业配准软件Elastix。     

Monocular 3D tracking of deformable surfaces using sequential second order cone programming

Reconstructing structures of deformable objects from monocular image sequences is important for applications like visual servoing and augmented reality. In this paper, we propose a method to recover 3D shapes of deformable surfaces using sequential second order cone programming (SOCP). The key of our approach is to represent the surface as a triangulated mesh and introduce two sets of constraints, one for model-to-image keypoint correspondences which are SOCP constraints, another for retaining the original lengths of the mesh edges which are non-convex constraints. In the process of tracking, the surface structure is iteratively updated by solving sequential SOCP feasibility problems in which the non-convex constraints are replaced by a set of convex constraints over a local convex region. The shape constraints used in our approach is more generic than previous methods, that enables us to reliably recover surface shapes with smooth, sharp and other complex deformations. The capability and efficiency of our approach are evaluated quantitatively with synthetic image sequences and qualitatively with real image sequences.     

基于肺部不连续运动的全变差正则化动态配准研究

胸部放射治疗计划的一个先决条件就是精确建模胸部器官的呼吸运动。虽然4D成像技术的出现使得呼吸过程中肺部能够可视化,但获得不同呼吸阶段体素的精确对应依然是一个充满挑战的难题。本文主要研究肺部放射治疗中的弹性配准问题,采用基于全变差(Total Variation,TV)正则化的快速自由形变(Fast Free-form Deformable,FFD)模型弹性配准方法处理胸部器官运动的不连续性。我们将配准问题建模为求解一个包含图像相似性测度与光滑性测度的能量泛函的最小值,通过变分法,将该能量泛函极小化问题转化为求解对应的Euler-Largange偏微分方程,利用有限差分法、三线性插值、牛顿迭代法,迭代求解出偏移场。我们在胸部二维CT影像、肺部三维CT影像、腹部三维MRI影像上对所提算法进行了验证,结果表明该算法在处理器官不连续性运动的配准方面比传统的最小二乘化优化算法以及基于2范数正则化的算法更具有优越性。该算法结合了TV norm保持图像边缘的能力以及FFD自由度高这两个优点,精度高、速度快、且全自动。随着现代4D放射治疗的影响日益增大,该方法能够在以后的临床中发挥作用。     

Topological active volumes: A topology-adaptive deformable model for volume segmentation

This paper proposes a generic methodology for segmentation and reconstruction of volumetric datasets based on a deformable model, the topological active volumes (TAV). This model, based on a polyhedral mesh, integrates features of region-based and boundary-based segmentation methods in order to fit the contours of the objects and model its inner topology. Moreover, it implements automatic procedures, the so-called topological changes, that alter the mesh structure and allow the segmentation of complex features such as pronounced curvatures or holes, as well as the detection of several objects in the scene. This work presents the TAV model and the segmentation methodology and explains how the changes in the TAV structure can improve the adjustment process. In particular, it is focused on the increase of the mesh density in complex image areas in order to improve the adjustment to object surfaces. The suitability of the mesh structure and the segmentation methodology is analyzed and the accuracy of the proposed model is proved with both synthetic and real images.     

配准算法对 PCA 单幅投影肺部重建的影响

肺癌放射治疗中,肺部肿瘤位置实成像对于临床意义重大。在一种利用单X射线投影进行成像的实时肺部3D成像算法中,图像配准过程引入的不准确对于PCA模型构建以及重建过程有重大影响。文章分析了光流法、Demons算法、水平集算法三种配准算法对重建效果的影响,并通过定性以及定量实验分析验证。结果表明,光流法配准在配准结果以及模型构建方面有较好的效果。     

Symmetric deformable image registration via optimization of information theoretic measures

The use of information theoretic measures (ITMs) has been steadily growing in image processing, bioinformatics, and pattern classification. Although the ITMs have been extensively used in rigid and affine registration of multi-modal images, their computation and accuracy are critical issues in deformable image registration. Three important aspects of using ITMs in multi-modal deformable image registration are considered in this paper: computation, inverse consistency, and accuracy; a symmetric formulation of the deformable image registration problem through the computation of derivatives and resampling on both source and target images, and sufficient criteria for inverse consistency are presented for the purpose of achieving more accurate registration. The techniques of estimating ITMs are examined and analytical derivatives are derived for carrying out the optimization in a computationally efficient manner. ITMs based on Shannon’s and Renyi’s definitions are considered and compared. The obtained evaluation results via registration functions, and controlled deformable registration of multi-modal digital brain phantom and in vivo magnetic resonance brain images show the improved accuracy and efficiency of the developed formulation. The results also indicate that despite the recent favorable studies towards the use of ITMs based on Renyi’s definitions, these measures are seen not to provide improvements in this type of deformable registration as compared to ITMs based on Shannon’s definitions.     

梯度与曲率变形力相结合的图像配准方法——大形变非刚性图像配准

Active Demons算法是Demons算法的改进形式,其将形变配准视作扩散问题,利用牛顿作用力与反作用力思想,仅依靠梯度信息确定浮动图像的位移,在处理大形变配准问题时存在配准精度不高的弊病。将等照度线曲率作为一个控制形变的驱动力因素引入Active Demons扩散方程,建立了一个具有梯度与曲率双重驱动力相结合的非线性扩散模型（Active G&C model）,并在Active G&C模型应用于大形变图像配准的算法实现过程中加入多分辨率策略,以提高大形变图像的配准精度。实验结果表明,这一模型较经典的Active Demons算法具有更好的配准性能。     

一种特征约束的有限元人脑图像配准方法

提出了一种结合图像的解剖标记点和自适应有限元网格进行人脑图像的精确配准方法.首先利用Forstner算子提取对应图像的解剖标记点,并作初始的图像刚性变换.为了使有限元网格能更加准确地刻画图像解剖结构分布特征,本文利用图像的梯度分布建立了自适应的有限元网格剖分,结合标记点作为有限元的形变约束,使得配准的精度和有限元的计算效率得到提高.人脑图像配准的实验结果表明,该方法能有效地解决图像弹性配准问题.     

Deformable mesh for virtual shape sculpting

A novel, interactive virtual sculpting framework based upon a deformable mesh model generated by a self-organizing feature map (SOFM) is described in this paper. The three-dimensional lattice of the SOFM maintains the relative connectivity of neighbouring nodes in the hexahedral mesh as it transforms from the initial reference geometry into the desired shape. Material and dynamic properties are incorporated into the deformable mesh by treating surface and internal nodes as point masses connected by a network of springs. The initial SOFM mesh can be either retrieved from a library of primitive shapes, or created by automatically adapting the 3D mesh to fit selected surface points. Once the initial mesh has been generated, the designer reshapes the virtual object by introducing external forces to the nodal mesh. The process of virtual sculpting is analogous to hand moulding of clay in the physical world where the material mass remains constant. During sculpting, the dynamically changing mesh can be easily rendered in VRML for visualization in a virtual reality environment. The deformable mesh generator and shape-sculpting system are illustrated by reshaping solid meshes created from scanned human heads.     

视差图像配准技术研究综述

传统图像配准技术受限于严苛的初始输入已难以满足人们的需求,近年来,视差图像的配准逐渐成为图像拼接技术中的研究热点。基于视差形成原理介绍了视差图像配准的难点与一般流程。主要研究了基于特征的视差图像配准技术,对近年来视差图像配准的研究成果进行了归纳梳理,并从基于多平面对齐的图像配准、基于网格变形的图像配准以及缝合线驱动的图像配准三个方面进行阐述。通过对各种典型的视差图像配准算法的算法思想、特点以及局限性进行描述和比较,提供该领域研究现状的系统综述,并对视差图像配准技术的研究趋势进行了展望。     

基于感兴趣窄带区域的同步分割与配准方法及在IGRT中的应用

医学图像分割与配准是图像引导放疗(Image guided radiation therapy, IGRT)系统中的关键技术. 为提高基于CBCT (Cone beam CT)的IGRT系统实施胸腹部肿瘤放疗的实时性与自适应性, 特别是实现重要危及器官肝脏区域照射剂量的合理控制, 本文提出一种基于感兴趣窄带区域的同步分割与配准方法, 目标是实现放疗计划系统中计划CT和CBCT图像目标区域的分割与配准. 通过构建感兴趣窄带模型, 并且与活动轮廓模型相结合实现初始分割, 然后与基于光流场(Optical flow field, OFF)的形变配准方法进行循环迭代, 从而构造ASOR分割与配准同步模型(Active contour segmentation and optical flow registration synchronously, ASOR). 在方法实施时, 首先利用非线性扩散模型和窄带活动轮廓模型在CT图像中提取肝脏空间初始位置信息, 为同步模型提供合理的肝脏初始轮廓. 然后将该轮廓及相应窄带区域经仿射变换映射到CBCT图像, 进而结合构造的ASOR同步模型, 用光流场确定活动轮廓水平集的运动情况, 使分割与配准在同一个演化过程中完成迭代. 实验结果和临床应用表明, 本文提出的方法应用于基于CBCT的IGRT系统时, 可实现肝脏组织的自动分割与放疗剂量分布的快速计算. 同时, 我们将同步过程中获得的形变域用于实现肝脏与肿瘤靶区等剂量线从计划CT到CBCT的自适应转移, 进行自适应放疗效果的临床测评.     

Information Theoretic Deformable Registration Using Local Image Information

We present a deformable registration algorithm for multi-modality images based on information theoretic similarity measures at the scale of individual image voxels. We derive analytical expressions for the mutual information, the joint entropy, and the sum of marginal entropies of two images over a small neighborhood in terms of image gradients. Using these expressions, we formulate image registration algorithms maximizing local similarity over the whole image domain in an energy minimization framework. This strategy produces highly elastic image alignment as the registration is driven by voxel similarities between the images, the algorithms are easily implementable using the closed-form expressions for the derivative of the optimization function with respect to the deformation, and avoid estimation of joint and marginal probability densities governing the image intensities essential to conventional information theoretic image registration methods. This work has been supported in part by NIH grants R01-NS42645 and R01-AG14971.     

Learning an epipolar shift compensation for light field image super-resolution

Light field imaging has drawn broad attention since the advent of practical light field capturing systems that facilitate a wide range of applications in computer vision. However, existing learning-based methods for improving the spatial resolution of light field images neglect the shifts in the sub-pixel domain that are widely used by super-resolution techniques, thus, fail in recovering rich high-frequency information. To fully exploit the shift information, our method attempts to learn an epipolar shift compensation for light field image super-resolution that allows the restored light field image to be angular coherent with the enhancement of spatial resolution. The proposed method first utilizes the rich surrounding views along some typical epipolar directions to explore the inter-view correlations. We then implement feature-level registration to capture accurate sub-pixel shifts of central view, which is constructed by the compensation module equipped with dynamic deformable convolution. Finally, the complementary information from different spatial directions is fused to provide high-frequency details for the target view. By taking each sub-aperture image as a central view, our method could be applied for light field images with any angular resolution. Extensive experiments on both synthetic and real scene datasets demonstrate the superiority of our method over the state-of-the-art qualitatively and quantitatively. Moreover, the proposed method shows good performance in preserving the inherent epipolar structures in light field images. Specifically, our LFESCN method outperforms the state-of-the-art method with about 0.7 dB (PSNR) on average.     

简单人工辅助下的无限长遥感图像自动配准

遥感图像的配准是图像处理中的一个重要分支。部分遥感图像具有大尺度或无限长的特点,并且它们的失配是局部非线性的,直接进行通常意义上的全图配准很困难。该文假设这类图像的失配是连续变化的,建议了一种有初始人工辅助的自动流水线式的图像配准方法。该方法先在人工辅助下在某个初始区域建立初始匹配关系,然后从初始区域逐步扩散匹配控制点,网格约束下的控制点搜索匹配过程保证了在全图建立均匀的密度可控的控制点集。最后用基于多项式的局部加权平均算法完成图像的校正,这样可以保证对于无限长的图像以流水线的方式逐段配准输出。模拟试验结果证实了该方法的有效性。     

Probability Density Estimation Using Isocontours and Isosurfaces: Applications to Information-Theoretic Image Registration

We present a new, geometric approach for determining the probability density of the intensity values in an image. We drop the notion of an image as a set of discrete pixels, and assume a piecewise-continuous representation. The probability density can then be regarded as being proportional to the area between two nearby isocontours of the image surface. Our paper extends this idea to joint densities of image pairs. We demonstrate the application of our method to affine registration between two or more images using information theoretic measures such as mutual information. We show cases where our method outperforms existing methods such as simple histograms, histograms with partial volume interpolation, Parzen windows, etc. under fine intensity quantization for affine image registration under significant image noise. Furthermore, we demonstrate results on simultaneous registration of multiple images, as well as for pairs of volume datasets, and show some theoretical properties of our density estimator. Our approach requires the selection of only an image interpolant. The method neither requires any kind of kernel functions (as in Parzen windows) which are unrelated to the structure of the image in itself, nor does it rely on any form of sampling for density estimation.     

Image fusion enhancement of deformable human structures using a two-stage warping-deformable strategy: A content-based image retrieval consideration

In medical image registration and content-based image retrieval, the rigid transformation model is not adequate for anatomical structures that are elastic or deformable. For human structures such as abdomen, registration would involve global features such as abdominal wall as well as local target organs such as liver or spleen. A general non-rigid registration may not be sufficient to produce image matching of both global and local structures. In this study, a warping-deformable model is proposed to register images of such structures. This model uses a two-stage strategy for image registration of abdomen. In the first stage, the global-deformable transformation is used to register the global wall. The warping-transformation is used in second stage to register the liver. There is a good match of images using the proposed method (mean similarity index = 0.73545).The image matching correlation coefficients calculated from eight pairs of CT and MR images of abdomen indicates that the warping-deformable transformation gives better matching of images than those without transformation (p < 0.001, paired t-test). This study has established a model for image registration of deformable structures. This is particularly important for data mining of image content retrieval for structures which are non-rigid. The result obtained is very promising but further clinical evaluation is needed     

Adaptive Nonlinear Finite Elements for Deformable Body Simulation Using Dynamic Progressive Meshes

Realistic behavior of deformable objects is essential for many applications such as simulation for surgical training. Existing techniques of deformable modeling for real time simulation have either used approximate methods that are not physically accurate or linear methods that do not produce reasonable global behavior. Nonlinear finite element methods (FEM) are globally accurate, but conventional FEM is not real time. In this paper, we apply nonlinear FEM using mass lumping to produce a diagonal mass matrix that allows real time computation. Adaptive meshing is necessary to provide sufficient detail where required while minimizing unnecessary computation. We propose a scheme for mesh adaptation based on an extension of the progressive mesh concept, which we call dynamic progressive meshes.     

Physical Scale Keypoints: Matching and Registration for Combined Intensity/Range Images

We present a new framework for detecting, describing, and matching keypoints in combined range-intensity data, resulting in what we call physical scale keypoints. We first produce an image mesh by backprojecting associated 2D intensity images onto the 3D range data. We detect and describe keypoints on the image mesh using an analogue of the SIFT algorithm for images with two key modifications: the process is made insensitive to viewpoint and structural discontinuities using a novel bilinear filter, and a physical scale space is constructed that exploits the reliable range measurements. Keypoints are matched between scans only when their physical scales agree, avoiding many potential false matches. Finally, the matches are rank-ordered using a new quality measure and supplied to a registration algorithm that refines each match into a rigid transformation for the entire scan pair. We report experimental results on keypoint detection and matching and range scan registration and verification in a set of difficult real-world scan pairs, showing that the new physical scale keypoints are demonstrably better than a competing approach based on backprojected SIFT keypoints.