3D anatomical shape atlas construction using mesh quality preserved deformable models

3D anatomical shape atlas construction has been extensively studied in medical image analysis research, owing to its importance in model-based image segmentation, longitudinal studies and populational statistical analysis, etc. Among multiple steps of 3D shape atlas construction, establishing anatomical correspondences across subjects, i.e., surface registration, is probably the most critical but challenging one. Adaptive focus deformable model (AFDM) [1] was proposed to tackle this problem by exploiting cross-scale geometry characteristics of 3D anatomy surfaces. Although the effectiveness of AFDM has been proved in various studies, its performance is highly dependent on the quality of 3D surface meshes, which often degrades along with the iterations of deformable surface registration (the process of correspondence matching). In this paper, we propose a new framework for 3D anatomical shape atlas construction. Our method aims to robustly establish correspondences across different subjects and simultaneously generate high-quality surface meshes without removing shape details. Mathematically, a new energy term is embedded into the original energy function of AFDM to preserve surface mesh qualities during deformable surface matching. More specifically, we employ the Laplacian representation to encode shape details and smoothness constraints. An expectation–maximization style algorithm is designed to optimize multiple energy terms alternatively until convergence. We demonstrate the performance of our method via a set of diverse applications, including a population of sparse cardiac MRI slices with 2D labels, 3D high resolution CT cardiac images and rodent brain MRIs with multiple structures. The constructed shape atlases exhibit good mesh qualities and preserve fine shape details. The constructed shape atlases can further benefit other research topics such as segmentation and statistical analysis.     

A hierarchy of cameras for 3D photography

The view-independent visualization of 3D scenes is most often based on rendering accurate 3D models or utilizes image-based rendering techniques. To compute the 3D structure of a scene from a moving vision sensor or to use image-based rendering approaches, we need to be able to estimate the motion of the sensor from the recorded image information with high accuracy, a problem that has been well-studied. In this work, we investigate the relationship between camera design and our ability to perform accurate 3D photography, by examining the influence of camera design on the estimation of the motion and structure of a scene from video data. By relating the differential structure of the time varying plenoptic function to different known and new camera designs, we can establish a hierarchy of cameras based upon the stability and complexity of the computations necessary to estimate structure and motion. At the low end of this hierarchy is the standard planar pinhole camera for which the structure from motion problem is non-linear and ill-posed. At the high end is a camera, which we call the full field of view polydioptric camera, for which the motion estimation problem can be solved independently of the depth of the scene which leads to fast and robust algorithms for 3D Photography. In between are multiple view cameras with a large field of view which we have built, as well as omni-directional sensors.     

基于射影不变量的三维重构

文中提出了利用射影不变量来求解基于图像对三维深度恢复问题。方法的基本思想是对于立体图像，利用密度段元素，引入了两个射影不变量来恢复密度段的深度信息。从这两个不变量，能推导立体图像中匹配的密度段对所满足的关系。利用这个关系，实现了密度段之间的匹配运算。这个方法能直接地从输入图像中得到密集和准确的深度，对变形的图像具有鲁棒性。     

基于射影不变量的三维重构

文中提出了利用射影不变量来求解基于图像对三维深度恢复问题。方法的基本思想是对于立体图像,利用密度段元素,引入了两个射影不变量来恢复密度段的深度信息。从这两个不变量,能推导立体图像中匹配的密度段对所满足的关系。利用这个关系,实现了密度段之间的匹配运算。这个方法能直接地从输入图像中得到密集和准确的深度,对变形的图像具有鲁棒性。     

GPU视频纹理技术在三维电影播放中的应用

研究了三维电影播放应用中的视频纹理技术,分析了该方法的国内外研究现状、难题和挑战,提出了有效地解决方案,并对该方案进行设计.采用 DirectShow进行视频数据的调度与管理,Direct3D作为图形渲染API,提取出传统渲染管道下的并行运算部分,利用可编程GPU并行运算的特性对其进行加速渲染,并利用GPU纹理压缩方法解决了显存容量有限的问题.实验结果表明,该设计方法能够有效地提高帧速率,缓解CPU负担,解决实时渲染中的性能瓶颈问题,在实际应用中具有较强的使用价值.     

面向手持式3维扫描设备的本征纹理重建

目的 针对目前手持式3维扫描设备生成的模型纹理分辨率不够,且部分区域存在高光、阴影及明暗变化等问题,提出一种基于多幅实拍照片的纹理重建方法。方法 首先使用基于特征匹配的方法将照片图像与几何模型进行配准;其次根据重建纹理大小,采用特殊编码方式的位置纹理建立照片像素到纹理像素直接且精确的对应关系;然后根据多幅使用闪光灯作为光源拍摄的照片,通过位置纹理建立联立方程,求解漫反射分量;最后采用改进的基于混合权重的融合方法对求解的漫反射分量进行纹理融合。结果 使用本文方法对3个实验模型进行本征纹理重建,与3维扫描设备生成纹理和直接用照片生成纹理相比,该方法操作简单、使用方便,可获得高度清晰的,不含高光和明暗效果的本征纹理图像。结论 实验结果表明,重建纹理质量在分辨率、色彩还原性及一致性方面明显优于原有纹理,且该方法具有很高的精确性和鲁棒性,可满足高质量的纹理重建需求。     

Rich-information watermarking scheme for 3D models of oblique photography

The security issues of copyright information arise rapidly along with the widely application of 3D models of oblique photography. In this paper, a reversible watermarking scheme is proposed for 3D models of oblique photography, aiming at robustness enhancing and rich-information watermark embedding. To realize the goal of rich-information watermark embedding, an encoding method of original copyright information is designed based on QR code, which can shorten significantly the length of the final watermark data without affecting the copyright expression. A 3D point grouping method is designed and both watermark embedding and extraction are completed group by group, so that the robustness of this scheme under certain attacks can be enhanced, e.g., model segmentation, data compression, point randomly deleting, etc. Moreover, watermarks are embedded into spherical coordinates of 3D points so that the ability of the proposed scheme in resisting geometric transformation can be improved. Results of simulation experiments have demonstrated that the proposed scheme can support rich-information watermark embedding, and it has a satisfying robustness under common geometric and non-geometric attacks.

     

Texture evolution: 3D texture synthesis from single 2D growable texture pattern

An efficient model-independent 3D texture synthesis algorithm based on texture growing and texture turbulence is presented to create vivid 3D solid texture from a single 2D growable texture pattern. Given a 2D texture pattern of some growable material, our technique is able to create an anisotropic 3D volumetric texture cube to simulate the evolution of the material in 3D. An effective tiling scheme is designed to save computation and storage costs. Target objects are directly dipped into the synthesized 3D texture volume to generate creative, sculpture-like models that can be visualized with interactive speed. Our method is conceptually intuitive, computationally fast, and storage efficient compared with other solid texturing methods. As opposed to conventional 2D texture mapping work on polygonal surfaces, our approach is capable of decorating 3D point-rendering systems seamlessly. Furthermore, our combination of texture turbulence and texture growing techniques provides an attractive way to synthesize and tile natural 2D texture patterns, or generate simple but interesting motion textures.     

3D face computational photography using PCA spaces

In this paper, we present a 3D face photography system based on a facial expression training dataset, composed of both facial range images (3D geometry) and facial texture (2D photography). The proposed system allows one to obtain a 3D geometry representation of a given face provided as a 2D photography, which undergoes a series of transformations through the texture and geometry spaces estimated. In the training phase of the system, the facial landmarks are obtained by an active shape model (ASM) extracted from the 2D gray-level photography. Principal components analysis (PCA) is then used to represent the face dataset, thus defining an orthonormal basis of texture and another of geometry. In the reconstruction phase, an input is given by a face image to which the ASM is matched. The extracted facial landmarks and the face image are fed to the PCA basis transform, and a 3D version of the 2D input image is built. Experimental tests using a new dataset of 70 facial expressions belonging to ten subjects as training set show rapid reconstructed 3D faces which maintain spatial coherence similar to the human perception, thus corroborating the efficiency and the applicability of the proposed system.     

基于纹理集的大规模场景模型优化算法

为了提高大规模场景模型的渲染帧率,提出了一种基于纹理集(texture atlas)技术的优化算法。通过动态空间分配算法,将模型中的纹理以最小空间代价合并为数个大纹理。在此基础上,通过"冗余存储"的方式解决重复纹理模式下的子纹理无法正常显示的问题,并且将模型中节点的纹理坐标更新。实验结果表明,该算法有效且可行,通过该算法优化后的模型的纹理状态切换次数大大减少,同时最大程度地节约了纹理空间,渲染帧率明显提高。     

结合三维几何形状信息和二维纹理的3D目标匹配

为了实现场景中三维目标与模型之间的匹配,提出了一种结合三维几何形状信息和二维纹理的三维目标匹配方法。首先提取场景中深度图像的尺度不变特征变换(SIFT)特征,用SIFT算法与三维模型重建时所用到的一系列2.5维深度图像进行一一匹配,找到与场景中目标姿态最为相似的深度图像,提取此深度图像的三维几何形状特征与模型进行匹配,实现模型的初始化,即将模型重置到与场景目标相接近的姿态。最后用融合二维纹理信息的迭代就近点（ICP）算法实现场景中目标与模型之间的匹配,从而得到场景中三维目标的准确姿态。实验结果验证了方法的可行性与精确性。     

A Geometric Approach for the Theory and Applications of 3D Projective Invariants

A central task of computer vision is to automatically recognize objects in real-world scenes. The parameters defining image and object spaces can vary due to lighting conditions, camera calibration and viewing position. It is therefore desirable to look for geometric properties of the object which remain invariant under such changes in the observation parameters. The study of such geometric invariance is a field of active research. This paper presents the theory and computation of projective invariants formed from points and lines using the geometric algebra framework. This work shows that geometric algebra is a very elegant language for expressing projective invariants using n views. The paper compares projective invariants involving two and three cameras using simulated and real images. Illustrations of the application of such projective invariants in visual guided grasping, camera self-localization and reconstruction of shape and motion complement the experimental part.     

Shadow based texture registration for 3D modeling of outdoor scenes

This paper presents a novel method for the registration of texture images with a 3D model of outdoor scenes. We pose image registration as an optimization problem that uses knowledge of the sun’s position to estimate shadows in a scene, and use the shadows produced as a cue to solve for the registration parameters. Results are presented on a controlled experiment and for a large scale model of an archaeological site in Sicily.     

结合曲面局部纹理特征的3维人脸识别

目的 人脸2维图像反映出来的纹理并非是3维人脸曲面真实的纹理,并且受光照和妆容的影响很大,因此探索3维局部纹理特征对于人脸识别任务有着重要的意义。为此详细分析了一种新颖的3维局部纹理特征mesh-LBP对于人脸纹理的描述能力。方法 首先,在特征提取和识别任务之前,进行一系列的预处理：人脸分割、离群点移除和孔洞填补;接着,在预处理后的人脸曲面上,提取原始mesh-LBP特征,以及基于阈值化策略的3种改进特征：mesh-tLBP、mesh-MBP和mesh-LTP;然后,对于上述提取的4种特征,采用不同的统计方法,包括整体直方图、局部分块直方图和整体编码图像,用做人脸纹理的特征描述。最后,针对CASIA3D数据集中不同表情和姿态变化的人脸,采用余弦相似度进行人脸的识别任务。结果 通过对比人脸曲面和普通物体曲面的纹理特征,发现人脸纹理完全不同于普通纹理,不规则并且难以描述;通过对比mesh-LBP两种变体,发现mesh-LBP（α₁）适用于姿态变化,而mesh-LBP（α₂）适用于表情变化;通过对比原始mesh-LBP及其3种改进,发现mesh-tLBP对于人脸不同表情变化下的识别准确率最高有0.5%的提升;通过对比3种不同的统计方法,发现采用整体编码图像进行统计的特征尽管弱于局部分块直方图,但相比整体直方图,识别率在不同表情变化下最高有46.8%的提升。结论 mesh-LBP特征是一种优良的3维局部纹理特征,未来将会在3维医学处理、3维地形起伏检测以及3维人脸识别中得到更多的应用。     

3D object retrieval in an atlas of neuronal structures

Circuit neuroscience tries to solve one of the most challenging questions in biology: How does the brain work? An important step toward an answer to this question is to gather detailed knowledge about the neuronal circuits of the model organism Drosophila melanogaster. Geometric representations of neuronal objects of the Drosophila are acquired using molecular genetic methods, confocal microscopy, nonrigid registration and segmentation. These objects are integrated into a constantly growing common atlas. The comparison of new segmented neuronal objects to already known neuronal structures is a frequent task, which evolves with a growing amount of data into a bottleneck of the knowledge discovery process. Thus, the exploration of the atlas by means of domain specific similarity measures becomes a pressing need. To enable similarity based retrieval of neuronal objects, we defined together with domain experts tailored dissimilarity measures for each of the three typical neuronal structures cell body, projection, and arborization. Moreover, we defined the neuron enhanced similarity for projections and arborizations. According to domain experts, the developed system has big advantages for all tasks, which involve extensive data exploration.     

Projective Feature Learning for 3D Shapes with Multi‐View Depth Images

Feature learning for 3D shapes is challenging due to the lack of natural paramterization for 3D surface models. We adopt the multi‐view depth image representation and propose Multi‐View Deep Extreme Learning Machine (MVD‐ELM) to achieve fast and quality projective feature learning for 3D shapes. In contrast to existing multi‐view learning approaches, our method ensures the feature maps learned for different views are mutually dependent via shared weights and in each layer, their unprojections together form a valid 3D reconstruction of the input 3D shape through using normalized convolution kernels. These lead to a more accurate 3D feature learning as shown by the encouraging results in several applications. Moreover, the 3D reconstruction property enables clear visualization of the learned features, which further demonstrates the meaningfulness of our feature learning.     

Invariant feature extraction for 3D texture analysis using the autocorrelation function

A new method for invariant feature extraction on textured images undergoing affine transformations is presented. This is performed by transformation of the autocorrelation function (ACF) followed by determination of an invariant criterion which is the sum of the coefficients of the discrete correlation matrix. Experimental results support the effectiveness of the proposed approach.     

A JPEG-like texture compression with adaptive quantization for 3D graphics application

Published online: 2 October 2001     

高质量的三维纹理硬件体绘制

与光线投射法相比,传统的3D纹理体绘制算法通常难以产生高质量的图像。为了增强渲染图像的真实感与质量,在基于GPU（Graphics Processing Unit）的三维纹理体绘制过程中以交互的速率实现了体阴影效果,并考虑现实图像合成中的可视化感知,提出将基于GPU的高动态范围色调映射技术应用到体绘制得到的结果图片中。最后对一些体数据集进行绘制,实验表明这些技术较好地解决了传统纹理绘制方法的缺点,提高了图像的质量。