G-quadtree: A hierarchical representation of gray-scale digital images

We proposeG-quadtree as a hierarchical representation method for gray-scale digital images. G-quadtree is an extended quadtree each leaf of which holds a multi level-value. An algorithm constructing a G-quadtree from the array representation of a gray-scale image is described, implemented and tested. The algorithm is established in such a way that the conventional binary array-to-quadtree conversion algorithm is applied to each bit of array elements repeatedly in descending order of significance. Space efficiency analysis reveals that G-quadtree representation is sufficient in a particular application to a color coding of macroautoradiography of rat brains.     

2D/3D image registration on the GPU

We present a method that performs a rigid 2D/3D image registration efficiently on the Graphical Processing Unit (GPU). As one main contribution of this paper, we propose an efficient method for generating realistic DRRs that are visually similar to x-ray images. Therefore, we model some of the electronic post-processes of current x-ray C-arm-systems. As another main contribution, the GPU is used to compute eight intensity-based similarity measures between the DRR and the x-ray image in parallel. A combination of these eight similarity measures is used as a new similarity measure for the optimization. We evaluated the performance and the precision of our 2D/3D image registration algorithm using two phantom models. Compared to a CPU + GPU algorithm, which calculates the similarity measures on the CPU, our GPU algorithm is between three and six times faster. In contrast to single similarity measures, our new similarity measure achieved precise and robust registration results for both phantom models.     

3D reconstruction from 2D images with hierarchical continuous simplices

This paper presents an effective framework for the reconstruction of volumetric data from a sequence of 2D images. The 2D images are first aligned to generate an initial 3D volume, followed by the creation of a tetrahedral domain using the Carver algorithm. The resulting tetrahedralization preserves both the geometry and topology of the original dataset. Then a solid model is reconstructed using simplex splines with fitting and faring procedures. The reconstructed heterogenous volumetric model can be quantitatively analyzed and easily visualized. Our experiments demonstrated that our approach can achieve high accuracy in the data reconstruction. The novel techniques and algorithms proposed in this paper can be applied to reconstruct a heterogeneous solid model with complex geometry and topology from other visual data.     

2D/3D image registration using regression learning

In computer vision and image analysis, image registration between 2D projections and a 3D image that achieves high accuracy and near real-time computation is challenging. In this paper, we propose a novel method that can rapidly detect an object’s 3D rigid motion or deformation from a 2D projection image or a small set thereof. The method is called CLARET (Correction via Limited-Angle Residues in External Beam Therapy) and consists of two stages: registration preceded by shape space and regression learning. In the registration stage, linear operators are used to iteratively estimate the motion/deformation parameters based on the current intensity residue between the target projection(s) and the digitally reconstructed radiograph(s) (DRRs) of the estimated 3D image. The method determines the linear operators via a two-step learning process. First, it builds a low-order parametric model of the image region’s motion/deformation shape space from its prior 3D images. Second, using learning-time samples produced from the 3D images, it formulates the relationships between the model parameters and the co-varying 2D projection intensity residues by multi-scale linear regressions. The calculated multi-scale regression matrices yield the coarse-to-fine linear operators used in estimating the model parameters from the 2D projection intensity residues in the registration. The method’s application to Image-guided Radiation Therapy (IGRT) requires only a few seconds and yields good results in localizing a tumor under rigid motion in the head and neck and under respiratory deformation in the lung, using one treatment-time imaging 2D projection or a small set thereof.     

Radial invariant of 2D and 3D Racah moments

In this paper, we introduce new sets of 2D and 3D rotation, scaling and translation invariants based on orthogonal radial Racah moments. We also provide theoretical mathematics to derive them. Thus, this work proposes in the first case a new 2D radial Racah moments based on polar representation of an object by one-dimensional orthogonal discrete Racah polynomials on non-uniform lattice, and a circular function. In the second case, we present new 3D radial Racah moments using a spherical representation of volumetric image by one-dimensional orthogonal discrete Racah polynomials and a spherical function. Further 2D and 3D invariants are extracted from the proposed 2D and 3D radial Racah moments respectively will appear in the third case. To validate the proposed approach, we have resolved three problems. The 2D/ 3D image reconstruction, the invariance of 2D/3D rotation, scaling and translation, and the pattern recognition. The result of experiments show that the Racah moments have done better than the Krawtchouk moments, with and without noise. Simultaneously, the mentioned reconstruction converges rapidly to the original image using 2D and 3D radial Racah moments, and the test 2D/3D images are clearly recognized from a set of images that are available in COIL-20 database for 2D image, and PSB database for 3D image.     

基于改进平衡优化器的医学2D/3D图像快速配准算法

医学三维图像(如CT、MRI等)和二维图像(如X光)的配准技术已经被广泛应用于临床诊断和手术规划中. 医学图像配准的实质为使用优化算法寻找某种空间变换, 使两张图像在空间以及结构上对齐. 配准过程中往往由于优化算法寻优精度不高、易陷入局部极值的问题导致配准质量低. 针对此问题, 提出一种改进的平衡优化器算法(improved equilibrium optimizer based on Logistic-Tent chaos map and Levy flight, LTEO), 首先针对种群初始化容易分布不均匀, 且随机性太高的问题, 引入Logistic-Tent混沌映射对种群进行初始化, 提高种群多样性, 使它们尽可能地分布于搜索空间内; 对迭代函数进行更新, 使得优化算法更注重全局范围的搜索, 提高算法收敛速度并利于找到全局最优解; 引入Levy飞行策略对停滞粒子进行扰动, 防止算法陷入局部极值. 最后将改进的平衡优化器算法用于2D/3D医学图像配准任务, 并对配准过程中数据的频繁传输进行优化, 降低配准耗时. 通过基准函数测试和临床配准实验对算法进行验证, 改进后的平衡优化器可有效提高寻优精度和稳定性, 并提高医学图像配准的质量.     

Switching directional BLUs for full‐resolution auto‐stereoscopic 3D/2D and 2D/3D LCDs

Directional backlight unit (BLU) design concept was applied to realize full‐resolution field alternate auto‐stereoscopic liquid crystal displays (LCDs) with built‐in 3D/2D and 2D/3D. The application‐oriented BLU design requires priority in realizing 2D mode or 3D mode. The switching characteristic of BLU with two confronting illuminating sets was applied to build 2D mode into 3D display and 3D mode into 2D display. An LCD with 2D mode as primary function requires higher double‐sided luminance uniformity than 3D mode. On the other hand, an LCD with 3D mode requires higher single‐sided luminance uniformity than 2D mode. For increasing the double‐sided luminance uniformity for 2D mode as a primary function of the display, the former BLU design was realized by using uniform prismatic structure, that is, using uniform bilateral prismatic structure with a cross section of isosceles triangle on the back surface of the light‐guide plate. For increasing single‐sided luminance uniformity for 3D mode as a primary function of the display, the latter was realized by using graded angle design, that is, unilateral prisms with a cross section of a scalene triangle on the back surface of the light‐guide plate. The LCDs of small‐sized‐handheld game devices and still cameras were fabricated using the 3D/2D and 2D/3D displays.     

Transformations between Pictures from 2D to 3D

This paper describes programs for 3-dimensional engraving. The programs use raster or vector images to create a 3D model and, subsequently, convert this model into a sequence of control commands for 3D engraving machines. Three programs have been developed. A program for engraving general 3D surfaces from grey-scale images, a program for preparing these grey-scale images from patterns and vector images, and a program for fast 2D engraving. A simple and fast preparation of the 3D model, a user-friendly environment, and small hardware requirements were the principal goals.     

3D reconstruction of a femoral shape using a parametric model and two 2D fluoroscopic images

In medical diagnostic imaging, the X-ray CT scanner and the MRI system have been widely used to examine 3D shapes and internal structures of living organisms and bones. However, these apparatuses are generally large and very expensive. Since an appointment is also required before examination, these systems are not suitable for urgent fracture diagnosis in emergency treatment. However, X-ray/fluoroscopy has been widely used as traditional medical diagnosis. Therefore, the realization of the reconstruction of precise 3D shapes of living organisms or bones from a few conventional 2D fluoroscopic images might be very useful in practice, in terms of cost, labor, and radiation exposure. The present paper proposes a method by which to estimate a patient-specific 3D shape of a femur from only two fluoroscopic images using a parametric femoral model. First, we develop a parametric femoral model by the statistical analysis of 3D femoral shapes created from CT images of 56 patients. Then, the position and shape parameters of the parametric model are estimated from two 2D fluoroscopic images using a distance map constructed by the Level Set Method. Experiments using synthesized images, fluoroscopic images of a phantom femur, and in vivo images for hip prosthesis patients are successfully carried out, and it is verified that the proposed system has practical applications.     

Context-based vision: recognizing objects using information fromboth 2D and 3D imagery

Results from an ongoing project concerned with recognizing objects in complex scene domains, especially in the domain that includes the natural outdoor world, are described. Traditional machine recognition paradigms assume either that all objects of interest are definable by a relatively small number of explicit shape models or that all objects of interest have characteristic, locally measurable features. The failure of both assumptions has a dramatic impact on the form of an acceptable architecture for an object recognition system. In this work, the use of the contextual information is a central issue, and a system is explicitly designed to identify and use context as an integral part of recognition that eliminates the traditional dependence on stored geometric models and universal image partitioning algorithms. This paradigm combines the results of many simple procedures that analyze monochrome, color, stereo, or 3D range images. Interpreting the results along with relevant contextual knowledge makes it possible to achieve a reliable recognition result, even when using imperfect visual procedures. Initial experimentation with the system on ground-level outdoor imagery has demonstrated competence beyond what is attainable with other vision systems     

Learning 3D Deformation of Animals from 2D Images

Understanding how an animal can deform and articulate is essential for a realistic modification of its 3D model. In this paper, we show that such information can be learned from user‐clicked 2D images and a template 3D model of the target animal. We present a volumetric deformation framework that produces a set of new 3D models by deforming a template 3D model according to a set of user‐clicked images. Our framework is based on a novel locally‐bounded deformation energy, where every local region has its own stiffness value that bounds how much distortion is allowed at that location. We jointly learn the local stiffness bounds as we deform the template 3D mesh to match each user‐clicked image. We show that this seemingly complex task can be solved as a sequence of convex optimization problems. We demonstrate the effectiveness of our approach on cats and horses, which are highly deformable and articulated animals. Our framework produces new 3D models of animals that are significantly more plausible than methods without learned stiffness.     

An algebraic approach for morphological operations on 2D and 3D images

This paper describes an algebraic approach to morphological operations on 2D and 3D images. For 2D images, a matching algorithm based on mathematical morphology is given. For 3D images represented by voxels, algorithms are developed for the shape decomposition and thinning. The possible advantages of the algebraic approach are indicated by showing that a considerable speedup takes place (for large-sized templates) if the Fast Fourier Transform (FFT) is used to compute the convolutions.     

Compression-Based 3D Texture Mapping for Real-Time Rendering

While 2D texture mapping is one of the most effective of the rendering techniques that make 3D objects appear visually interesting, it often suffers from visual artifacts produced when 2D image patterns are wrapped onto the surfaces of objects with arbitrary shapes. On the other hand, 3D texture mapping generates highly natural visual effects in which objects appear carved from lumps of materials rather than laminated with thin sheets as in 2D texture mapping. Storing 3D texture images in a table for fast mapping computations, instead of evaluating procedures on the fly, however, has been considered impractical due to the extremely high memory requirement. In this paper, we present a new effective method for 3D texture mapping designed for real-time rendering of polygonal models. Our scheme attempts to resolve the potential texture memory problem by compressing 3D textures using a wavelet-based encoding method. The experimental results on various nontrivial 3D textures and polygonal models show that high compression rates are achieved with few visual artifacts in the rendered images and a small impact on rendering time. The simplicity of our compression-based scheme will make it easy to implement practical 3D texture mapping in software/hardware rendering systems including real-time 3D graphics APIs such as OpenGL and Direct3D.     

An evaluation of multimodal 2D+3D face biometrics

We report on the largest experimental study to date in multimodal 2D+3D face recognition, involving 198 persons in the gallery and either 198 or 670 time-lapse probe images. PCA-based methods are used separately for each modality and match scores in the separate face spaces are combined for multimodal recognition. Major conclusions are: 1) 2D and 3D have similar recognition performance when considered individually, 2) combining 2D and 3D results using a simple weighting scheme outperforms either 2D or 3D alone, 3) combining results from two or more 2D images using a similar weighting scheme also outperforms a single 2D image, and 4) combined 2D+3D outperforms the multi-image 2D result. This is the first (so far, only) work to present such an experimental control to substantiate multimodal performance improvement.     

人脸识别:从二维到三维

人脸识别是生物特征识别技术的一个重要方向。虽然目前大部分研究都还只是针对二维人脸图像,但是3D人脸模型包含更丰富的人脸信息,有助于机器对人脸的识别。从二维到三维,人脸识别研究进入了一个新的阶段。从3D人脸数据的获取方式入手,介绍最近提出的一系列3D人脸识别算法,并进行归类。最后提出"有针对性地获取3D人脸模型数据是进行有效识别的基础"这一结论。     

Topological model for 3D image representation: Definition and incremental extraction algorithm

In this paper, we define the three-dimensional topological map, a model which represents both the topological and geometrical information of a three-dimensional labeled image. Since this model describes the image’s topology in a minimal way, we can use it to define efficient image processing algorithms. The topological map is the last level of map hierarchy. Each level represents the region boundaries of the image and is defined from the previous level in the hierarchy, thus giving a simple constructive definition. This model is an extension of the similar model defined for 2D images. Progressive definition based on successive map levels allows us to extend this model to higher dimension. Moreover, with progressive definition, we can study each level separately. This simplifies the study of disconnection cases and the proofs of topological map properties. Finally, we provide an incremental extraction algorithm which extracts any map of the hierarchy in a single image scan. Moreover, we show that this algorithm is very efficient by giving the results of our experiments made on artificial images.     

3D and 2D/3D holograms model

The estimating problem of 3D holograms orientation selectivity on angular, orthogonal, and azimuthal sensitivity parameters is formulated and solved. Tenfold increase of density 3D, 2D/3D holograms in comparison with 2D holograms at given selectivity for ones is shown in theory and experimentally.     

Visual comfort in 2D/3D modes of glasses‐type stereoscopic displays

In this study, we compared visual comfort in 2D/3D modes of the pattern retarder (PR) and shutter glasses (SG) stereoscopic displays by changing viewing factors and image contents. The viewing factors include ambient illuminance/monitor luminance/background luminance and image contents mainly are determined with different disparity limits. The degrees of 2D/3D visual comfort were investigated by using various combinations of ambient illuminance, monitor luminance, background luminance, and disparity limit. A series of psychological experiments were also performed to compare 2D and 3D viewing experiences for the passive PR and active SG stereoscopic displays and to discover more comfortable conditions under various variable combinations. The experiment results show that the various variable combinations affecting visual comfort in the passive PR and active SG stereoscopic displays were significantly different. Finally, we suggest more comfortable conditions of viewing 2D and 3D images for the PR and SG stereoscopic displays.     

2D face recognition based on supervised subspace learning from 3D models

One of the main challenges in face recognition is represented by pose and illumination variations that drastically affect the recognition performance, as confirmed by the results of recent face recognition large-scale evaluations. This paper presents a new technique for face recognition, based on the joint use of 3D models and 2D images, specifically conceived to be robust with respect to pose and illumination changes. A 3D model of each user is exploited in the training stage (i.e. enrollment) to generate a large number of 2D images representing virtual views of the face with varying pose and illumination. Such images are then used to learn in a supervised manner a set of subspaces constituting the user's template. Recognition occurs by matching 2D images with the templates and no 3D information (neither images nor face models) is required. The experiments carried out confirm the efficacy of the proposed technique.     

3D Euclidean versus 2D non-Euclidean: two approaches to 3D recoveryfrom images

Methods of 3D recovery in computer vision for computing the shape and motion of an object from projected images when an object model is available are classified into two types: the 3D Euclidean approach, which is based on geometrical constraints in 3D Euclidean space, and the 2D non-Euclidean space. Implications of these two approaches are discussed, and some illustrating examples are presented