All-in-focus and multi-focus color image reconstruction from a database of color and depth image pairs

Owing to recent advances in depth sensors and computer vision algorithms, depth images are often available with co-registered color images. In this paper, we propose a simple but effective method for obtaining an all-in-focus (AIF) color image from a database of color and depth image pairs. Since the defocus blur is inherently depth-dependent, the color pixels are first grouped according to their depth values. The defocus blur parameters are then estimated using the amount of the defocus blur of the grouped pixels. Given a defocused color image and its estimated blur parameters, the AIF image is produced by adopting the conventional pixel-wise mapping technique. In addition, the availability of the depth image disambiguates the objects located far or near from the in-focus object and thus facilitates image refocusing. We demonstrate the effectiveness of the proposed algorithm using both synthetic and real color and depth images.     

Affine reconstruction of curved surfaces from uncalibrated views ofapparent contours

In this paper, we consider uncalibrated reconstruction of curved surfaces from apparent contours. Since apparent contours are not fixed features (viewpoint independent), we cannot directly apply the recent results of the uncalibrated reconstruction from fixed features. We show that, nonetheless, curved surfaces can be reconstructed up to an affine ambiguity from their apparent contours viewed from uncalibrated cameras with unknown linear translations. Furthermore, we show that, even if the reconstruction is nonmetric (non-Euclidean), we can still extract useful information for many computer vision applications just from the apparent contours. We first show that if the camera motion is linear translation (but arbitrary direction and magnitude), the epipolar geometry can be recovered from the apparent contours without using any optimization process. The extracted epipolar geometry is next used for reconstructing curved surfaces from the deformations of the apparent contours viewed from uncalibrated cameras. The result is applied to distinguishing curved surfaces from fixed features in images. It is also shown that the time-to-contact to the curved surfaces can be computed from simple measurements of the apparent contours     

Detecting siblings in image pairs

In everyday life, face similarity is an important kinship clue. Computer algorithms able to infer kinship from pairs of face images could be applied in forensics, image retrieval and annotation, and historical studies. So far, little work in this area has been presented, and only one study, using a small set of low quality images, tackles the problem of identifying siblings pairs. The purpose of our paper is to present a comprehensive investigation on this subject, aimed at understanding which are, on the average, the most relevant facial features, how effective can be computer algorithms for detecting siblings pairs, and if they can outperform human evaluation. To avoid problems due to low quality pictures and uncontrolled imaging conditions, as for the heterogeneous datasets collected for previous researches, we prepared a database of high quality pictures of sibling pairs, shot in controlled conditions and including frontal, profile, expressionless, and smiling faces. Then we constructed various classifiers of image pairs using different types of facial data, based on various geometric, textural, and holistic features. The classifiers were first tested separately, and then the most significant facial data, selected with a two stage feature selection algorithm were combined into a unique classifier. The discriminating ability of the automatic classifier combining features of different nature has been found to outperform that of a panel of human raters. We also show the good generalization capabilities of the algorithm by applying the classifier, in a cross-database experiment, to a low quality database of images collected from the Internet.     

3D corrective nose reconstruction from a single image

There is a steadily growing range of applications that can benefit from facial reconstruction techniques, leading to an increasing demand for reconstruction of high-quality 3D face models. While it is an important expressive part of the human face, the nose has received less attention than other expressive regions in the face reconstruction literature.When applying existing reconstruction methods to facial images, the reconstructed nose models are often inconsistent with the desired shape and ex...     

Non-rigid metric reconstruction from perspective cameras

The metric reconstruction of a non-rigid object viewed by a generic camera poses new challenges since current approaches for Structure from Motion assume the rigidity constraint of a shape as an essential condition. In this work, we focus on the estimation of the 3-D Euclidean shape and motion of a non-rigid shape observed by a perspective camera. In such case deformation and perspective effects are difficult to decouple – the parametrization of the 3-D non-rigid body may mistakenly account for the perspective distortion. Our method relies on the fact that it is often a reasonable assumption that some of the points on the object’s surface are deforming throughout the sequence while others remain rigid. Thus, relying on the rigidity constraints of a subset of rigid points, we estimate the perspective to metric upgrade transformation. First, we use an automatic segmentation algorithm to identify the set of rigid points. These are then used to estimate the internal camera calibration parameters and the overall rigid motion. Finally, we formulate the problem of non-rigid shape and motion estimation as a non-linear optimization where the objective function to be minimized is the image reprojection error. The prior information that some of the points in the object are rigid can also be added as a constraint to the non-linear minimization scheme in order to avoid ambiguous configurations. We perform experiments on different synthetic and real data sets which show that even when using a minimal set of rigid points and when varying the intrinsic camera parameters it is possible to obtain reliable metric information.     

Fingerprint image reconstruction from standard templates

A minutiae-based template is a very compact representation of a fingerprint image and for a long time it has been assumed that it did not contain enough information to allow the reconstruction of the original fingerprint. This work proposes a novel approach to reconstruct fingerprint images from standard templates and investigates to what extent the reconstructed images are similar to the original ones (i.e., those the templates were extracted from). The efficacy of the reconstruction technique has been assessed by estimating the success chances of a masquerade attack against nine different fingerprint recognition algorithms. The experimental results show that the reconstructed images are very realistic and that, although it is unlikely they can fool a human expert, there is a high chance to deceive state-of-the-art commercial fingerprint recognition systems.     

基于医学图像的复杂曲面重建

为了更准确地重建复杂的三维医学数据模型,把二维医学图像轮廓线上的像素点转化为三维点云,引入经典的泊松点云重建技术。由于泊松重建的效果依赖于点云法向的准确性,针对轮廓线数据的特点,结合图像二维梯度方向,给出了点云法向的一致定向及基于已知定向进一步精确估计法向的方法。泊松重建方法作为一种隐式重建技术,可以很好地处理医学图像数据中经常存在的噪声、拓扑复杂等问题。鼻咽喉模型重建的结果说明,一致定向方法改进了传统的一致定向方法,得到了更为准确的医学重建模型。     

压缩感知图像重构中矩阵互相关性的研究

从测量矩阵和稀疏矩阵的互相关性角度出发,通过对测量矩阵和稀疏矩阵所构成的Gram矩阵进行门限选择,进而经过相应的缩放处理降低互相关性,这样不仅可以获取更多有信息量的测量值,而且可以完成对测量矩阵的优化改进.通过在DWT、DCT下的压缩感知图像重构实验验证了该方法的可行性,恢复效果得到一定程度的提高,相比于传统的小波恢复重构,达到了预期的效果.     

迭代图像重建中系统矩阵与重建图像质量关系研究

针对现有系统矩阵算法复杂、效率低的问题,提出了一种简易的线段加权算法。对比传统线段加权算法,所提算法大大减少了对直线与网格相交情况的判断,使用二维方式对网格进行编号,并且基于提出的算法对传统线段加权法计算系统矩阵过程进行了改进。采用改进方法计算的系统矩阵重建图像,并且对重建图像质量进行评价。实验结果表明,所提算法的运算速度比Siddon改进算法提高了3倍以上,并且重建图像的质量随着线段加权法中计算线段数目增加而逐步提高。     

基于单张图像的三维人脸重建

利用一种基于法线的模型变形方法,从单张图像重建高质量的三维人脸.利用球谐函数和一个初始参考模型计算得到模型上每个顶点的法线,利用法线使参考模型变形.实验结果表明:提出的算法可以从单幅图像重建具有细节的高质量三维人脸.     

未标定影像序列三维重建的混合方法

基于非标定序列影像的目标三维重建是一项非常重要的技术和研究热点,它使数据获取变得十分方便。基于影像序列的点匹配,得到的是一些点云,基于此,提出一个混合的三维重建方法：第一,通过物体三维点建立物体的数字形状模型（DSM）;第二,通过提取物体轮廓线,尤其是相互的平行直段和垂直线段,构建物体的轮廓线;第三,给合现存的三维数据模型,在目标显示和数据结构方面构建恢复三维物体。实验以一个茶筒为例,采用Java3D显示结果,取得良好的结果。     

Validation of a parallel genetic algorithm for image reconstruction from projections

The problem of accurate image reconstruction from projections has repeatedly arisen over the last decades in a large number of scientific, medical and technical fields. Reconstruction algorithms use data from electron microscopes to reconstruct molecular structures or X-ray projection data to compute medical images. Usually, the applied projection data are noisy and therefore iterative algorithms are used to solve numerically a number of equations. Theory and empirical results demonstrate that genetic algorithms (GA) can accurately solve a broad class of problems, especially if noisy input data are used. GA are based on the evolution of random tries by individuals, and therefore the time to find an appropriate solution is rather long. In this work, we use a parallel approach using JavaSpaces to speed up a genetic reconstruction algorithm.     

Interactive image completion with perspective correction

We present an interactive system for fragment-based image completion which exploits information about the approximate 3D structure in a scene in order to estimate and apply perspective corrections when copying a source fragment to a target position. Even though implicit 3D information is used, the interaction is strictly 2D, which makes the user interface very simple and intuitive. We propose different interaction metaphors in our system for providing 3D information interactively. Our search and matching procedure is done in the Fourier domain, and hence it is very fast and it allows us to use large fragments and multiple source images with high resolution while still obtaining interactive response times. Our image completion technique also takes user-specified structure information into account where we generalize the concept of feature curves to arbitrary sets of feature pixels. We demonstrate our technique on a number of difficult completion tasks.     

Dense 3D facial reconstruction from a single depth image in unconstrained environment

Virtual Reality - With the increasing demands of applications in virtual reality such as 3D films, virtual human–machine interactions and virtual agents, the analysis of 3D human face is...     

RGB-D image saliency detection from 3D perspective

Multimedia Tools and Applications - With the advent of stereo camera saliency object detection for RGB-D image is attracting more and more interest. Most existing algorithms treat RGB-D image as...     

Harmony search algorithm for image reconstruction from projections

Image reconstruction from projections is an important problem in the areas of microscopy, geophysics, astrophysics, satellite and medical imaging. The problem of image reconstruction from projections is considered as an optimization problem where a meta-heuristic technique can be used to solve it. In this paper, we propose a new method based on harmony search (HS) meta-heuristic for image reconstruction from projections. The HS method is combined then with a local search method (LS) to improve the quality of reconstructed images in tomography. The two proposed methods (HS and hybrid HS) are validated on some images and compared with both the filtered back-projection (FBP) and the simultaneous iterative reconstruction technique (SIRT) methods. The numerical results are encouraging and demonstrate the benefits of the proposed methods for image reconstruction in tomography.     

Estimating 3D egomotion from perspective image sequence

The computation of sensor motion from sets of displacement vectors obtained from consecutive pairs of images is discussed. The problem is investigated with emphasis on its application to autonomous robots and land vehicles. The effects of 3D camera rotation and translation upon the observed image are discussed, particularly the concept of the focus of expansion (FOE). It is shown that locating the FOE precisely is difficult when displacement vectors are corrupted by noise and errors. A more robust performance can be achieved by computing a 2D region of possible FOE locations (termed the fuzzy FOE) instead of looking for a single-point FOE. The shape of this FOE region is an explicit indicator of the accuracy of the result. It has been shown elsewhere that given the fuzzy FOE, a number of powerful inferences about the 3D sense structure and motion become possible. Aspects of computing the fuzzy FOE are emphasized, and the performance of a particular algorithm on real motion sequences taken from a moving autonomous land vehicle is shown     

A shape from shading analysis for a single perspective image of apolyhedron

A shape-from-shading analysis for a single perspective image of a polyhedron is presented. Given a single perspective image of a polyhedron, the depth of any point of the polyhedron from the camera, the direction of the light source illuminating the polyhedron and the albedo of the polyhedron, a system of algebraic equations are derived, which, when combined with edge information, quantitatively describes the shape of the polyhedron. This analysis is best possible in the sense that if any component of what the author has assumed is omitted, no similar analysis can provide the same results     

Topological equivalence between a 3D object and the reconstruction of its digital image

Digitization is not as easy as it looks. If one digitizes a 3D object even with a dense sampling grid, the reconstructed digital object may have topological distortions and, in general, there exists no upper bound for the Hausdorff distance. This explains why so far no algorithm has been known which guarantees topology preservation. However, as we will show, it is possible to repair the obtained digital image in a locally bounded way so that it is homeomorphic and close to the 3D object. The resulting digital object is always well-composed, which has nice implications for a lot of image analysis problems. Moreover, we will show that the surface of the original object is homeomorphic to the result of the marching cubes algorithm. This is really surprising since it means that the well-known topological problems of the marching cubes reconstruction simply do not occur for digital images of r-regular objects. Based on the trilinear interpolation, we also construct a smooth isosurface from the digital image that has the same topology as the original surface. Finally, we give a surprisingly simple topology preserving reconstruction method by using overlapping balls instead of cubical voxels. This is the first approach of digitizing 3D objects which guarantees topology preservation and gives an upper bound for the geometric distortion. Since the output can be chosen as a pure voxel presentation, a union of balls, a reconstruction by trilinear interpolation, a smooth isosurface, or the piecewise linear marching cubes surface, the results are directly applicable to a huge class of image analysis algorithms. Moreover, we show how one can efficiently estimate the volume and the surface area of 3D objects by looking at their digitizations. Measuring volume and surface area of digital objects are important problems in 3D image analysis. Good estimators should be multigrid convergent, i.e., the error goes to zero with increasing sampling density. We will show that every presented reconstruction method can be used for volume estimation and we will give a solution for the much more difficult problem of multigrid-convergent surface area estimation. Our solution is based on simple counting of voxels and we are the first to be able to give absolute bounds for the surface area.     

Object detection using image reconstruction with PCA

In this paper, we present an object detection system and its application to pedestrian detection in still images, without assuming any a priori knowledge about the image. The system works as follows: in a first stage a classifier examines each location in the image at different scales. Then in a second stage the system tries to eliminate false detections based on heuristics. The classifier is based on the idea that Principal Component Analysis (PCA) can compress optimally only the kind of images that were used to compute the principal components (PCs), and that any other kind of images will not be compressed well using a few components. Thus the classifier performs separately the PCA from the positive examples and from the negative examples; when it needs to classify a new pattern it projects it into both sets of PCs and compares the reconstructions, assigning the example to the class with the smallest reconstruction error. The system is able to detect frontal and rear views of pedestrians, and usually can also detect side views of pedestrians despite not being trained for this task. Comparisons with other pedestrian detection systems show that our system has better performance in positive detection and in false detection rate. Additionally, we show that the performance of the system can be further improved by combining the classifier based on PCA reconstruction with a conventional classifier using a Support Vector Machine.