3D face reconstruction using images from cameras with varying parameters

In this paper, we present a new technique of 3D face reconstruction from a sequence of images taken with cameras having varying parameters without the need to grid. This method is based on the estimation of the projection matrices of the cameras from a symmetry property which characterizes the face, these projections matrices are used with points matching in each pair of images to determine the 3D points cloud, subsequently, 3D mesh of the face is constructed with 3D Crust algorithm. Lastly, the 2D image is projected on the 3D model to generate the texture mapping. The strong point of the proposed approach is to minimize the constraints of the calibration system: we calibrated the cameras from a symmetry property which characterizes the face, this property gives us the opportunity to know some points of 3D face in a specific well-chosen global reference, to formulate a system of linear and nonlinear equations according to these 3D points, their projection in the image plan and the elements of the projections matrix. Then to solve these equations, we use a genetic algorithm which consists of finding the global optimum without the need of the initial estimation and allows to avoid the local minima of the formulated cost function. Our study is conducted on real data to demonstrate the validity and the performance of the proposed approach in terms of robustness, simplicity, stability and convergence.     

PCA-based 3D shape reconstruction of human foot using multiple viewpoint cameras

This paper describes a multiple camera-based method to reconstruct the 3D shape of a human foot. From a foot database, an initial 3D model of the foot represented by a cloud of points is built. The shape parameters, which can characterize more than 92% of a foot, are defined by using the principal component analysis method. Then, using ＂active shape models＂, the initial 3D model is adapted to the real foot captured in multiple images by applying some constraints （edge points＇ distance and color variance）. We insist here on the experiment part where we demonstrate the efficiency of the proposed method on a plastic foot model, and also on real human feet with various shapes. We propose and compare different ways of texturing the foot which is needed for reconstruction. We present an experiment performed on the plastic foot model and on human feet and propose two different ways to improve the final 3D shapers accuracy according to the previous experiments＇ results. The first improvement proposed is the densification of the cloud of points used to represent the initial model and the foot database. The second improvement concerns the projected patterns used to texture the foot. We conclude by showing the obtained results for a human foot with the average computed shape error being only 1.06 mm.     

Photorealistic 3D reconstruction from handheld cameras

One of the major challenges in the fields of computer vision and computer graphics is the construction and representation of life-like virtual 3D scenarios within a computer. The VISIRE project attempts to reconstruct photo-realistic 3D models of large scenarios using as input multiple freehand video sequences, while rendering the technology accessible to the non-expert. VISIRE is application oriented and hence must deal with multiple issues of practical relevance that were commonly overlooked in past experiences. The paper presents both an innovative approach for the integration of previously unrelated experiences, as well as a number of novel contributions, such as: an innovative algorithm to enforce closedness of the trajectories, a new approach to 3D mesh generation from sparse data, novel techniques dealing with partial occlusions and a method for using photo-consistency and visibility constrains to refine the 3D mesh. Tomas Rodriguez was born in Madrid in 1961. Bachelor in Physics and Master in Electronics by the Universidad Complutense de Madrid. He started his career in the private R&D sector in 1987, when he specialized in computer vision and parallel processing systems. In the early nineties he participated in the EVA project; one of the most outstanding traffic monitoring system of the time. For more than 10 years, he has been involved in international research projects within the ambit of EUREKA, ESPRIT, V and VI Framework programmes. During this time, he coordinated eight international projects (CAMELOT, CITRONE, ON-LIVE, SAID, VISIRE, EVENTS, ITALES, HOLONICS) and acted as principal investigator in two additional ones (CITRUS and VICTORIA). Since the early days, he had the opportunity to collaborate with some of the most prestigious research institutions in Europe: Franhoufer Inst., INRIA, CNRS, University of Oxford, University of Lund, DFKI, Siemens C-Lab, Philips Research Labs, etc. Evaluator of R&D projects for the Spanish Ministry for Science and reviewer of international scientific journals, he is currently the R&D manager and coordinator for European projects at Eptron SA. His recent interests include: computer vision, real time software, industrial control, parallel processing, iTV, and mobile technologies, etc.     

A hierarchy of cameras for 3D photography

The view-independent visualization of 3D scenes is most often based on rendering accurate 3D models or utilizes image-based rendering techniques. To compute the 3D structure of a scene from a moving vision sensor or to use image-based rendering approaches, we need to be able to estimate the motion of the sensor from the recorded image information with high accuracy, a problem that has been well-studied. In this work, we investigate the relationship between camera design and our ability to perform accurate 3D photography, by examining the influence of camera design on the estimation of the motion and structure of a scene from video data. By relating the differential structure of the time varying plenoptic function to different known and new camera designs, we can establish a hierarchy of cameras based upon the stability and complexity of the computations necessary to estimate structure and motion. At the low end of this hierarchy is the standard planar pinhole camera for which the structure from motion problem is non-linear and ill-posed. At the high end is a camera, which we call the full field of view polydioptric camera, for which the motion estimation problem can be solved independently of the depth of the scene which leads to fast and robust algorithms for 3D Photography. In between are multiple view cameras with a large field of view which we have built, as well as omni-directional sensors.     

3D model reconstruction with common hand-held cameras

A 3D model reconstruction workflow with hand-held cameras is developed. The exterior and interior orientation models combined with the state-of-the-art structure from motion and multi-view stereo techniques are applied to extract dense point cloud and reconstruct 3D model from digital images. An overview of the presented 3D model reconstruction methods is given. The whole procedure including tie point extraction, relative orientation, bundle block adjustment, dense point production and 3D model reconstruction is all reviewed in brief. Among them, we focus on bundle block adjustment procedure; the mathematical and technical details of bundle block adjustment are introduced and discussed. Finally, four scenes of images collected by hand-held cameras are tested in this paper. The preliminary results have shown that sub-pixel (<1 pixel) accuracy can be achieved with the proposed exterior–interior orientation models and satisfactory 3D models can be reconstructed using images collected by hand-held cameras. This work can be applied in indoor navigation, crime scene reconstruction, heritage reservation and other applications in geosciences.     

Virtual pottery: a virtual 3D audiovisual interface using natural hand motions

In this paper, we present our approach towards designing and implementing a virtual 3D sound sculpting interface that creates audiovisual results using hand motions in real time. In the interface “Virtual Pottery,” we use the metaphor of pottery creation in order to adopt the natural hand motions to 3D spatial sculpting. Users can create their own pottery pieces by changing the position of their hands in real time, and also generate 3D sound sculptures based on pre-existing rules of music composition. The interface of Virtual Pottery can be categorized by shape design and camera sensing type. This paper describes how we developed the two versions of Virtual Pottery and implemented the technical aspects of the interfaces. Additionally, we investigate the ways of translating hand motions into musical sound. The accuracy of the detection of hand motions is crucial for translating natural hand motions into virtual reality. According to the results of preliminary evaluations, the accuracy of both motion-capture tracking system and portable depth sensing camera is as high as the actual data. We carried out user studies, which took into account information about the two exhibitions along with the various ages of users. Overall, Virtual Pottery serves as a bridge between the virtual environment and traditional art practices, with the consequence that it can lead to the cultivation of the deep potential of virtual musical instruments and future art education programs.     

3D scanning of cultural heritage with consumer depth cameras

Three dimensional reconstruction of cultural heritage objects is an expensive and time-consuming process. Recent consumer real-time depth acquisition devices, like Microsoft Kinect, allow very fast and simple acquisition of 3D views. However 3D scanning with such devices is a challenging task due to the limited accuracy and reliability of the acquired data. This paper introduces a 3D reconstruction pipeline suited to use consumer depth cameras as hand-held scanners for cultural heritage objects. Several new contributions have been made to achieve this result. They include an ad-hoc filtering scheme that exploits the model of the error on the acquired data and a novel algorithm for the extraction of salient points exploiting both depth and color data. Then the salient points are used within a modified version of the ICP algorithm that exploits both geometry and color distances to precisely align the views even when geometry information is not sufficient to constrain the registration. The proposed method, although applicable to generic scenes, has been tuned to the acquisition of sculptures and in this connection its performance is rather interesting as the experimental results indicate.     

Naked-eye 3D imaging model using the embedded micro-conjugate mirrors within the medical micro-needle device

Microsystem Technologies - A micro-conjugate mirror (MCC) is designed and modified using a nonlinear microring resonator system, a system consists of a nonlinear microring resonator known as a...     

Constructing 3D motions from curvature and torsion profiles

A means for constructing a free-form motion in three dimensions is given. A sparse sequence of prescribed poses is used to guide the motion, and some of these can also be specified to be precision poses through which the motion must pass. The resultant motion is given in a sequence of refined poses. Geometric algebra is used to specify these in a way that is analogous to the use of complex numbers in an equivalent approach for free-form curves.     

3D GIS环境下监控摄像头空间布局设计

安防监控系统中,传统方法令摄像头的空间布局很难取得最优效果。在3DGIS环境下,提出一种视频监控系统中摄像头空间布局的方法,基于特尔斐法提取建筑布局、景观效果、安全等级、区域出入口范围、可持续发展5个影响因子,并在此基础上确定监控点选址的基本原则和评价体系,通过综合多因子评价,结合摄像头成像模型与3DGIS的分析功能,建立一个通用的监控摄像头空间布局优化方案。应用案例表明该方法可快速、有效地实现监控点的空间布局。     

FusionMLS: Highly dynamic 3D reconstruction with consumer-grade RGB-D cameras

Computational Visual Media - Multi-view dynamic three-dimensional reconstruction has typically required the use of custom shutter-synchronized camera rigs in order to capture scenes containing...     

A novel approach to 3-D gaze tracking using stereo cameras

A novel approach to three-dimensional (3-D) gaze tracking using 3-D computer vision techniques is proposed in this paper. This method employs multiple cameras and multiple point light sources to estimate the optical axis of user's eye without using any user-dependent parameters. Thus, it renders the inconvenient system calibration process which may produce possible calibration errors unnecessary. A real-time 3-D gaze tracking system has been developed which can provide 30 gaze measurements per second. Moreover, a simple and accurate calibration method is proposed to calibrate the gaze tracking system. Before using the system, each user only has to stare at a target point for a few (2-3) seconds so that the constant angle between the 3-D line of sight and the optical axis can be estimated. The test results of six subjects showed that the gaze tracking system is very promising achieving an average estimation error of under 1 degrees.     

Shape difference visualization for ancient bronze mirrors through 3D range images

Japanese archaeologists have paid special attention to ancient Chinese bronze mirrors because the mirrors may provide a key for the exact location of Yamatai State, which is one of the major archaeological controversies. Currently, archaeologists visually analyse ancient Chinese bronze mirrors for their shape difference. The practice requires a huge amount of time and effort. In this paper, we propose an automatic method for detecting the shape difference between a pair of ancient mirrors. The 3D data of the mirrors are obtained using a laser range scanner. Our algorithm then aligns them into the same coordinate and visualizes their shape differences. Our proposed algorithm provides fast and non‐damaging analysis for shape difference. Further analysis can be evaluated on our data instead of the actual mirror, so it can be performed by more than one group of archaeologists. Copyright © 2003 John Wiley & Sons, Ltd.     

Using ToF and RGBD cameras for 3D robot perception and manipulation in human environments

Robots, traditionally confined into factories, are nowadays moving to domestic and assistive environments, where they need to deal with complex object shapes, deformable materials, and pose uncertainties at human pace. To attain quick 3D perception, new cameras delivering registered depth and intensity images at a high frame rate hold a lot of promise, and therefore many robotics researchers are now experimenting with structured-light RGBD and Time-of-Flight (ToF) cameras. In this paper both technologies are critically compared to help researchers to evaluate their use in real robots. The focus is on 3D perception at close distances for different types of objects that may be handled by a robot in a human environment. We review three robotics applications. The analysis of several performance aspects indicates the complementarity of the two camera types, since the user-friendliness and higher resolution of RGBD cameras is counterbalanced by the capability of ToF cameras to operate outdoors and perceive details.     

3D interactive system based on vision computing of direct‐flective cameras

A bare‐finger 3D interactive technology for portable devices was developed. Using directive‐flective cameras to reform the field of viewing, a blind working range close to the camera is eliminated. Moreover, the algorithm of vision computing, different from skin color detection, is presented to determine the positions of fingertips. The interactive range is workable from 1.5 to 50 cm above the entire surface of the display. The mean position error of less than 1 cm is achieved. This accuracy realizes a camera‐based 3D interactive system allowing for near‐distance functionality. Therefore, floating 3D images can be touched and interacted with, potentially creating more application and intuitive user‐machine interface.     

3-D analysis of functional instabilities of the ankle using digital still cameras

A technique to analyze the functional instability of an ankle is proposed. The technique is based on measurements using two digital still cameras. We carried out several experiments using this technique. Subjects are requested to stand on an experimental table whose angle of inclination is adjusted. By comparing the postures of both feet, we could tell that the subjects had experienced an ankle sprain and had functional instabilities in their ankles.This work was presented in part at the 8th International Symposium on Artificial Life and Robotics, Oita, Japan, January 24–26, 2003     

物体三维形状恢复的遗传算法方法

提出了一种新的物体三维形状恢复的遗传算法方法。用固定位置的摄像机在不同位置的球扩展光源下获取图象序列,并用遗传算法对物体表面每一点的法线矢量进行快速搜索。实验结果表明,此方法能有效地恢复扩展光源下物体的三维形状。不仅放宽了对光源和物体表面的限制,而且精度及鲁棒性均有很大提高。     

从二次曲线对应恢复刚体三维运动和结构

研究从二次曲线恢复刚体的三维运动参数和结构参数问题.建立了基于二次曲线的几何模型和运动模型,从理论上证明最少需要3条对应的二次曲线,建立一个由6个非线性方程构成的非线性方程组.能求出刚体的旋转运动参数和包含一个比例因子(不等于零)的平移运动参数.在此基础上得到空间二次曲线的参数,对应于每一组运动参数,空间二次曲线的参数有两个解.     

3D Structure Recovery and Unwarping of Surfaces Applicable to Planes

The deformation of applicable surfaces such as sheets of paper satisfies the differential geometric constraints of isometry (lengths and areas are conserved) and vanishing Gaussian curvature. We show that these constraints lead to a closed set of equations that allow recovery of the full geometric structure from a single image of the surface and knowledge of its undeformed shape. We show that these partial differential equations can be reduced to the Hopf equation that arises in non-linear wave propagation, and deformations of the paper can be interpreted in terms of the characteristics of this equation. A new exact integration of these equations is developed that relates the 3-D structure of the applicable surface to an image. The solution is tested by comparison with particular exact solutions. We present results for both the forward and the inverse 3D structure recovery problem.