GPU-based parallel construction of compact visual hull meshes

Building a visual hull model from multiple two-dimensional images provides an effective way of understanding the three-dimensional geometries inherent in the images. In this paper, we present a GPU accelerated algorithm for volumetric visual hull reconstruction that aims to harness the full compute power of the many-core processor. From a set of binary silhouette images with respective camera parameters, our parallel algorithm directly outputs the triangular mesh of the resulting visual hull in the indexed face set format for a compact mesh representation. Unlike previous approaches, the presented method extracts a smooth silhouette contour on the fly from each binary image, which markedly reduces the bumpy artifacts on the visual hull surface due to a simple binary in/out classification. In addition, it applies several optimization techniques that allow an efficient CUDA implementation. We also demonstrate that the compact mesh construction scheme can easily be modified for also producing a time- and space-efficient GPU implementation of the marching cubes algorithm.     

三维人体点云模型多约束肢体分割

针对人体点云模型的肢体分割这一动作识别和虚拟重建领域的重要问题,提出了一种基于分类骨架线、测地距离、特征点和姿态分析的多约束肢体分割算法,通过生成点云模型的分类骨架线,配合测地距离获得人体各部位粗分割点云集,利用测地路径方法实现关键特征点的定位,并利用曲线拟合方式进行定位优化,针对头颈、上肢、下肢和躯干之间关联部位的解剖学特征,构造多种约束条件,对各部位粗分割点云集进行了优化再分割。实验结果表明,所提算法对站姿条件下的不同动作、不同体型、不同精度人体点云模型均能取得与视觉理解相吻合的分割效果。通过该算法得到的肢体各部分点云数据可用于姿态分析等后续研究。     

A volumetric fusion technique for surface reconstruction from silhouettes and range data

Optical triangulation, an active reconstruction technique, is known to be an accurate method but has several shortcomings due to occlusion and laser reflectance properties of the object surface, that often lead to holes and inaccuracies on the recovered surface. Shape from silhouette, on the other hand, as a passive reconstruction technique, yields robust, hole-free reconstruction of the visual hull of the object. In this paper, a hybrid surface reconstruction method that fuses geometrical information acquired from silhouette images and optical triangulation is presented. Our motivation is to recover the geometry from silhouettes on those parts of the surface which the range data fail to capture. A volumetric octree representation is first obtained from the silhouette images and then carved by range points to amend the missing cavity information. An isolevel value on each surface cube of the carved octree structure is accumulated using local surface triangulations obtained separately from range data and silhouettes. The marching cubes algorithm is then applied for triangulation of the volumetric representation. The performance of the proposed technique is demonstrated on several real objects.     

Seamless,Static Multi‐Texturing of 3D Meshes

In the context of 3D reconstruction, we present a static multi‐texturing system yielding a seamless texture atlas calculated by combining the colour information from several photos from the same subject covering most of its surface. These pictures can be provided by shooting just one camera several times when reconstructing a static object, or a set of synchronized cameras, when dealing with a human or any other moving object. We suppress the colour seams due to image misalignments and irregular lighting conditions that multi‐texturing approaches typically suffer from, while minimizing the blurring effect introduced by colour blending techniques. Our system is robust enough to compensate for the almost inevitable inaccuracies of 3D meshes obtained with visual hull–based techniques: errors in silhouette segmentation, inherently bad handling of concavities, etc.     

Image-based Virtual Exhibit and Its Extension to 3D

In this paper we introduce an image-based virtual exhibition system especially for clothing product.It can provide a powerful material substitution function,which is very useful for customization clothing-built.A novel color substitution algorithm and two texture morphing methods are designed to ensure realistic substitution result.To extend it to 3D,we need to do the model reconstruction based on photos.Thus we present an improved method for modeling human body.It deforms a generic model with shape details extracted from pictures to generate a new model.Our method begins with model image generation followed by silhouette extraction and segmentation.Then it builds a mapping between pixeis inside every pair of silhouette segments in the model image and in the picture.Our mapping algorithm is based on a slice space representation that conforms to the natural features of human body.     

Silhouette coherence for camera calibration under circular motion

We present a new approach to camera calibration as a part of a complete and practical system to recover digital copies of sculpture from uncalibrated image sequences taken under turntable motion. In this paper, we introduce the concept of the silhouette coherence of a set of silhouettes generated by a 3D object. We show how the maximization of the silhouette coherence can be exploited to recover the camera poses and focal length. Silhouette coherence can be considered as a generalization of the well-known epipolar tangency constraint for calculating motion from silhouettes or outlines alone. Further, silhouette coherence exploits all the geometric information encoded in the silhouette (not just at epipolar tangency points) and can be used in many practical situations where point correspondences or outer epipolar tangents are unavailable. We present an algorithm for exploiting silhouette coherence to efficiently and reliably estimate camera motion. We use this algorithm to reconstruct very high quality 3D models from uncalibrated circular motion sequences, even when epipolar tangency points are not available or the silhouettes are truncated. The algorithm has been integrated into a practical system and has been tested on more than 50 uncalibrated sequences to produce high quality photo-realistic models. Three illustrative examples are included in this paper. The algorithm is also evaluated quantitatively by comparing it to a state-of-the-art system that exploits only epipolar tangents     

基于shape from silhouette方法的虚拟人体构造

逼真的人体三维模型（虚拟人体）在产品设计，计算机动画，虚拟现实等领域都有着广泛的需求，为此，提出了一种基于shape from silhouette的人体三维模型构造方法，该方法所需要的器材只是一个普通的数码相机，通过从不同角度对人体进行拍摄，然后经过摄像机定标，体积生成，表面重建，纹理映射后就可重建出逼真的人体三维模型。实验结果表明，此方法精度较高，整个过程简单快捷，是一种非常实用的方法。     

基于图像轮廓的三维重建方法

根据平面镜成像原理,提出了一种基于单幅图像的侧影轮廓线可见外壳重建方法。利用成角度平面镜装置模拟实现多相机同时拍摄,对图像中各轮廓线之间极线几何关系的分析得到对应的相机参数,实现目标物体的三维重建。实验结果表明,该方法简单有效,不需要通过实验室条件下的特殊仪器进行标定即可实现目标物体的三维重建,具有一定的实用价值。     

采用变形模型的图像序列可视外壳计算方法

针对基于图像序列变形模型三维重建中至关重要的初始模型计算问题,提出一种基于图像序列计算可视外壳的方法.该方法基于Snake活动轮廓变形模型,将真实物体可视外壳的计算问题转化为初始曲面通过内外力作用驱动收敛于目标物体的问题.以真实物体图像序列中提取的物体轮廓为输入,根据轮廓信息和光滑度信息计算内力及轮廓力大小,驱动球体初始曲面变形收敛于可视外壳;在变形过程中加入删除短边、分裂长边、对角线翻转等网格优化操作,以避免发生网格错乱.实验结果表明,文中方法有效地克服了传统的基于体元素细分产生网格算法网格质量不高的缺点,且参数可调,易于实现,占用内存少,生成的曲面更加光滑,细节恢复效果理想.     

Reconstruction of dynamic 3D scene based on visual hull and view morphing

In this paper, we propose a new method that processes multiple synchronized video sequences and generates 3D rendering of dynamic objects in the video. It exploits an efficient image‐based reconstruction scheme that constructs and shades 3D models of objects from silhouette images by combining image‐based visual hull and view morphing. The proposed hybrid method improves the speed and the quality of the previous visual hull sampling methods. We designed and implemented a system based on this method which is relatively low cost and does not require any special hardware or specific environment. Copyright © 2003 John Wiley & Sons, Ltd.     

Incorporating object-centered sampling and Delaunay tetrahedrization for visual hull reconstruction

In this paper we present a novel shape from silhouette algorithm. For an object to be modeled, the algorithm first computes a cloud of points located on a pencil of rays and distributed evenly on the visual hull surface, inside and outside the visual hull. Then Delaunay tetrahedrization is applied to the point cloud to partition its convex hull into a set of tetrahedrons. Finally, outlier tetrahedrons are removed by tetrahedron peeling, and a mesh model of the visual hull is extracted. The algorithm is robust, free from discretization artifacts, and produces a mesh model composed of well-shaped triangles.     

Computer Vision-Based Human Body Segmentation and Posture Estimation

This paper proposes a new method for vision-based human body posture estimation using body silhouette and skin-color information. A moving object segmentation algorithm is first proposed to distinguish the human body from the background using a sequence of images. This algorithm uses a fast Euler number computation technique to automatically determine the threshold of both frame and background differences. After segmentation, a sequence of image processing approaches then creates a complete silhouette of the human body. The objective of posture estimation is to locate five significant body points, including the head, tips of the feet, and tips of the hands. These significant points are first selected from convex points on a defined distance curve. A number of heuristic rules based on body shape characteristics are used to select the proper points among these convex candidates. These rules use features like the principal and minor axes of the human body, their interactions with the silhouette contour, the relative distances between convex points, and the curvature of convex points. An auxiliary skin-color feature is used when the silhouette shape features alone are not sufficient to estimate the significant points. Experimental results show that the proposed approach can efficiently and effectively locate the significant body points for most postures.      

利用运动线索的单目深度测量

目的 传统的单目视觉深度测量方法具有设备简单、价格低廉、运算速度快等优点,但需要对相机进行复杂标定,并且只在特定的场景条件下适用。为此,提出基于运动视差线索的物体深度测量方法,从图像中提取特征点,利用特征点与图像深度的关系得到测量结果。方法 对两幅图像进行分割,获取被测量物体所在区域;然后采用本文提出的改进的尺度不变特征变换SIFT（scale-invariant feature transtorm）算法对两幅图像进行匹配,结合图像匹配和图像分割的结果获取被测量物体的匹配结果;用Graham扫描法求得匹配后特征点的凸包,获取凸包上最长线段的长度;最后利用相机成像的基本原理和三角几何知识求出图像深度。结果 实验结果表明,本文方法在测量精度和实时性两方面都有所提升。当图像中的物体不被遮挡时,实际距离与测量距离之间的误差为2.60%,测量距离的时间消耗为1.577 s;当图像中的物体存在部分遮挡时,该方法也获得了较好的测量结果,实际距离与测量距离之间的误差为3.19%,测量距离所需时间为1.689 s。结论 利用两幅图像上的特征点来估计图像深度,对图像中物体存在部分遮挡情况具有良好的鲁棒性,同时避免了复杂的摄像机标定过程,具有实际应用价值。     

System for reconstruction of three-dimensional micro objects from multiple photographic images

We introduce a system to reconstruct a three-dimensiojnal (3D) polygonal model of 3D micro objects with outer dimensions ranging from several hundred microns to several millimeters from multiple two-dimensional (2D) images of an object taken from different views. The data acquisition system consists of a digital microscope that captures still images at a resolution of 1600 × 1200 pixels and a computer-controlled turntable. We employ the shape-from-silhouette (SFS) method to construct a voxel-based 3D model from silhouette images. The concave shapes are further carved by using the space carving technique. In order to make the resulting model compatible with a commercial CAD/CAM system, the voxel model is converted into a triangular mesh using the marching cubes algorithm. Because the mesh generated from the voxel model by using the marching cubes algorithm inherits the staircase effect, the mesh is adjusted to recover the object precisely by using silhouette images. Finally, we evaluate the accuracy of the proposed method. The reconstructed models of complex micro objects indicate the effectiveness of the 3D shape reconstruction system for micro objects.     

Reduced Depth and Visual Hulls of Complex 3D Scenes

Depth and visual hulls are useful for quick reconstruction and rendering of a 3D object based on a number of reference views. However, for many scenes, especially multi‐object, these hulls may contain significant artifacts known as phantom geometry. In depth hulls the phantom geometry appears behind the scene objects in regions occluded from all the reference views. In visual hulls the phantom geometry may also appear in front of the objects because there is not enough information to unambiguously imply the object positions. In this work we identify which parts of the depth and visual hull might constitute phantom geometry. We define the notion of reduced depth hull and reduced visual hull as the parts of the corresponding hull that are phantom‐free. We analyze the role of the depth information in identification of the phantom geometry. Based on this, we provide an algorithm for rendering the reduced depth hull at interactive frame‐rates and suggest an approach for rendering the reduced visual hull. The rendering algorithms take advantage of modern GPU programming techniques. Our techniques bypass explicit reconstruction of the hulls, rendering the reduced depth or visual hull directly from the reference views.     

Fast joint estimation of silhouettes and dense 3D geometry from multiple images

We propose a probabilistic formulation of joint silhouette extraction and 3D reconstruction given a series of calibrated 2D images. Instead of segmenting each image separately in order to construct a 3D surface consistent with the estimated silhouettes, we compute the most probable 3D shape that gives rise to the observed color information. The probabilistic framework, based on Bayesian inference, enables robust 3D reconstruction by optimally taking into account the contribution of all views. We solve the arising maximum a posteriori shape inference in a globally optimal manner by convex relaxation techniques in a spatially continuous representation. For an interactively provided user input in the form of scribbles specifying foreground and background regions, we build corresponding color distributions as multivariate Gaussians and find a volume occupancy that best fits to this data in a variational sense. Compared to classical methods for silhouette-based multiview reconstruction, the proposed approach does not depend on initialization and enjoys significant resilience to violations of the model assumptions due to background clutter, specular reflections, and camera sensor perturbations. In experiments on several real-world data sets, we show that exploiting a silhouette coherency criterion in a multiview setting allows for dramatic improvements of silhouette quality over independent 2D segmentations without any significant increase of computational efforts. This results in more accurate visual hull estimation, needed by a multitude of image-based modeling approaches. We made use of recent advances in parallel computing with a GPU implementation of the proposed method generating reconstructions on volume grids of more than 20 million voxels in up to 4.41 seconds.     

基于标定物的相机标定及三维重建

针对传统三维建模的局限性,讨论了一种基于图像建模的技术,提出了利用普通相机和标定物对物体进行三维建模的方法,该方法利用一个在左右图像都存在的物体,对相机进行标定[1]。然后利用左右相机的相机矩阵,反算空间中的对应点和需要求取的关键点。最后利用这些点计算出来的空间位置[2],对物体进行重建,并用OpenGL进行漫游显示。实验表明,该算法计算准确,鲁棒性很高,能够满足于虚拟现实的需要。     

A robust hybrid image-based modeling system

This paper presents a new robust image-based modeling system for creating high-quality 3D models of complex objects from a sequence of unconstrained photographs. The images can be acquired by a video camera or hand-held digital camera without the need of camera calibration. In contrast to previous methods, we integrate correspondence-based and silhouette-based approaches, which significantly enhances the reconstruction of objects with few visual features (e.g., uni-colored objects) and improves surface smoothness. Our solution uses a mesh segmentation and charting approach in order to create a low-distortion mesh parameterization suitable for objects of arbitrary genus. A high-quality texture is produced by first parameterizing the reconstructed objects using a segmentation and charting approach, projecting suitable sections of input images onto the model, and combining them using a graph-cut technique. Holes in the texture due to surface patches without projecting input images are filled using a novel exemplar-based inpainting method which exploits appearance space attributes to improve patch search, and blends patches using Poisson-guided interpolation. We analyzed the effect of different algorithm parameters, and compared our system with a laser scanning-based reconstruction and existing commercial systems. Our results indicate that our system is robust, superior to other image-based modeling techniques, and can achieve a reconstruction quality visually not discernible from that of a laser scanner.     

A multiview 3D modeling system based on stereo vision techniques

This paper introduces a stereo vision system to automatically generate 3D models of real objects. 3D model generation is based on the merging of multiview range images obtained from a digital stereo camera. Stereo images obtained from the camera are rectified, and a correlation-based stereo matching technique reconstructs range images from them. A turntable stage is also employed to obtain multiple range images of the objects. To register range images into a common coordinate system automatically, we introduce and calibrate a turntable coordinate system with respect to the camera coordinate system. After the registration of multiview range images, a 3D model is reconstructed using a volumetric integration technique. Error analysis on turntable calibration and 3D model reconstruction shows the accuracy of our 3D modeling system.Received: 2 August 2003, Accepted: 20 September 2004, Published online: 25 February 2005 Correspondence to: S.Y. Park