Silhouette and stereo fusion for 3D object modeling

In this paper, we present a new approach to high quality 3D object reconstruction. Starting from a calibrated sequence of color images, the algorithm is able to reconstruct both the 3D geometry and the texture. The core of the method is based on a deformable model, which defines the framework where texture and silhouette information can be fused. This is achieved by defining two external forces based on the images: a texture driven force and a silhouette driven force. The texture force is computed in two steps: a multi-stereo correlation voting approach and a gradient vector flow diffusion. Due to the high resolution of the voting approach, a multi-grid version of the gradient vector flow has been developed. Concerning the silhouette force, a new formulation of the silhouette constraint is derived. It provides a robust way to integrate the silhouettes in the evolution algorithm. As a consequence, we are able to recover the contour generators of the model at the end of the iteration process. Finally, a texture map is computed from the original images for the reconstructed 3D model.     

Match propagation for image-based modeling and rendering

This paper presents a quasi-dense matching algorithm between images based on the match propagation principle. The algorithm starts from a set of sparse seed matches, then propagates to the neighboring pixels by the best-first strategy, and produces a quasi-dense disparity map. The quasi-dense matching aims at broad modeling and visualization applications which rely heavily on matching information. Our algorithm is robust to initial sparse match outliers due to the best-first strategy. It is efficient in time and space as it is only output sensitive. It handles half-occluded areas because of the simultaneous enforcement of newly introduced discrete 2D gradient disparity limit and the uniqueness constraint. The properties of the algorithm are discussed and empirically demonstrated. The quality of quasi-dense matching are validated through intensive real examples.     

Interactive image-based modeling of macrostructured textures

This article addresses the modeling aspects of macrostructured texture synthesis often avoided by other methods. We aim to develop a system that provides an intuitive, interactive, continuous, and easy-to-use control to users during the entire synthesis process. Therefore, we propose efficient new solutions for the following four problems: interactive and easy control of the macrostructure attributes; easy specification of the shapes; controlled deformations and interactions to avoid repetitions. In particular, we use a random technique, which is an important factor in getting more natural-looking structures; use of digitized texture pictures as natural models to guide the synthesis and obtain high-quality results     

A robust hybrid image-based modeling system

This paper presents a new robust image-based modeling system for creating high-quality 3D models of complex objects from a sequence of unconstrained photographs. The images can be acquired by a video camera or hand-held digital camera without the need of camera calibration. In contrast to previous methods, we integrate correspondence-based and silhouette-based approaches, which significantly enhances the reconstruction of objects with few visual features (e.g., uni-colored objects) and improves surface smoothness. Our solution uses a mesh segmentation and charting approach in order to create a low-distortion mesh parameterization suitable for objects of arbitrary genus. A high-quality texture is produced by first parameterizing the reconstructed objects using a segmentation and charting approach, projecting suitable sections of input images onto the model, and combining them using a graph-cut technique. Holes in the texture due to surface patches without projecting input images are filled using a novel exemplar-based inpainting method which exploits appearance space attributes to improve patch search, and blends patches using Poisson-guided interpolation. We analyzed the effect of different algorithm parameters, and compared our system with a laser scanning-based reconstruction and existing commercial systems. Our results indicate that our system is robust, superior to other image-based modeling techniques, and can achieve a reconstruction quality visually not discernible from that of a laser scanner.     

Radiance-based color calibration for image-based modeling with multiple cameras

Photo-consistency estimation is an important part for many image-based modeling techniques.This paper presents a novel radiance-based color calibration method to reduce the uncertainty of photo-consistency estimation across multiple cameras.The idea behind our method is to convert colors into a uniform radiometric color space in which multiple image data are corrected.Experimental results demonstrate that our method can achieve comparable color calibration effect without adjusting camera parameters and is more robust than other existing method.Additionally,we obtain an auto-determined threshold for photo-consistency check,which will lead to a better performance than existing photo-consistency based reconstruction algorithms.     

Sketch recognition by fusion of temporal and image-based features

The increasing availability of pen-based hardware has recently resulted in a parallel growth in sketch-based user interfaces. Sketch-based user interfaces aim to combine the expressive power of free-hand sketching with the processing power of computers. Most sketch-based systems require intelligent ink processing capabilities, which makes the development of robust sketch recognition algorithms a primary concern in the field. So far, the research in sketch recognition has produced various independent approaches to recognition, each of which uses a particular kind of information (e.g., geometric and spatial constraints, image-based features, temporal stroke-ordering patterns). These methods were designed in isolation as stand-alone algorithms, and there has been little work treating various recognition methods as alternative sources of information that can be combined to increase sketch recognition accuracy. In this paper, we focus on two such methods and fuse an image-based method with a time-based method in an attempt to combine the knowledge of how objects look (image data) with the knowledge of how they are drawn (temporal data). In the course of combining spatial and temporal information, we also introduce a mathematically well founded fusion method for combining recognizers. Our combination method can be used for isolated sketch recognition as well as full diagram recognition. Our evaluation with two databases shows that fusing image-based and temporal features yields higher recognition rates. These results are the first to confirm the complementary nature of image-based and temporal recognition methods for full sketch recognition, which has long been suggested, but never supported by data.     

Probabilistic fusion of stereo with color and contrast for bilayer segmentation

This paper describes models and algorithms for the real-time segmentation of foreground from background layers in stereo video sequences. Automatic separation of layers from color/contrast or from stereo alone is known to be error-prone. Here, color, contrast, and stereo matching information are fused to infer layers accurately and efficiently. The first algorithm, layered dynamic programming (LDP), solves stereo in an extended six-state space that represents both foreground/background layers and occluded regions. The stereo-match likelihood is then fused with a contrast-sensitive color model that is learned on-the-fly and stereo disparities are obtained by dynamic programming. The second algorithm, layered graph cut (LGC), does not directly solve stereo. Instead, the stereo match likelihood is marginalized over disparities to evaluate foreground and background hypotheses and then fused with a contrast-sensitive color model like the one used in LDP. Segmentation is solved efficiently by ternary graph cut. Both algorithms are evaluated with respect to ground truth data and found to have similar performance, substantially better than either stereo or color/contrast alone. However, their characteristics with respect to computational efficiency are rather different. The algorithms are demonstrated in the application of background substitution and shown to give good quality composite video output.     

基于图形图像合成技术的植物建模与风中模拟

在建模过程中根据植物枝干和叶片的不同特征,采用分形图形学的方法绘制植物的枝干,并通过控制函数调整每个分枝的生长方向、长度和粗细程度等参数,可以生成不同种类、形态各异的植物分支结构,改变了原有分形方法生成结果过于规则的特点;采用图像方法绘制叶片,利用Alpha测试技术去掉图片中叶子的背景,保留复杂的边缘信息和颜色信息,通过旋转缩放等方法,可以生成各种逼真的植物叶片,并且方法简单,计算速度较快。另外,从形态学角度出发,根据枝条在风力影响下的不同形变,近似地模拟出植物在风中摇曳的过程。     

BIM-based task-level planning for robotic brick assembly through image-based 3D modeling

The application of robotics in the assembly of building bricks has become a popular topic, while the planning of construction robots is still lagged far behind the manufacturing industry. New robotic assembly task with manual teaching–planning method is always time consuming. A task-level planning method was proposed, and the implementation details were described to improve the planning efficiency of robotic brick assembly without affecting accuracy. In this work, a BIM (Building Information Model)-based robotic assembly model that contains all the required information for planning was proposed. Image-based 3D modeling was utilized to help the calibration of the robotic assembly scene and building task models. The placement point coordinates of each assembly brick were generated in the robot base coordinate system. Finally, three different building information model tasks of modular structures (e.g., wall, stair, and pyramid) were designed. The feasibility and effectiveness of the proposed method were verified by comparing the efficiency and accuracy of three models through manual teaching and task-level planning.     

Volumetric modeling in laser BPH therapy simulation

In this paper, we introduce a novel application of volume modeling techniques on laser Benign Prostatic Hyperplasia (BPH) therapy simulation. The core technique in our system is an algorithm for simulating the tissue vaporization process by laser heating. Different from classical volume CSG operations, our technique takes experimental data as the guidance to determine the vaporization amount so that only a specified amount of tissue is vaporized in each time. Our algorithm uses a predictor-corrector strategy. First, we apply the classical CSG algorithm on a tetrahedral grid based distance field to estimate the vaporized tissue amount. Then, a volume-correction phase is applied on the distance field. To improve the performance, we further propose optimization approaches for efficient implementation.     

Omnidirectional image-based modeling: three approaches to approximated plenoptic representations

In this paper we present a set of novel methods for image-based modeling using omnidirectional vision sensors. The basic idea is to directly and efficiently acquire plenoptic representations by using omnidirectional vision sensors. The three methods, in order of increasing complexity, are direct memorization, discrete interpolation, and smooth interpolation. Results of these methods are compared visually with ground-truth images taken from a standard camera walking along the same path. The experimental results demonstrate that our methods are successful at generating high-quality virtual images. In particular, the smooth interpolation technique approximates the plenoptic function most closely. A comparative analysis of the computational costs associated with the three methods is also presented. Correspondence to: H. Ishiguro (e-mail: ishiguro@sys.wakayama-u.ac.jp)     

一种基于双目SLAM的环境建模方法

利用计算机视觉对环境进行建模是机器人环境感知相关研究的一项重要工作。提出了一种基于双目SLAM的建模方法。该方法的主要思想是利用双目SLAM(Simultaneous Localization and Mapping,同步定位与地图构建)技术获取相机位姿,并利用双目相机直接获取场景的深度图,然后将二者融合成场景的点云地图。由于双目相机直接获取的深度图像误差点较多,因此提出了一种基于视线约束的点云滤波方法,来消除场景点云中的误差点。经过实验验证了建模的效果,以及视线约束点云滤波的有效性。     

基于四目立体视觉的三维全自动建模

为保护并传承敦煌莫高窟世界文化遗产,研究基于四目立体视觉系统的三维全自动建模技术。由于提取的点云数量较大,导致自动建模过程中,点云的三角构网步骤无法顺利进行,因此着重研究点云的精简算法。将包围盒算法与点云的曲率特征相结合,提出一种点云精简算法,即改进的包围盒法。实验结果表明,该方法对点云的采样精度高于原始包围盒法,点云精简率可达93.6%,点云数据精简后三维建模效率大大提高,生成的三维模型精度较高。     

复杂背景下基于图像融合的运动目标轮廓提取算法

运动目标轮廓的有效提取对于目标识别、跟踪和行为的理解等后期的处理是非常重要的。受背景复杂性的影响,当背景灰度和运动目标的灰度相近时,提取的运动目标易产生空洞,某些部位无法完全恢复。根据帧差法的基本原理,提出了一种针对复杂背景的运动目标检测、轮廓提取方法。首先,对图像进行滤波处理,采用最大方差比阈值法消除了剩余部分噪声和背景,然后在三帧时间差分法基础上,利用序列中多帧图像融合运动信息,并确定参考区域,通过对原图像进行回扫描,最终提取出完整的运动目标轮廓。实验结果验证了算法的稳健性和有效性。     

基于双目融合的无参考立体图像质量评价

针对对称失真和非对称失真图像的评价问题,提出了一种基于双目融合的无参考立体图像质量评价方法。首先,分别将立体图像的左、右视点图像分解成拉普拉斯金字塔序列,利用图像平均梯度和区域能量确定各层融合系数,在双目加权模型的基础上逐层融合两序列并重构合成图像。然后,提取左、右视点图像、合成图像的多尺度多方向频域变换特征和对比度、熵、能量、逆差分矩特征。最后,将特征参数作为支持向量回归模型的输入进行训练并预测图像质量。在LIVE 3D phaseⅠ和LIVE 3D phaseⅡ图像库上作相关性分析,其Pearson线性相关系数和Spearman等级相关系数均分别达到0.96和0.95以上。结果表明,本文方法对立体图像质量的预测结果与主观评价值具有较高的一致性。     

基于立体视觉和RBF变形技术的个性化人手建模

由于人手形状复杂、细节丰富,构建个性化的人手模型充满挑战。提出一种基于立体视觉和RBF几何变形技术的个性化人手建模方法,通过综合采用轮廓点和SIFT点描述手部特征,提取用户手部3维特征点云,并以点云为约束,采用几何对齐两步法对齐手模板,结合RBF插值算法,得到具有用户特征的个性化手模型。实验结果表明,该方法可以有效地重建出用户手的几何模型。     

Near-real-time stereo matching with slanted surface modeling and sub-pixel accuracy

This paper presents a new stereo matching algorithm which takes into consideration surface orientation at the per-pixel level. Two disparity calculation passes are used. The first pass assumes that surfaces in the scene are fronto-parallel and generates an initial disparity map, from which the disparity plane orientations of all pixels are estimated and refined. In the second pass, the matching costs for different pixels are aggregated along the estimated disparity plane orientations using adaptive support weights, where the support weights of neighboring pixels are calculated using a combination of four terms: a spatial proximity term, a color similarity term, a disparity similarity term, and an occlusion handling term. The disparity search space is quantized at sub-pixel level to improve the accuracy of the disparity results. The algorithm is designed for parallel execution on Graphics Processing Units (GPUs) for near-real-time processing speed. The evaluation using Middlebury benchmark shows that the presented approach outperforms existing real-time and near-real-time algorithms in terms of subpixel level accuracy.     

Three-dimensional human motion modeling by back-projection based on image-based camera calibration

This article proposes a back-projection technique for the modeling of 3-D human motion that performs camera calibration using the multiple video sequences obtained. This technique calculates an affine camera matrix by the factorization method, and performs back-projection under the affine camera model. The proposed technique needs neither a 3-D camera calibration tool nor markers for shape recovery, and can recover human motion from silhouette images. We also propose a shadow detector and eliminator using color information and normalized cross correlation for the robust extraction and elimination of shadows. Experimental results show the effectiveness of the proposed technique.