Real-Time Omnidirectional Image Sensors

Conventional T.V. cameras are limited in their field of view. A real-time omnidirectional camera which can acquire an omnidirectional (360 degrees) field of view at video rate and which could be applied in a variety of fields, such as autonomous navigation, telepresence, virtual reality and remote monitoring, is presented. We have developed three different types of omnidirectional image sensors, and two different types of multiple-image sensing systems which consist of an omnidirectional image sensor and binocular vision. In this paper, we describe the outlines and fundamental optics of our developed sensors and show examples of applications for robot navigation.     

Multiple homographies with omnidirectional vision for robot homing

Two relevant issues in vision-based navigation are the field-of-view constraints of conventional cameras and the model and structure dependency of standard approaches. A good solution of these problems is the use of the homography model with omnidirectional vision. However, a plane of the scene will cover only a small part of the omnidirectional image, missing relevant information across the wide range field of view, which is the main advantage of omnidirectional sensors. The interest of this paper is in a new approach for computing multiple homographies from virtual planes using omnidirectional images and its application in an omnidirectional vision-based homing control scheme. The multiple homographies are robustly computed, from a set of point matches across two omnidirectional views, using a method that relies on virtual planes independently of the structure of the scene. The method takes advantage of the planar motion constraint of the platform and computes virtual vertical planes from the scene. The family of homographies is also constrained to be embedded in a three-dimensional linear subspace to improve numerical consistency. Simulations and real experiments are provided to evaluate our approach.     

Acquiring a radiance distribution to superimpose virtual objectsonto a real scene

This paper describes a new method for superimposing virtual objects with correct shadings onto an image of a real scene. Unlike the previously proposed methods, our method can measure a radiance distribution of a real scene automatically and use it for superimposing virtual objects appropriately onto a real scene. First, a geometric model of the scene is constructed from a pair of omnidirectional images by using an omnidirectional stereo algorithm. Then, radiance of the scene is computed from a sequence of omnidirectional images taken with different shutter speeds and mapped onto the constructed geometric model. The radiance distribution mapped onto the geometric model is used for rendering virtual objects superimposed onto the scene image. As a result, even for a complex radiance distribution, our method can superimpose virtual objects with convincing shadings and shadows cast onto the real scene. We successfully tested the proposed method by using real images to show its effectiveness     

遥感图像辅助的折反射全向图三维重构

提出了一种针对室外场景的单幅折反射全向图三维重构方法,能够自动重构出全向图中360°视野内景物的三维模型,并实现自由漫游。基于全向图与遥感图匹配把全向图分为水平地面、垂直建筑物立面和垂直背景景物面三类区域,得到全向图场景的基本结构;在此基础上利用折反射光路投射模型计算出全向图中每个像素点的三维几何位置,从而实现了折反射全向图的重构。实验证明该方法具有采集简单、视野大、处理过程全自动化、能够重构非平面场景等特点。     

Effects of characteristics of image quality in an immersive environment

Image quality issues such as field of view (FOV) and resolution are important for evaluating "presence" and simulator sickness (SS) in virtual environments (VEs). This research examined effects on postural stability of varying FOV, image resolution, and scene content in an immersive visual display. Two different scenes (a photograph of a fountain and a simple radial pattern) at two different resolutions were tested using six FOVs (30, 60, 90, 120, 150, and 180 deg.). Both postural stability, recorded by force plates, and subjective difficulty ratings varied as a function of FOV, scene content, and image resolution. Subjects exhibited more balance disturbance and reported more difficulty in maintaining posture in the wide-FOV, high-resolution, and natural scene conditions.     

基于图象的绘制中的纹理编辑

现有的基于图象的绘制 (IBR)系统中 ,只能提供新视点的图象 ,并不具有对场景编辑的功能 .由于 IBR系统的场景模型不同于传统图形学 ,难于直接应用传统图形学中已有的方法进行编辑 .为此提出了一种修改 IBR场景中物体的表面纹理的方法 ,这种方法通过在原始的参考图象上指出编辑区域并给出纹理定义 ,修改各个视点上的纹理 .这种方法不限制编辑区域的形状、尺寸 ,并且考虑了亮度信息 ,适应了环境光照 ,使新纹理与原有场景更好地融合在一起 .实验证明 ,场景的复杂程度并不影响本方法的效果 ,但是由于本方法不能得到场景物体准确的三维数学模型 ,无法用现有图形引擎进行绘制 ,因此无法实现硬件加速 ,不适用于过大的场景     

基于AutoCAD和3DSMAX的虚拟校园三维场景构建研究

当前的虚拟校园三维场景构建方法是利用osgEarth获取GoogleEarth数据,构建虚拟校园三维地图,然后依据本地地形信息与三维场景模型完成对虚拟校园的三维场景构建;该方法在三维场景构建过程中没有考虑到校园后续建设发展,只凭现在所需进行三维场景构建,导致虚拟校园三维场景杂乱,存在虚拟校园三维场景构建顾虑不周的问题;为此,提出一种基于AutoCAD和3DSMAX的虚拟校园三维场景构建方法;该方法首先利用Kinect的RGB深度摄像头获取校园平面数据、影像数据和贴图资料,采用相机阵列合成算法对所得校园图像进行特征提取,然后依据非局部均值滤波方法对获取的校园图像进行去噪操作,最后利用边缘方向和梯度表示中的结构方向与强度,对校园图像外轮廓的纹理特征进行收敛,依据外轮廓纹理特征收敛实现校园图像内轮廓的准确定位,将校园图像与虚拟校园三维场景构建所需图像进行匹配,由此完成对虚拟校园三维场景的构建;实验结果证明,所提方法是虚拟校园三维场景构建领域革命性的突破,提高了学校的招生率,增加了校园文化历史的展现机会,为该领域的研究发展提供了有效依据。     

反馈控制采集的双模式折反射全景成像处理系统

为解决折反射全景成像系统中的采样分辨率与帧速之间的矛盾,通过反馈控制图像传感器的参数配置,设计并实现了一个全局与局部双模式的全景成像处理系统.在全局模式下,为保证采集帧速率,使用行列跳跃模式采集完整的环状全向图,并将其展开为包含360°全局信息的柱面全景图像;在局部模式下,根据感兴趣区域的空间位置设置采样窗口大小及位置,以逐像素采样模式只采集全向图中相应的一块矩形区域,并将其展开为对应视线方向上高清晰度的局部透视图;最后给出了DSP+FPGA结构的系统硬件设计方案.实验结果表明了该系统设计的科学性及实用性.     

Omnidirectional visual control of mobile robots based on the 1D trifocal tensor

The precise positioning of robotic systems is of great interest particularly in mobile robots. In this context, the use of omnidirectional vision provides many advantages thanks to its wide field of view. This paper presents an image-based visual control to drive a mobile robot to a desired location, which is specified by a target image previously acquired. It exploits the properties of omnidirectional images to preserve the bearing information by using a 1D trifocal tensor. The main contribution of the paper is that the elements of the tensor are introduced directly in the control law and neither any a priori knowledge of the scene nor any auxiliary image are required. Our approach can be applied with any visual sensor obeying approximately a central projection model, presents good robustness to image noise, and avoids the problem of a short baseline by exploiting the information of three views. A sliding mode control law in a square system ensures stability and robustness for the closed loop. The good performance of the control system is proven via simulations and real world experiments with a hypercatadioptric imaging system.     

大规模虚拟环境中实时阴影生成技术

为了解决大规模场景中阴影生成的真实性和实时性问题,通过平均分割和对数分割相结合的方法,利用平行于视觉投影面的分割面将视截体划分成不同的深度部分;然后对每个分割部分生成对应的阴影图;最后利用多个阴影图代替单个阴影图进行渲染。这种方法既减少了运行时的缓冲空间,又提高了阴影质量,尤其适合于动态大规模环境中实时阴影的生成。用VS2005和OpenSceneGraph实现了算法的绘制,达到满意的视觉效果和生成速度。     

Omnidirectional Vision and Inertial Clues for Robot Navigation

The structural features inherent in the visual motion field of a mobile robot contain useful clues about its navigation. The combination of these visual clues and additional inertial sensor information may allow reliable detection of the navigation direction for a mobile robot and also the independent motion that might be present in the 3D scene. The motion field, which is the 2D projection of the 3D scene variations induced by the camera‐robot system, is estimated through optical flow calculations. The singular points of the global optical flow field of omnidirectional image sequences indicate the translational direction of the robot as well as the deviation from its planned path. It is also possible to detect motion patterns of near obstacles or independently moving objects of the scene. In this paper, we introduce the analysis of the intrinsic features of the omnidirectional motion fields, in combination with gyroscopical information, and give some examples of this preliminary analysis. © 2004 Wiley Periodicals, Inc.     

全方位图像展开成全景和透视图的实现方法

全方位视觉系统生成的全方位图像能获得水平方向3600环境信息，非常适合于某些有大视场需求的应用。该文介绍了全方位图像的特点、生成原理及其展开成全景图及透视图的具体实现方法。同时也介绍了以DirectShow为工具，全方位视频的展开实现方法。最后通过实验证明了全景图及透视图生成方法的有效性。     

基于全景视觉的监控系统设计

设计了一个基于全景视觉的多摄像机监控网络。全景相机视野广,可以实现大范围的目标检测与跟踪。云台摄像机视角具有一定的自由度,可以捕捉目标的高分辨率图像。将全景相机与云台相机相互配合,通过多传感器的数据融合,分层次的跟踪算法及多相机调度算法,实现了大范围的多个运动目标的检测与跟踪,并能捕获目标的清晰图像。实验验证了该系统的有效性和合理性。     

面向数字地球的虚拟现实系统关键技术研究

研究了面向数字地球的虚拟现实系统关键技术，提出了多层场景引擎（MLSE）、视点相关动态多分辨率地形模型（VDDMTM）和ActiveX构件网络模型。MLSE采用层机制集成多元数据，实现场景绘制和交互。VDDMTM基于四叉树表示多分辨率地形，并结合多分辨率纹理映射，根据视点位置对大规模地形实时简化，既提高了场景绘制速度，又保留了视点相关细节。网络模型采用基于Client/Server结构的ActiveX构件方案，既满足数字地球的建模特性，又支持标准的WWW浏览器。在此基础上开发了一个面向数字地球的原型系统－虚拟地球系统VES（VirtualEarthSystem），并用于虚拟永定河和喜马拉雅山。     

基于球面全景图的虚拟场景实时漫游系统

设计并实现一种基于球面全景图的虚拟场景实时漫游系统。虚拟场景的实时漫游可以通过对球面全景图进行重投影完成。通过重投影球面全景图的可视部分到视平面上,可以生成虚拟场景在不同视线方向上的透视视图。使用球面全景图的重投影算法可以模拟相机的旋转运动,通过改变相机的视域,可以模拟相机的变焦运动。针对直接使用重投影不能满足球面全景图实时绘制的问题,在对球面全景图的重投影算法进行仔细分析的基础上,提出使用查找表和增量计算进行加速的策略。实验结果表明,优化后的系统能够对基于球面全景图的虚拟场景进行实时漫游     

Spatial layout recovery from a single omnidirectional image and its matching-free sequential propagation

The goal of this work is to recover the spatial layout of indoor environments from omnidirectional images assuming a Manhattan world structure. We propose a new method for scene structure recovery from a single image. This method is based on the line extraction for omnidirectional images, line classification, and vanishing points estimation combined with a new hierarchical expansion procedure for detecting floor and wall boundaries. Each single omnidirectional image independently provides a useful hypothesis of the 3D scene structure. In order to enhance the robustness and accuracy of this single image-based hypothesis, we extend this estimation with a new homography-based procedure applied to the various hypotheses obtained along the sequence of consecutive images. A key point in this contribution is the use of geometrical constraints for computing the homographies from a single line of the floor. The homography parametrization proposed allows the design of a matching-free method for spatial layout propagation along a sequence of images. Experimental results show single image layout recovery performance and the improvement obtained with the propagation of the hypothesis through the image sequence.     

A novel mobile robot localization approach based on topological maps using classification with reject option in omnidirectional images

Mobile robot localization, which allows a robot to identify its position, is one of main challenges in the field of Robotics. In this work, we provide an evaluation of consolidated feature extractions and machine learning techniques from omnidirectional images focusing on topological map and localization tasks. The main contributions of this work are a novel method for localization via classification with reject option using omnidirectional images, as well as two novel omnidirectional image data sets. The localization system was analyzed in both virtual and real environments. Based on the experiments performed, the Minimal Learning Machine with Nearest Neighbors classifier and Local Binary Patterns feature extraction proved to be the best combination for mobile robot localization with accuracy of 96.7% and an Fscore of 96.6%.     

基于大位移视点图像的单帧图像修复

提出一种基于大位移视点图像的单帧图像修复算法,利用大位移视点图像中的可见信息修补目标图像中的被遮挡或信息丢失区域.算法的关键在于如何转化大位移视点图像的可见信息为可用信息,以及如何利用得到的可用信息来有效地修补目标图像.在交互指定待修复的目标区域后,算法首先将所有图像分割为不同的平面场景区域,并基于图像匹配将大位移视点图像中的平面场景区域变换到当前视点.因此,其中的可见信息就可被直接使用.进而通过定义合适的修复和融合优先级函数,提出基于纹理合成和图像融合的图像修复算法,利用获得的可用信息来修补目标区域.修复区域和目标图像之间的鬼影现象使用Poisson图像融合算法来消除,以达到无缝的修复结果.实验结果表明,该算法能够修复较大的丢失信息区域中的结构和纹理信息,具有一定的实用价值.     

视野驱动的自适应阴影图

阴影图是当前实时阴影绘制中的一种经典算法。该算法基于图像空间,当有限分辨率的阴影图映射到较大场景中时,就会由于采样不足造成锯齿形变走样。提出了一种实时的反走样阴影图算法,该算法首先获取当前视点所能够看到的场景范围,然后绘制该范围内的阴影图,并映射到场景中生成实时阴影。该方法同经典的阴影图算法相比,避免了场景中不必要的阴影绘制,提高了阴影图的利用率,反走样的效果很好。而且,该方法只需要绘制一到两幅阴影图,算法的实时性很强,可以满足一个上百万面片的大规模场景中实时阴影绘制的需要。