Railroad online: acquiring and visualizing route panoramas of rail scenes

A patrol type of surveillance has been performed everywhere from police city patrol to railway inspection. Different from static cameras or sensors distributed in a space, such surveillance has its benefits of low cost, long distance, and efficiency in detecting infrequent changes. However, the challenges are how to archive daily recorded videos in the limited storage space and how to build a visual representation for quick and convenient access to the archived videos. We tackle the problems by acquiring and visualizing route panoramas of rail scenes. We analyze the relation between train motion and the video sampling and the constraints such as resolution, motion blur and stationary blur etc. to obtain a desirable panoramic image. The route panorama generated is a continuous image with complete and non-redundant scene coverage and compact data size, which can be easily streamed over the network for fast access, maneuver, and automatic retrieval in railway environment monitoring. Then, we visualize the railway scene based on the route panorama rendering for interactive navigation, inspection, and scene indexing.     

面向应急系统的城市虚拟环境构建方法研究

提出面向应急系统的一种路途全景图像与GE相结合的虚拟环境构建方法。以汽车为移动平台,利用摄像机和GPS设备采集数据,通过视频拼接生成路途全景图像,根据GPS位置信息生成三维景物立面,再以路途全景图像为纹理对景物立面进行贴图,最后把生成的虚拟模型导入Google Earth。实验证明该方法简单实用,对硬件以及网络环境要求低,且具有较强的扩展能力,能够在一定程度上提高指挥员应急指挥和处置能力。     

Scanning scene tunnel for city traversing

This paper proposes a visual representation named scene tunnel for capturing urban scenes along routes and visualizing them on the Internet. We scan scenes with multiple cameras or a fish-eye camera on a moving vehicle, which generates a real scene archive along streets that is more complete than previously proposed route panoramas. Using a translating spherical eye, properly set planes of scanning, and unique parallel-central projection, we explore the image acquisition of the scene tunnel from camera selection and alignment, slit calculation, scene scanning, to image integration. The scene tunnels cover high buildings, ground, and various viewing directions and have uniformed resolutions along the street. The sequentially organized scene tunnel benefits texture mapping onto the urban models. We analyze the shape characteristics in the scene tunnels for designing visualization algorithms. After combining this with a global panorama and forward image caps, the capped scene tunnels can provide continuous views directly for virtual or real navigation in a city. We render scene tunnel dynamically by view warping, fast transmission, and flexible interaction. The compact and continuous scene tunnel facilitates model construction, data streaming, and seamless route traversing on the Internet and mobile devices.     

Omnistereo: panoramic stereo imaging

An omnistereo panorama consists of a pair of panoramic images, where one panorama is for the left eye and another panorama is for the right eye. The panoramic stereo pair provides a stereo sensation up to a full 360 degrees. Omnistereo panoramas can be constructed by mosaicing images from a single rotating camera. This approach also enables the control of stereo disparity, giving larger baselines for faraway scenes, and a smaller baseline for closer scenes. Capturing panoramic omnistereo images with a rotating camera makes it impossible to capture dynamic scenes at video rates and limits omnistereo imaging to stationary scenes. We present two possibilities for capturing omnistereo panoramas using optics without any moving parts. A special mirror is introduced such that viewing the scene through this mirror creates the same rays as those used with the rotating cameras. The lens used for omnistereo panorama is also introduced, together with the design of the mirror. Omnistereo panoramas can also be rendered by computer graphics methods to represent virtual environments     

单幅自然场景深度恢复

离焦测距算法是一种用于恢复场景深度信息的常用算法。传统的离焦测距算法通常需要采集多幅离焦图像,实际应用中具有很大的制约性。文中基于局部模糊估计提出单幅离焦图像深度恢复算法。基于局部模糊一致性的假设,本文采用简单而有效的两步法恢复输入图像的深度信息：1)通过求取输入离焦图和利用已知高斯核再次模糊图之间的梯度比得到边缘处稀疏模糊图 2)将边缘位置模糊值扩离至全部图像,完整的相对深度信息即可恢复。为了获得准确的场景深度信息,本文加入几何条件约束、天空区域提取策略来消除颜色、纹理以及焦点平面歧义性带来的影响,文中对各种类型的图片进行对比实验,结果表明该算法能在恢复深度信息的同时有效抑制图像中的歧义性。     

Influence of depth and 3D crosstalk on blur in multi‐view 3D displays

Current 3D crosstalk equation was defined from the characteristics of 3D display using glasses. This equation is not suitable for multi‐view 3D display with larger view number as it gives the inappropriately large value. In 3D display using eyeglass, double images occur at large depth. But, in multi‐view 3D display with larger view number, blur occurs to larger width for the larger depth. Hence, blur phenomenon of multi‐view 3D display was investigated to understand the unique characteristics of multi‐view 3D display. For this purpose, ray tracing S/W was used to simulate 3D display image seen at the designed viewing distance, to calculate the relative luminance distribution, and to quantify the relation between blur and depth. Calculated results showed that incomplete image separation caused the overlap of multiple view images and the blur. Blur edge width (BEW) was proportional to the horizontal disparity and related to the depth. BEWR = (BEW) / (binocular disparity) was newly defined, and its usefulness for 3D characterization was investigated. BEW and BEWR might be useful as new measuring items to characterize multi‐view 3D display regarding 3D crosstalk.     

高速铁路运行环境视频自适应去模糊方法

运动模糊是车载高速铁路运行环境视频中普遍存在的图像退化现象,对基于视频的高速铁路运行环境安全检查工作造成很大挑战。提出一种新的车载高速铁路运行环境视频采集系统,集成高清车载摄像机、信号同步单元、RFID标签阅读器、GNSS单元和惯性测量单元,能够采集高分辨率的运行环境视频数据。提出一种高速铁路运行环境视频自适应去模糊方法,利用惯性测量单元计算每帧图像曝光时间内摄像机的运动轨迹,并设计新的去卷积算法处理空间变化的模糊图像,从而得到复原图像。利用检测列车在实际高速铁路线路进行试验验证,结果表明：该方法能够自适应地去除车载高速铁路运行环境视频图像的模糊,实现了采集高质量运行环境视频数据的目的。     

3D车载环视全景生成方法

通过单应矩阵生成的鸟瞰视角全景图像存在严重的信息损失和边缘模糊形变。为三维空间点加入约束,将其限制在二维空间中,可以实现一幅图像从一个视角唯一变换另一视角。假设3D空间点来自于一个“包裹”车辆的曲面,使得原始摄像头图像上每一点对应的投影线都和曲面有交点,摄像头从不同角度观察曲面和车辆,便可生成多视角全景图像。针对相交区域也进行了特殊处理。最后的实验表明,生成的多视角的全景图像既可以充分利用原始图像信息从不同视角观测车辆周边场景,同时也减少了边缘模糊和扭曲,相交区域的过渡也显得更加自然、平滑。     

散焦模糊量估计的相机加权标定方法

相机标定在计算机视觉领域中有着至关重要的作用.绝大多数相机标定方法假设相机为针孔模型,且需要良好聚焦的图像来保证相机内外参估计的准确性.然而,这些条件会受到相机景深的影响.在薄透镜相机模型假设下,提出了一种加权相机标定的方法,其权重考虑了控制点的模糊量信息.首先对棋盘格标定物上的每一个角点进行散焦模糊量估计,在标定过程中,将散焦模糊量的大小作为一个权重加入到标定能量函数最小化过程中,使得标定精度得到提高.该方法简单高效,不需要额外的数码设备或者特别定做的标定物.在Intel Core i7处理器的计算机下,使用合成数据以及真实数据上进行的实验结果表明,文中方法能够有效减小重投影误差,提高张正友标定方法的标定精度.     

Image blurring effects due to depth discontinuities: blurring that creates emergent image details

A new model (called multi-component blurring— or MCB) to account for image blurring effects due to depth discontinuities is presented. We show that blurring processes operating in the vicinity of large depth discontinuties can give rise to emergent image details, quite distinguishable but nevertheless unexplained by previously available blurring models. In other words, the maximum principle for scale space¹ does not hold. It is argued that blurring in high-relief 3D scenes should be more accurately modelled as a multi-component process. We present results from extensive and carefully designed experiments, with many images of real scenes taken by a CCD camera with typical parameters. These results have consistently supported our new blurring model. Due care was taken to ensure that the image phenomena observed are mainly due to de-focussing and not due to mutual illuminations², specularity³, objects' ‘finer’ structures, coherent diffraction, or incidental image noises⁴. We also hypothesize on the role of blurring on human depth-from-blur perception, based on correlation with recent results from human blur perception⁵.     

Shape from Sharp and Motion-Blurred Image Pair

Motion blur due to camera shake is a common occurrence. During image capture, the apparent motion of a scene point in the image plane varies according to both camera motion and scene structure. Our objective is to infer the camera motion and the depth map of static scenes using motion blur as a cue. To this end, we use an unblurred–blurred image pair. Initially, we develop a technique to estimate the transformation spread function (TSF) which symbolizes the camera shake. This technique uses blur kernels estimated at different points across the image. Based on the estimated TSF, we recover the complete depth map of the scene within a regularization framework.     

Performance Analysis of Maximum Likelihood Estimator for Recovery of Depth from Defocused Images and Optimal Selection of Camera Parameters

The recovery of depth from defocused images involves calculating the depth of various points in a scene by modeling the effect that the focal parameters of the camera have on images acquired with a small depth of field. In the existing methods on depth from defocus (DFD), two defocused images of a scene are obtained by capturing the scene with different sets of camera parameters. Although the DFD technique is computationally simple, the accuracy is somewhat limited compared to the stereo algorithms. Further, an arbitrary selection of the camera settings can result in observed images whose relative blurring is insufficient to yield a good estimate of the depth. In this paper, we address the DFD problem as a maximum likelihood (ML) based blur identification problem. We carry out performance analysis of the ML estimator and study the effect of the degree of relative blurring on the accuracy of the estimate of the depth. We propose a criterion for optimal selection of camera parameters to obtain an improved estimate of the depth. The optimality criterion is based on the Cramer-Rao bound of the variance of the error in the estimate of blur. A number of simulations as well as experimental results on real images are presented to substantiate our claims.     

一种运动模糊图像的模糊参数估计方法

在摄取图像的过程中,物体间的高速运动及景物与成像设备的相对运动是引发图像退化的主要原因之一,称之为运动模糊。模糊长度和模糊方向是运动模糊中影响图像模糊程度的主要参数。提出了一种改进的误差参数分析方法,对模糊长度进行估计并比较了不同的复原方法对参数误差曲线法估计的效果,同时提出运用傅里叶分解和Hough变换从模糊图像的频谱特性上对运动模糊方向进行了估计。实验结果表明,所提出的方法对运动模糊图像的复原有良好的效果。     

Effects of frame rate on vection and postural sway

The quality of stereoscopic 3D cinematic content is a major determinant for user experience in immersive cinema in both traditional theatres and cinematic virtual reality. One of the most important parameters is the frame rate of the content which has historically been 24 frames per second for movies, but higher frame rates are being considered for cinema and are standard for virtual reality. A typical behavioural response to immersive stereoscopic 3D content is vection, the visually-induced perception of self-motion elicited by moving scenes. In this work we investigated how participants’ vection varied with simulated virtual camera speed, frame rate, and motion blur produced by the virtual camera’s exposure, while viewing depictions of movement through a realistic virtual environment. We also investigated how their postural sway varied with these parameters and how sway covaried with levels of perceived self-motion. Results show that while average perceived vection significantly increased with 3D content frame rate and motion speed, motion blur had no significant effect on perceived vection. We also found that levels of postural sway induced by vection correlated positively with subjective ratings.     

Improved estimation of defocus blur and spatial shifts in spatial domain: a homotopy-based approach

This paper presents a homotopy-based algorithm for the recovery of depth cues in the spatial domain. The algorithm specifically deals with defocus blur and spatial shifts, that is 2D motion, stereo disparities and/or zooming disparities. These cues are estimated from two images of the same scene acquired by a camera evolving in time and/or space. We show that they can be simultaneously computed by resolving a system of equations using a homotopy method. The proposed algorithm is tested using synthetic and real images. The results confirm that the use of a homotopy method leads to a dense and accurate estimation of depth cues. This approach has been integrated into an application for relief estimation from remotely sensed images.     

全向立体成像系统场景深度估计

基于单相机双抛物面反射镜设计了一种共轴结构的折反射全向立体成像装置,给出了针对相应展开柱面全景立体图像对深度估计的应点匹配方法,最后通过3D Max构造相应虚拟装置和虚拟场景进行了仿真实验,初步证明了该结构设计和对应深度估计方法的有效性。     

基于RGB-D相机的室内环境3D地图创建

RGB-D相机（如微软的Kinect）能够在获取彩色图像的同时得到每个像素的深度信息,在移动机器人三维地图创建方向具有广泛应用。本文设计了一种利用RGB-D相机进行机器人自定位及创建室内场景三维模型的方法,该方法首先由RGB-D相机获取周围环境的连续帧信息;其次提取并匹配连续帧间的SURF特征点,通过特征点的位置变化计算机器人的位姿并结合非线性最小二乘优化算法最小化对应点的双向投影误差;最后结合关键帧技术及观察中心法将相机观测到的三维点云依据当前位姿投影到全局地图。本文选择三个不同的场景试验了该方法,并对比了不同特征点下该方法的效果,试验中本文方法在轨迹长度为5.88m情况下误差仅为0.023,能够准确地创建周围环境的三维模型。     

Video super-resolution using controlled subpixel detector shifts

Video cameras must produce images at a reasonable frame-rate and with a reasonable depth of field. These requirements impose fundamental physical limits on the spatial resolution of the image detector. As a result, current cameras produce videos with a very low resolution. The resolution of videos can be computationally enhanced by moving the camera and applying super-resolution reconstruction algorithms. However, a moving camera introduces motion blur, which limits super-resolution quality. We analyze this effect and derive a theoretical result showing that motion blur has a substantial degrading effect on the performance of super-resolution. The conclusion is that, in order to achieve the highest resolution motion blur should be avoided. Motion blur can be minimized by sampling the space-time volume of the video in a specific manner. We have developed a novel camera, called the "jitter camera," that achieves this sampling. By applying an adaptive super-resolution algorithm to the video produced by the jitter camera, we show that resolution can be notably enhanced for stationary or slowly moving objects, while it is improved slightly or left unchanged for objects with fast and complex motions. The end result is a video that has a significantly higher resolution than the captured one.     

Real-time exploration and photorealistic reconstruction of large natural environments

This paper presents a hybrid (geometry- and image-based) framework suitable for providing photorealistic walkthroughs of large, complex outdoor scenes, based only on a small set of real images from the scene. To this end, a novel data representation of a 3D scene is proposed, which is called morphable 3D panoramas. Motion is assumed to be taking place along a predefined path of the 3D environment and the input to the system is a sparse set of stereoscopic views at certain positions (key positions) along that path (one view per position). An approximate local 3D model is constructed from each view, capable of capturing the photometric and geometric properties of the scene only locally. Then, during the rendering process, a continuous morphing (both photometric as well as geometric) takes place between successive local 3D models, using what we call a ‘morphable 3D model’. For the estimation of the photometric morphing, a robust algorithm capable of extracting a dense field of 2D correspondences between wide-baseline images is used, whereas, for the geometric morphing, a novel method of computing 3D correspondences between local models is proposed. In this way, a physically valid morphing is always produced, which is thus kept transparent from the user. Moreover, a highly optimized rendering path is used during morphing. Thanks to the use of appropriate pixel and vertex shaders, this rendering path can be run fully in 3D graphics hardware and thus allows for high frame rates. Our system can be extended to handle multiple stereoscopic views (and therefore multiple local models) per key position of the path (related by a camera rotation). In this case, one local 3D panorama (per key position) is constructed, comprising all local 3D models therein, and so a ‘morphable 3D panorama’ is now used during the rendering process. For handling the geometric consistency of each 3D panorama, a technique which is based on solving a partial differential equation is adopted. The effectiveness of our framework is demonstrated by using it for the 3D visual reconstruction of the Samaria Gorge in Crete.     

视频图像中的车型识别

文章介绍一种在固定单摄像头拍摄交通图像序列中检测车辆的方法。处理过程大致分为以下三步:重建不含运动目标的自然背景及图像分割;摄像机标定;目标区域的跟踪和车型识别。实验证明方法是可行的。