Rectified Surface Mosaics

We approach mosaicing as a camera tracking problem within a known parameterized surface. From a video of a camera moving within a surface, we compute a mosaic representing the texture of that surface, flattened onto a planar image. Our approach works by defining a warp between images as a function of surface geometry and camera pose. Globally optimizing this warp to maximize alignment across all frames determines the camera trajectory, and the corresponding flattened mosaic image. In contrast to previous mosaicing methods which assume planar or distant scenes, or controlled camera motion, our approach enables mosaicing in cases where the camera moves unpredictably through proximal surfaces, such as in medical endoscopy applications.     

Single-view-point omnidirectional catadioptric cone mirror imager

We present here a comprehensive imaging theory about cone mirrors in a single-view-point (SVP) configuration and show that an SVP cone mirror catadioptric system is not only practical but also has unique advantages for certain applications. We show its merits and weaknesses and how to build a workable system.     

Central catadioptric image processing with geodesic metric

Because of the distortions produced by the insertion of a mirror, catadioptric images cannot be processed similarly to classical perspective images. Now, although the equivalence between such images and spherical images is well known, the use of spherical harmonic analysis often leads to image processing methods which are more difficult to implement. In this paper, we propose to define catadioptric image processing from the geodesic metric on the unitary sphere. We show that this definition allows to adapt very simply classical image processing methods. We focus more particularly on image gradient estimation, interest point detection, and matching. More generally, the proposed approach extends traditional image processing techniques based on Euclidean metric to central catadioptric images. We show in this paper the efficiency of the approach through different experimental results and quantitative evaluations.     

Markov random fields for catadioptric image processing

Images obtained with catadioptric sensors contain significant deformations which prevent the direct use of classical image treatments. Thus, Markov random fields (MRF) whose usefulness is now obvious for projective image processing, cannot be used directly on catadioptric images because of the inadequacy of the neighborhood. In this paper, we propose to define a new neighborhood for MRF by using the equivalence theorem developed for central catadioptric sensors. We show the importance of this adaptation for segmentation, image restoration and motion detection.     

Self-calibration of hybrid central catadioptric and perspective cameras

Hybrid central catadioptric and perspective cameras are desired in practice, because the hybrid camera system can capture large field of view as well as high-resolution images. However, the calibration of the system is challenging due to heavy distortions in catadioptric cameras. In addition, previous calibration methods are only suitable for the camera system consisting of perspective cameras and catadioptric cameras with only parabolic mirrors, in which priors about the intrinsic parameters of perspective cameras are required. In this work, we provide a new approach to handle the problems. We show that if the hybrid camera system consists of at least two central catadioptric and one perspective cameras, both the intrinsic and extrinsic parameters of the system can be calibrated linearly without priors about intrinsic parameters of the perspective cameras, and the supported central catadioptric cameras of our method can be more generic. In this work, an approximated polynomial model is derived and used for rectification of catadioptric image. Firstly, with the epipolar geometry between the perspective and rectified catadioptric images, the distortion parameters of the polynomial model can be estimated linearly. Then a new method is proposed to estimate the intrinsic parameters of a central catadioptric camera with the parameters in the polynomial model, and hence the catadioptric cameras can be calibrated. Finally, a linear self-calibration method for the hybrid system is given with the calibrated catadioptric cameras. The main advantage of our method is that it cannot only calibrate both the intrinsic and extrinsic parameters of the hybrid camera system, but also simplify a traditional nonlinear self-calibration of perspective cameras to a linear process. Experiments show that our proposed method is robust and reliable.     

Omni-directional stereo

Omnidirectional views of an indoor environment at different locations are integrated into a global map. A single camera swiveling about the vertical axis takes consecutive images and arranges them into a panoramic representation, which provides rich information around the observation point: a precise omnidirectional view of the environment and coarse ranges to objects in it. Using the coarse map, the system autonomously plans consecutive observations at the intersections of lines connecting object points, where the directions of the imaging are estimated easily and precisely. From two panoramic views at the two planned locations, a modified binocular stereo method yields a more precise, but with direction-dependent uncertainties, local map. New observation points are selected to decrease the uncertainty, and another local map is yielded, which is then integrated into a more reliable global representation of the world with the adjacent local maps     

Assessing stereo blindness and stereo acuity on digital displays

Previous studies on stereoscopic acuity have shown that the percentage of stereo blind subjects is relevant. Moreover, stereoscopic visualization is becoming widely diffused in different fields, like, e.g., entertainment, surgery or VR training, where it is necessary an accurate assessment of stereoscopic abilities of the involved subjects. Therefore, there might be the need of performing a stereo blindness and stereo acuity test before each visualization session involving stereoscopic images. In this paper, we propose a method to assess stereo acuity and stereo blindness directly on the chosen device, under the same visualization condition and setup adopted for the tasks to perform, in order to have the same perceptual response. We present software-based tests suitable for a generic stereoscopic displays, and we compare their effectiveness performing a comparison with a standard physical, card-based, test commonly used in assessment of stereo acuity and stereo blindness. We provide to the reader all the details to perform autonomously the tests, of which images will be downloadable from web.     

A new linear algorithm for calibrating central catadioptric cameras

In this paper, a novel linear calibration algorithm based on lines is presented for central catadioptric cameras. We firstly derive the relationship between the projection on the viewing sphere of a space point and its catadioptric image. And then by the relationship we establish a group of linear constraints on the catadioptric parameters from the catadioptric projections of spatial lines. By using these linear constraints, any central catadioptric camera can be fully calibrated from a single view of three or more lines without prior knowledge on the camera. Extensive experiments show this algorithm can improve the calibration's robustness.     

Rectified nearest feature line segment for pattern classification

This paper points out and analyzes the advantages and drawbacks of the nearest feature line (NFL) classifier. To overcome the shortcomings, a new feature subspace with two simple and effective improvements is built to represent each class. The proposed method, termed rectified nearest feature line segment (RNFLS), is shown to possess a novel property of concentration as a result of the added line segments (features), which significantly enhances the classification ability. Another remarkable merit is that RNFLS is applicable to complex tasks such as the two-spiral distribution, which the original NFL cannot deal with properly. Finally, experimental comparisons with NFL, NN(nearest neighbor), k-NN and NNL (nearest neighbor line) using both artificial and real-world data-sets demonstrate that RNFLS offers the best performance.     

A multiple-baseline stereo

A stereo matching method that uses multiple stereo pairs with various baselines generated by a lateral displacement of a camera to obtain precise distance estimates without suffering from ambiguity is presented. Matching is performed simply by computing the sum of squared-difference (SSD) values. The SSD functions for individual stereo pairs are represented with respect to the inverse distance and are then added to produce the sum of SSDs. This resulting function is called the SSSD-in-inverse-distance. It is shown that the SSSD-in-inverse-distance function exhibits a unique and clear minimum at the correct matching position, even when the underlying intensity patterns of the scene include ambiguities or repetitive patterns. The authors first define a stereo algorithm based on the SSSD-in-inverse-distance and present a mathematical analysis to show how the algorithm can remove ambiguity and increase precision. Experimental results with real stereo images are presented to demonstrate the effectiveness of the algorithm     

Estimating parameters of noncentral catadioptric systems using bundle adjustment

This paper describes a new method to calibrate the intrinsic and extrinsic parameters of a generalized catadioptric camera (central or noncentral). The algorithm has two steps. The first one is the estimation of correspondences between incident lines in space and pixels (black box model calibration) in an arbitrary world reference frame. The second step is the calibration of the intrinsic parameters of the pinhole camera, the coefficients of the mirror expressed by a quadric (quadric mirror shape and the pose of the camera in relation to it), the position of the optical center of the camera in the world reference frame and its relative orientation (pose of the camera in world reference frame). A projection model relaxing Snell’s Law is derived. The deviations from Snell’s Law and the image reprojection errors are minimized by means of bundle adjustment. Information about the apparent contour of the mirror can be used to reduce the uncertainty in the estimation by introducing a new term in the cost function of the second step minimization process. Simulations and real experiments show good accuracy and robustness for this framework. However, the convergence is dependent on the initial guess as expected. A well-behaved algorithm to automatically generate the initial estimate to be used in the bundle adjustment is also presented.     

Multiview photometric stereo

This paper addresses the problem of obtaining complete, detailed reconstructions of textureless shiny objects. We present an algorithm which uses silhouettes of the object, as well as images obtained under changing illumination conditions. In contrast with previous photometric stereo techniques, ours is not limited to a single viewpoint but produces accurate reconstructions in full 3D. A number of images of the object are obtained from multiple viewpoints, under varying lighting conditions. Starting from the silhouettes, the algorithm recovers camera motion and constructs the object's visual hull. This is then used to recover the illumination and initialise a multi-view photometric stereo scheme to obtain a closed surface reconstruction. There are two main contributions in this paper: Firstly we describe a robust technique to estimate light directions and intensities and secondly, we introduce a novel formulation of photometric stereo which combines multiple viewpoints and hence allows closed surface reconstructions. The algorithm has been implemented as a practical model acquisition system. Here, a quantitative evaluation of the algorithm on synthetic data is presented together with complete reconstructions of challenging real objects. Finally, we show experimentally how even in the case of highly textured objects, this technique can greatly improve on correspondence-based multi-view stereo results.     

Accelerated catadioptric omnidirectional view image unwrapping processing using GPU parallelisation

Catadioptric omnidirectional view sensors have found increasing adoption in various robotic and surveillance applications due to their 360° field of view. However, the inherent distortion caused by the sensors prevents their direct utilisations using existing image processing techniques developed for perspective images. Therefore, a correction processing known as “unwrapping” is commonly performed. However, the unwrapping process incurs additional computational loads on central processing units. In this paper, a method to reduce this burden in the computation is investigated by exploiting the parallelism of graphical processing units (GPUs) based on the Compute Unified Device Architecture (CUDA). More specifically, we first introduce a general approach of parallelisation to the said process. Then, a series of adaptations to the CUDA platform is proposed to enable an optimised usage of the hardware platform. Finally, the performances of the unwrapping function were evaluated on a high-end and low-end GPU to demonstrate the effectiveness of the parallelisation approach.     

Occlusions and binocular stereo

Binocular stereo is the process of obtaining depth information from a pair of cameras. In the past, stereo algorithms have had problems at occlusions and have tended to fail there (though sometimes post-processing has been added to mitigate the worst effects). We show that, on the contrary, occlusions can help stereo computation by providing cues for depth discontinuities.We describe a theory for stereo based on the Bayesian approach, using adaptive windows and a prior weak smoothness constraint, which incorporates occlusion. Our model assumes that a disparity discontinuity, along the epipolar line, in one eyealways corresponds to an occluded region in the other eye thus, leading to anocclusion constraint. This constraint restricts the space of possible disparity values, thereby simplifying the computations. An estimation of the disparity at occluded features is also discussed in light of psychophysical experiments. Using dynamic programming we can find the optimal solution to our system and the experimental results are good and support the assumptions made by the model.     

Dynamic stereo with self-calibration

An approach for incremental refinement of disparity maps obtained from a dynamic stereo sequence of a static scene is presented. The approach has been implemented using a binocular stereo vision system mounted on a mobile robot. A robust least median of squares based algorithm is given for recovering the camera motion between successive viewpoints, which provides a self-calibration mechanism. The recovered motion is utilized for recursive disparity prediction and refinement using a robust Kalman filter model     

A topological stereo matcher

Presented here is a new stereo algorithm that produces dense, high-quality, subpixel disparity maps. It offers two improvements over previous algorithms. First, it does not blur disparity values across sharp changes in depth. Second, it can reconstruct the correct correspondence between two images even when there is substantial vertical displacement between them: this algorithm has been tested with rotations up to 10 degrees and vertical translations up to 16 pixels. Although such image pairs require extra processing time, this ability is vital when exact calibration cannot be maintained.The new algorithm depends on two new ideas. First, it exploits the fact that the correct vertical disparity field is due to camera misalignment and, thus, has only a few (significant) degrees of freedom. The algorithm passes camera alignment parameters, not raw disparity fields, between scales. Disparities at individual locations can diverge only slightly from this global model, greatly reducing the algorithm's search space.Second, the new algorithm uses a pre-match filter that prevents two patches of image from matching if they do not have the same (local) topological structure. This constraint subsumes previous figural continuity proposals and can be checked by simple, local operations. The filter seems to improve the algorithm's ability to select the correct match from many alternatives and it suppresses intermediate values near sharp changes in disparity. This technique can be extended to other matching tasks, such as motion tracking, analyzing texture periodicity, and evaluating the performance of edge finders.