A compact algorithm for rectification of stereo pairs

Abstract. We present a linear rectification algorithm for general, unconstrained stereo rigs. The algorithm takes the two perspective projection matrices of the original cameras, and computes a pair of rectifying projection matrices. It is compact (22-line MATLAB code) and easily reproducible. We report tests proving the correct behavior of our method, as well as the negligible decrease of the accuracy of 3D reconstruction performed from the rectified images directly. Received: 25 February 1999 / Accepted: 2 March 2000     

A hybrid approach for stereo disparity computation

An alternative, hybrid approach for disparity estimation, based on the phase difference technique, is presented. The proposed technique combines the robustness of the matching method with the sub-pixel accuracy of the phase difference approach. A matching between the phases of the left and right signals is introduced in order to allow the phase difference method to work in a reduced disparity range. In this framework, a new criterion to detect signal singularities is proposed. The presented test cases show that the performance of the proposed technique in terms of accuracy and density of the disparity estimates has greatly improved. Received: 24 June 1997 / Accepted: 15 September 1998     

Model-based interpretation of stereo imagery of textured surfaces

We present a scheme for reliable and accurate surface reconstruction from stereoscopic images containing only fine texture and no stable high-level features. Partial shape information is used to improve surface computation: first by fitting an approximate, global, parametric model, and then by refining this model via local correspondence processes. This scheme eliminates the window size selection problem in existing area-based stereo correspondence schemes. These ideas are integrated in a practical vision system that is being used by environmental scientists to study wind erosion of bulk material such as coal ore being transported in open rail cars. Received: 14 August 1995 / Accepted: 27 May 1997     

Multipass hierarchical stereo matching for generation of digital terrain models from aerial images

This paper presents a new multi-pass hierarchical stereo-matching approach for generation of digital terrain models (DTMs) from two overlapping aerial images. Our method consists of multiple passes which compute stereo matches with a coarse-to-fine and sparse-to-dense paradigm. An image pyramid is generated and used in the hierarchical stereo matching. Within each pass, the DTM is refined by using the image pyramid from the coarse to the fine level. At the coarsest level of the first pass, a global stereo-matching technique, the intra-/inter-scanline matching method, is used to generate a good initial DTM for the subsequent stereo matching. Thereafter, hierarchical block matching is applied to image locations where features are detected to refine the DTM incrementally. In the first pass, only the feature points near salient edge segments are considered in block matching. In the second pass, all the feature points are considered, and the DTM obtained from the first pass is used as the initial condition for local searching. For the passes after the second pass, 3D interactive manual editing can be incorporated into the automatic DTM refinement process whenever necessary. Experimental results have shown that our method can successfully provide accurate DTM from aerial images. The success of our approach and system has also been demonstrated with a flight simulation software. Received: 4 November 1996 / Accepted: 20 October 1997     

An automatic assessment scheme for steel quality inspection

This paper presents an automatic system for steel quality assessment, by measuring textural properties of carbide distributions. In current steel inspection, specially etched and polished steel specimen surfaces are classified manually under a light microscope, by comparisons with a standard chart. This procedure is basically two-dimensional, reflecting the size of the carbide agglomerations and their directional distribution. To capture these textural properties in terms of image fea tures, we first apply a rich set of image-processing operations, including mathematical morphology, multi-channel Gabor filtering, and the computation of texture measures with automatic scale selection in linear scale-space. Then, a feature selector is applied to a 40-dimensional feature space, and a classification scheme is defined, which on a sample set of more than 400 images has classification performance values comparable to those of human metallographers. Finally, a fully automatic inspection system is designed, which actively selects the most salient carbide structure on the specimen surface for subsequent classification. The feasibility of the overall approach for future use in the production process is demonstrated by a prototype system. It is also shown how the presented classification scheme allows for the definition of a new reference chart in terms of quantitative measures. Received: 16 February 1999 / Accepted: 29 March 2000     

Model-based detection,segmentation, and classification for image analysis using on-line shape learning

Detection, segmentation, and classification of specific objects are the key building blocks of a computer vision system for image analysis. This paper presents a unified model-based approach to these three tasks. It is based on using unsupervised learning to find a set of templates specific to the objects being outlined by the user. The templates are formed by averaging the shapes that belong to a particular cluster, and are used to guide a probabilistic search through the space of possible objects. The main difference from previously reported methods is the use of on-line learning, ideal for highly repetitive tasks. This results in faster and more accurate object detection, as system performance improves with continued use. Further, the information gained through clustering and user feedback is used to classify the objects for problems in which shape is relevant to the classification. The effectiveness of the resulting system is demonstrated in two applications: a medical diagnosis task using cytological images, and a vehicle recognition task. Received: 5 November 2000 / Accepted: 29 June 2001 Correspondence to: K.-M. Lee     

A golden-template self-generating method for patterned wafer inspection

This paper presents a novel golden-template self-generating technique for detecting possible defects in periodic two-dimensional wafer images. A golden template of the patterned wafer image under inspection can be obtained from the wafer image itself and no other prior knowledge is needed. It is a bridge between the existing self-reference methods and image-to-image reference methods. Spectral estimation is used in the first step to derive the periods of repeating patterns in both di r ections. Then a building block representing the structure of the patterns is extracted using interpolation to obtain sub-pixel resolution. After that, a new defect-free golden template is built based on the extracted building block. Finally, a pixel-to-pixel comparison is all we need to find possible defects. A comparison between the results of the proposed method and those of the previously published methods is presented. Received: 19 May 1999 / Accepted: 20 May 2000     

New Active Tools for Supporting Narrative Structures

Constructing stories is a type of playing that involves mobilizing the storyteller’s imagination and finding original ways to convey narrative intentions. When a child invents a story, there is a natural interaction with the local environment and the use of various means of expression. We adopted a user-centered approach to design POGO, a playful environment which utilizes the child’s physical environment and sensory modalities. Pogo is a system of active tools that enable children to create stories by connecting physical and virtual environments. By providing children with the possibility of capturing and manipulating images and various media, and combining them in sequential form, Pogo triggered new strategies in the construction of narrative logic, time and space, in the construction of the episodes and in the visual narration. Correspondence to: Fran?oise Decortis, Psychology and Education Sciences Department, University of Liege, 4000 Liege, Belgium. Email:francoise.decortis@ulg.ac.be     

Active rules for XML: A new paradigm for E-services

XML is rapidly becoming one of the most widely adopted technologies for information exchange and representation. As the use of XML becomes more widespread, we foresee the development of active XML rules, i.e., rules explicitly designed for the management of XML information. In particular, we argue that active rules for XML offer a natural paradigm for the rapid development of innovative e-services. In the paper, we show how active rules can be specified in the context of XSLT, a pattern-based language for publishing XML documents (promoted by the W3C) which is receiving strong commercial support, and Lorel, a query language for XML documents that is quite popular in the research world. We demonstrate, through simple examples of active rules for XSLT and Lorel, that active rules can be effective for the implementation of e-commerce services. We also discuss the various issues that need to be considered in adapting the notion of relational triggers to the XML context. Received: 30 October 2000 / Accepted: 19 December 2000 Published online: 27 April 2001     

Head tracking using stereo

Head tracking is an important primitive for smart environments and perceptual user interfaces where the poses and movements of body parts need to be determined. Most previous solutions to this problem are based on intensity images and, as a result, suffer from a host of problems including sensitivity to background clutter and lighting variations. Our approach avoids these pitfalls by using stereo depth data together with a simple human-torso model to create a head-tracking system that is both fast and robust. We use stereo data (Commercial equipment and materials are identified in order to adequately specify certain procedures. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the materials or equipment identified are necessarily the best available for the purpose.) to derive a depth model of the background that is then employed to provide accurate foreground segmentation. We then use directed local edge detectors on the foreground to find occluding edges that are used as features to fit to a torso model. Once we have the model parameters, the location and orientation of the head can be easily estimated. A useful side effect from using stereo data is the ability to track head movement through a room in three dimensions. Experimental results on real image sequences are given. Accepted: 13 August 2001     

The computation of cloud base height from paired whole-sky imaging cameras

We have developed a novel approach to the extraction of cloud base height (CBH) from pairs of whole-sky imagers (WSIs). The core problem is to spatially register cloud fields from widely separated WSIs; this complete, triangulation provides the CBH measurements. The wide camera separation and the self-similarity of clouds defeats standard matching algorithms when applied to static views of the sky. In response, we use optical flow methods that exploit the fact that modern WSIs provide image sequences. We will describe the algorithm, a confidence metric for its performance, a method to correct the severe projective effects of the WSI camera, and results on real data.     

Machine vision system for curved surface inspection

This application-oriented paper discusses a non-contact 3D range data measurement system to improve the performance of the existing 2D herring roe grading system. The existing system uses a single CCD camera with unstructured halogen lighting to acquire and analyze the shape of the 2D shape of the herring roe for size and deformity grading. Our system will act as an additional system module, which can be integrated into the existing 2D grading system, providing the additional third dimension to detect deformities in the herring roe, which were not detected in the 2D analysis. Furthermore, the additional surface depth data will increase the accuracy of the weight information used in the existing grading system. In the proposed system, multiple laser light stripes are projected into the herring roe and the single B/W CCD camera records the image of the scene. The distortion in the projected line pattern is due to the surface curvature and orientation. Utilizing the linear relation between the projected line distortion and surface depth, the range data was recovered from a single camera image. The measurement technique is described and the depth information is obtained through four steps: (1) image capture, (2) stripe extraction, (3) stripe coding, (4) triangulation, and system calibration. Then, this depth information can be converted into the curvature and orientation of the shape for deformity inspection, and also used for the weight estimation. Preliminary results are included to show the feasibility and performance of our measurement technique. The accuracy and reliability of the computerized herring roe grading system can be greatly improved by integrating this system into existing system in the future.     

Wood inspection with non-supervised clustering

Abstract. The appearance of sawn timber has huge natural variations that the human inspector easily compensates for mentally when determining the types of defects and the grade of each board. However, for automatic wood inspection systems these variations are a major source for complication. This makes it difficult to use textbook methodologies for visual inspection. These methodologies generally aim at systems that are trained in a supervised manner with samples of defects and good material, but selecting and labeling the samples is an error-prone process that limits the accuracy that can be achieved. We present a non-supervised clustering-based approach for detecting and recognizing defects in lumber boards. A key idea is to employ a self-organizing map (SOM) for discriminating between sound wood and defects. Human involvement needed for training is minimal. The approach has been tested with color images of lumber boards, and the achieved false detection and error escape rates are low. The approach also provides a self-intuitive visual user interface. Received: 16 December 2000 / Accepted: 8 December 2001 Correspondence to: O. Silvén     

Efficient support for interactive operations in multi-resolution video servers

In this paper, we present a placement algorithm that interleaves multi-resolution video streams on a disk array and enables a video server to efficiently support playback of these streams at different resolution levels. We then combine this placement algorithm with a scalable compression technique to efficiently support interactive scan operations (i.e., fast-forward and rewind). We present an analytical model for evaluating the impact of the scan operations on the performance of disk-arr ay-based servers. Our experiments demonstrate that: (1) employing our placement algorithm substantially reduces seek and rotational latency overhead during playback, and (2) exploiting the characteristics of video streams and human perceptual tolerances enables a server to support interactive scan operations without any additional overhead.     

The reconstruction of dynamic 3D structure of biological objects using stereo microscope images

In this paper, we address the analysis of 3D shape and shape change in non-rigid biological objects imaged via a stereo light microscope. We propose an integrated approach for the reconstruction of 3D structure and the motion analysis for images in which only a few informative features are available. The key components of this framework are: 1) image registration using a correlation-based approach, 2) region-of-interest extraction using motion-based segmentation, and 3) stereo and motion analysis using a cooperative spatial and temporal matching process. We describe these three stages of processing and illustrate the efficacy of the proposed approach using real images of a live frog's ventricle. The reconstructed dynamic 3D structure of the ventricle is demonstrated in our experimental results, and it agrees qualitatively with the observed images of the ventricle.     

Search strategies for shape regularized active contour

Nonlinear shape models have been shown to improve the robustness and flexibility of contour-based object segmentation when there are appearance ambiguities between the object and the background. In this paper, we focus on a new search strategy for the shape regularized active contour (ShRAC) model, which adopts existing nonlinear shape models to segment objects that are similar to a set of training shapes. The search for optimal contour is performed by a coarse-to-fine algorithm that iterates between combinatorial search and gradient-based local optimization. First, multi-solution dynamic programming (MSDP) is used to generate initial candidates by minimizing only the image energy. In the second step, a combination of image energy and shape energy is minimized starting from these initial candidates using a local optimization method and the best one is selected. To generate diverse initial candidates while reducing invalid shapes, we apply two pruning methods to the search space of MSDP. Our search strategy combines the advantages of global combinatorial search and local optimization, and has shown excellent robustness to local minima caused by distracting suboptimal solutions. Experimental results on segmentation of different anatomical structures using ShRAC, as well as preliminary results on human silhouette segmentation are provided.     

Mobile robot navigation and scene modeling using stereo fish-eye lens system

We present an autonomous mobile robot navigation system using stereo fish-eye lenses for navigation in an indoor structured environment and for generating a model of the imaged scene. The system estimates the three-dimensional (3D) position of significant features in the scene, and by estimating its relative position to the features, navigates through narrow passages and makes turns at corridor ends. Fish-eye lenses are used to provide a large field of view, which images objects close to the robot and helps in making smooth transitions in the direction of motion. Calibration is performed for the lens-camera setup and the distortion is corrected to obtain accurate quantitative measurements. A vision-based algorithm that uses the vanishing points of extracted segments from a scene in a few 3D orientations provides an accurate estimate of the robot orientation. This is used, in addition to 3D recovery via stereo correspondence, to maintain the robot motion in a purely translational path, as well as to remove the effects of any drifts from this path from each acquired image. Horizontal segments are used as a qualitative estimate of change in the motion direction and correspondence of vertical segment provides precise 3D information about objects close to the robot. Assuming detected linear edges in the scene as boundaries of planar surfaces, the 3D model of the scene is generated. The robot system is implemented and tested in a structured environment at our research center. Results from the robot navigation in real environments are presented and discussed. Received: 25 September 1996 / Accepted: 20 October 1996     

A new phase extraction algorithm for phase profilometry

This paper describes a new phase extraction algorithm for 3D optical profilometry based on the projection of a periodic light pattern and phase measurement (phase profilometry). The algorithm uses a square wave to demodulate phase and moving averages and comb-shaped filters to extract the phase information from low frequency. The proposed algorithm is compared with the two major profilometry techniques, namely Fourier domain profilometry and signal domain profilometry based on FIR low-pass filtering. Comparison is focused on adaptiveness to changes of the pattern frequency, ability to deal with nonuniform surfaces and computational complexity. Adaptiveness analysis is carried out by means of simulations. The issue of nonuniform surfaces is discussed on the basis of experimental results obtained from application of phase profilometry to on-line 3D printed circuit board inspection. With regard to complexity, theoretical estimates are verified by means of actual computation time measurements. Received: 30 November 1996 / Accepted: 6 June 1997