Real-time multiple vehicle detection and tracking from a moving vehicle

Abstract. A real-time vision system has been developed that analyzes color videos taken from a forward-looking video camera in a car driving on a highway. The system uses a combination of color, edge, and motion information to recognize and track the road boundaries, lane markings and other vehicles on the road. Cars are recognized by matching templates that are cropped from the input data online and by detecting highway scene features and evaluating how they relate to each other. Cars are also detected by temporal differencing and by tracking motion parameters that are typical for cars. The system recognizes and tracks road boundaries and lane markings using a recursive least-squares filter. Experimental results demonstrate robust, real-time car detection and tracking over thousands of image frames. The data includes video taken under difficult visibility conditions. Received: 1 September 1998 / Accepted: 22 February 2000     

Aware Community Portals: Shared Information Appliances for Transitional Spaces

People wish to maintain a level of awareness of timely information, including presence of others in the workplace and other social settings. We believe this provides better exchange, coordination and contact within a community, especially as people work in asynchronous times and distributed locations. The challenge is to develop lightweight techniques for awareness, interaction and communication using shared information appliances. In this paper, we describe the design of an exploratory responsive display projected within a shared workspace at the MIT Media Lab. The system uses visual sensing to provide relevant information and constructs traces of people’s activity over time. Such aware portals may be deployed in casual workplace domains, distributed workgroups, and everyday public spaces.     

A memory-based approach to sensory-motor coordination

In this work a visual-based autonomous system capable of memorizing and recalling sensory-motor associations is presented. The robot's behaviors are based on learned associations between its sensory inputs and its motor actions. Perception is divided into two stages. The first one is functional: algorithmic procedures extract in real time visual features such as disparity and local orientation from the input images. The second stage is mnemonic: the features produced by the different functional areas are integrated with motor information and memorized or recalled. An efficient memory organization and fast information retrieval enables the robot to learn to navigate and to avoid obstacles without need of an internal metric reconstruction of the external environment. Received: 22 November 1996 / Accepted: 18 November 1997     

Display Design of Process Systems Based on Functional Modelling

The prevalent way to present information in industrial computer displays is by using piping and instrumentation diagrams. Such interfaces have sometimes resulted in difficulties for operators because they are not sufficient to fulfil their needs. A systematic way that supports interface design therefore has to be considered. In the new design framework, two questions must be answered. Firstly, a modelling method is required to describe a process system. Such a modelling method can define the information content that must be displayed in interfaces. Secondly, how to communicate this information to operators efficiently must be considered. This will provide a basis for determining the visual forms that the information should take. This study discusses interface design of human–machine systems from these two points of view. Based on other scholars’ work, a comprehensive set of functional primitives is summarised as a basis to build a functional model of process systems. A library of geometrical presentations for these primitives is then developed. To support effective interface design, the concept of ‘functional macro’ is introduced and a way to map functional model to interface display is illustrated by applying several principles. To make our ideas clear, a central heating system is taken as an example and its functional model is constructed. Based on the functional model, the information to be displayed is determined. Several functional macros are then found in the model and their corresponding displays are constructed. Finally, by using the library of geometrical presentations for functional primitives and functional macros, the display hierarchy of the central heating system is developed. Reusability of functional primitives makes it possible to use the methodology to support interface design of different process systems.     

Frame rate as a QoS parameter and its influence on speech perception

The preservation of QoS for multimedia traffic through a data network is a difficult problem. We focus our attention on video frame rate and study its influence on speech perception. When sound and picture are discrepant (e.g., acoustic ‘ba’ combined with visual ‘ga’), subjects perceive a different sound (such as ‘da’). This phenomenon is known as the McGurk effect. In this paper, the influence of degraded video frame rate on speech perception is studied. It is shown that, when frame rate decreases, correct hearing is improved for discrepant stimuli and is degraded for congruent (voice and picture are the same) stimuli. Furthermore, we studied the case where lip closure was always captured by the synchronization of sampling time and lip position. In this case, frame rate has little effect on mishearing for congruent stimuli. For discrepant stimuli, mishearing is decreased with degraded frame rate. These results indicate that the stiff motion of lips resulting from low frame rate cannot give enough labial information for speech perception. In addition, the effect of delaying the picture to correct for low frame rate was studied. The results, however, were not as definitive as expected, because of compound effects related to the synchronization of sound and picture. Finally, we inspected the still pictures of normal Japanese speech and determined a lower limit of frame rate from the view point of assisting hearing.     

结合背景图的高分辨率视频人像实时抠图网络

目的 近年来,采用神经网络完成人像实时抠图已成为计算机视觉领域的研究热点,现有相关网络在处理高分辨率视频时还无法满足实时性要求,为此本文提出一种结合背景图的高分辨率视频人像实时抠图网络。方法 给出一种由基准网络和精细化网络构成的双层网络,在基准网络中,视频帧通过编码器模块提取图像的多尺度特征,采用金字塔池化模块融合这些特征作为循环解码器网络的输入;在循环解码器中,通过残差门控循环单元聚合连续视频帧间的时间信息,以此生成蒙版图、前景残差图和隐藏特征图,采用残差结构降低模型参数量并提高网络的实时性。为提高高分辨率图像实时抠图性能,在精细化网络中,设计高分辨率信息指导模块,通过高分辨率图像信息指导低分辨率图像的方式生成高质量人像抠图结果。结果 与近年来的相关网络模型进行实验对比,实验结果表明,本文方法在高分辨率数据集Human2K上优于现有相关方法,在评价指标(绝对误差、均方误差、梯度、连通性)上分别提升了18.8%、39.2%、40.7%、20.9%。在NVIDIA GTX 1080Ti GPU上处理4 K分辨率影像运行速率可达26帧/s,处理HD(high definition)分辨率影像运行速率可达43帧/s。结论 本文模型能够更好地完成高分辨率人像实时抠图任务,可以为影视、短视频社交以及网络会议等高级应用提供更好的支持。     

Secure buffering in firm real-time database systems

Many real-time database applications arise in electronic financial services, safety-critical installations and military systems where enforcing security is crucial to the success of the enterprise. We investigate here the performance implications, in terms of killed transactions, of guaranteeing multi-level secrecy in a real-time database system supporting applications with firm deadlines. In particular, we focus on the buffer management aspects of this issue. Our main contributions are the following. First, we identify the importance and difficulties of providing secure buffer management in the real-time database environment. Second, we present SABRE, a novel buffer management algorithm that provides covert-channel-free security. SABRE employs a fully dynamic one-copy allocation policy for efficient usage of buffer resources. It also incorporates several optimizations for reducing the overall number of killed transactions and for decreasing the unfairness in the distribution of killed transactions across security levels. Third, using a detailed simulation model, the real-time performance of SABRE is evaluated against unsecure conventional and real-time buffer management policies for a variety of security-classified transaction workloads and system configurations. Our experiments show that SABRE provides security with only a modest drop in real-time performance. Finally, we evaluate SABRE's performance when augmented with the GUARD adaptive admission control policy. Our experiments show that this combination provides close to ideal fairness for real-time applications that can tolerate covert-channel bandwidths of up to one bit per second (a limit specified in military standards). Received March 1, 1999 / Accepted October 1, 1999     

A reconfigurable architecture for autonomous visual-navigation

Abstract. This paper describes the design of a reconfigurable architecture for implementing image processing algorithms. This architecture is a pipeline of small identical processing elements that contain a programmable logic device (FPGA) and double port memories. This processing system has been adapted to accelerate the computation of differential algorithms. The log-polar vision selectively reduces the amount of data to be processed and simplifies several vision algorithms, making possible their implementation using few hardware resources. The reconfigurable architecture design has been devoted to implementation, and has been employed in an autonomous platform, which has power consumption, size and weight restrictions. Two different vision algorithms have been implemented in the reconfigurable pipeline, for which some experimental results are shown. Received: 30 March 2001 / Accepted: 11 February 2002 RID="*" ID="*" This work has been supported by the Ministerio de Ciencia y Tecnología and FEDER under project TIC2001-3546 Correspondence to: J.A. Boluda     

Using Personal Traces in Context Space: Towards Context Trace Technology

Wearables are often described with a focus on providing the user with wearable information access and communication means. The contextual information retrieval aspect is, however, an essential feature of such systems, as in, for example, the Remembrance Agent [1] where manually entered search-terms are used for presenting relevant situational information, or as in different location-based systems [2]. In this position paper we outline a general framework of contextually aware wearable systems, and suggest how such mechanisms, collecting massive traces of the user context, may lead to several other interesting uses in what we will call context trace technology.     

A system to navigate a robot into a ship structure

Abstract. A prototype system has been built to navigate a walking robot into a ship structure. The 8-legged robot is equipped with an active stereo head. From the CAD-model of the ship good view points are selected, such that the head can look at locations with sufficient edge features, which are extracted automatically for each view. The pose of the robot is estimated from the features detected by two vision approaches. One approach searches in stereo images for junctions and measures the 3-D position. The other method uses monocular image and tracks 2-D edge features. Robust tracking is achieved with a method of edge projected integration of cues (EPIC). Two inclinometres are used to stabilise the head while the robot moves. The results of the final demonstration to navigate the robot within centimetre accuracy are given.     

Image feedback path tracking control using an uncalibrated CCD camera

Abstract. Image feedback path tracking (IFPT) control of a laser light point (LLP) using a CCD camera is studied in this paper. The tracking path and the LLP are assumed clearly focused in the scene, but no camera calibration is needed. A modified version of the thinning algorithm SPTA is proposed to skeletonize the path in a piecewise manner. The proposed thinning algorithm takes less computer time than the original SPTA and makes the real-time skeletonization possible. Included in the paper is also the development of a control algorithm with image feedback to assure LLP tracking along the required path, as well as an experimental study to demonstrate how IFPT control can be realized in practice. Received: 10 November 1999 / Accepted: 9 March 2000     

Fast prototyping of parallel-vision applications using functional skeletons

We present a design methodology for real-time vision applications aiming at significantly reducing the design-implement-validate cycle time on dedicated parallel platforms. This methodology is based upon the concept of algorithmic skeletons, i.e., higher order program constructs encapsulating recurring forms of parallel computations and hiding their low-level implementation details. Parallel programs are built by simply selecting and composing instances of skeletons chosen in a predefined basis. A complete parallel programming environment was built to support the presented methodology. It comprises a library of vision-specific skeletons and a chain of tools capable of turning an architecture-independent skeletal specification of an application into an optimized, deadlock-free distributive executive for a wide range of parallel platforms. This skeleton basis was defined after a careful analysis of a large corpus of existing parallel vision applications. The source program is a purely functional specification of the algorithm in which the structure of a parallel application is expressed only as combination of a limited number of skeletons. This specification is compiled down to a parametric process graph, which is subsequently mapped onto the actual physical topology using a third-party CAD software. It can also be executed on any sequential platform to check the correctness of the parallel algorithm. The applicability of the proposed methodology and associated tools has been demonstrated by parallelizing several realistic real-time vision applications both on a multi-processor platform and a network of workstations. It is here illustrated with a complete road-tracking algorithm based upon white-line detection. This experiment showed a dramatic reduction in development times (hence the term fast prototyping), while keeping performances on par with those obtained with the handcrafted parallel version. Received: 22 July 1999 / Accepted: 9 November 2000     

MOBSY: Integration of vision and dialogue in service robots

Abstract. This contribution introduces MOBSY, a fully integrated, autonomous mobile service robot system. It acts as an automatic dialogue-based receptionist for visitors to our institute. MOBSY incorporates many techniques from different research areas into one working stand-alone system. The techniques involved range from computer vision over speech understanding to classical robotics. Along with the two main aspects of vision and speech, we also focus on the integration aspect, both on the methodological and on the technical level. We describe the task and the techniques involved. Finally, we discuss the experiences that we gained with MOBSY during a live performance at our institute.     

Real-time CAM-based Hough transform algorithm and its performance evaluation

Abstract. This paper proposes a highly parallel Hough transform algorithm for real-time straight-line extraction and its hardware implementation on a content-addressable memory (CAM). To achieve high-speed processing, incrementation for voting, which composes the Hough transform, and calculations for coordinate updating are carried out for the every scan line, not every edge pixel, and extracting maxima in Hough space is executed by parallel comparing. Moreover, variously weighted voting achieves more accurate line extraction in spite of the quantization error and noise in the image space. In the implementation, the CAM acts as a PE (processing-element) array that effectively performs highly parallel processing for the Hough transform and also as a memory for two-dimensional Hough space, and both voting and peak extraction are directly executed by the CAM. Evaluations of CAM hardware size, processing time and the accuracy of line extraction show that a real-time and high-resolution Hough transform for a 256256 picture can be achieved using a single CAM chip with current VLSI technology. This CAM-based Hough transform algorithm promises to be an important step towards the realization of a real-time and compact image-understanding system. Received: 15 August 1998 / Accepted: 15 March 2000     

Motion detection with nonstationary background

Abstract. This paper proposes a new background subtraction method for detecting moving foreground objects from a nonstationary background. While background subtraction has traditionally worked well for a stationary background, the same cannot be implied for a nonstationary viewing sensor. To a limited extent, motion compensation for the nonstationary background can be applied. However, in practice, it is difficult to realize the motion compensation to sufficient pixel accuracy, and the traditional background subtraction algorithm will fail for a moving scene. The problem is further complicated when the moving target to be detected/tracked is small, since the pixel error in motion that is compensating the background will subsume the small target. A spatial distribution of Gaussians (SDG) model is proposed to deal with moving object detection having motion compensation that is only approximately extracted. The distribution of each background pixel is temporally and spatially modeled. Based on this statistical model, a pixel in the current frame is then classified as belonging to the foreground or background. For this system to perform under lighting and environmental changes over an extended period of time, the background distribution must be updated with each incoming frame. A new background restoration and adaptation algorithm is developed for the nonstationary background. Test cases involving the detection of small moving objects within a highly textured background and with a pan-tilt tracking system are demonstrated successfully. Received: 30 July 2001 / Accepted: 20 April 2002 Correspondence to: Chin-Seng Chau     

UnQL: a query language and algebra for semistructured data based on structural recursion

Abstract. This paper presents structural recursion as the basis of the syntax and semantics of query languages for semistructured data and XML. We describe a simple and powerful query language based on pattern matching and show that it can be expressed using structural recursion, which is introduced as a top-down, recursive function, similar to the way XSL is defined on XML trees. On cyclic data, structural recursion can be defined in two equivalent ways: as a recursive function which evaluates the data top-down and remembers all its calls to avoid infinite loops, or as a bulk evaluation which processes the entire data in parallel using only traditional relational algebra operators. The latter makes it possible for optimization techniques in relational queries to be applied to structural recursion. We show that the composition of two structural recursion queries can be expressed as a single such query, and this is used as the basis of an optimization method for mediator systems. Several other formal properties are established: structural recursion can be expressed in first-order logic extended with transitive closure; its data complexity is PTIME; and over relational data it is a conservative extension of the relational calculus. The underlying data model is based on value equality, formally defined with bisimulation. Structural recursion is shown to be invariant with respect to value equality. Received: July 9, 1999 / Accepted: December 24, 1999     

Towards robust multi-cue integration for visual tracking

Abstract. Even though many of today's vision algorithms are very successful, they lack robustness, since they are typically tailored to a particular situation. In this paper, we argue that the principles of sensor and model integration can increase the robustness of today's computer-vision systems substantially. As an example, multi-cue tracking of faces is discussed. The approach is based on the principles of self-organization of the integration mechanism and self-adaptation of the cue models during tracking. Experiments show that the robustness of simple models is leveraged significantly by sensor and model integration.     

The Design of Human–Machine Interface for Accident Support in Nuclear Power Plants

As feedback from Three Mile Island No. 2, a large amount of human–machine interface (HMI) design has been proposed to support operators during accidents by presenting information on plant status, some of which is implemented in commercial nuclear power plants (NPPs). However, it has not yet been discussed what role HMI should play under conditions where operators must take action as instructed under emergency operating procedure. Regarding this principal issue, an HMI design is proposed together with specific screen images. The advantage of this design is based on the involvement of an ex-operator in NPP in this paper. The participation of users in the design of HMI has been rare. Accordingly the proposed HMI is comprehensive for average operators and is expected to be acceptable for future implementation in commercial NPPs. The effectiveness of the proposed HMI has been examined in a small experiment.     

Wood inspection with non-supervised clustering

Abstract. The appearance of sawn timber has huge natural variations that the human inspector easily compensates for mentally when determining the types of defects and the grade of each board. However, for automatic wood inspection systems these variations are a major source for complication. This makes it difficult to use textbook methodologies for visual inspection. These methodologies generally aim at systems that are trained in a supervised manner with samples of defects and good material, but selecting and labeling the samples is an error-prone process that limits the accuracy that can be achieved. We present a non-supervised clustering-based approach for detecting and recognizing defects in lumber boards. A key idea is to employ a self-organizing map (SOM) for discriminating between sound wood and defects. Human involvement needed for training is minimal. The approach has been tested with color images of lumber boards, and the achieved false detection and error escape rates are low. The approach also provides a self-intuitive visual user interface. Received: 16 December 2000 / Accepted: 8 December 2001 Correspondence to: O. Silvén