Robust multimodal audio–visual processing for advanced context awareness in smart spaces

Identifying people and tracking their locations is a key prerequisite to achieving context awareness in smart spaces. Moreover, in realistic context-aware applications, these tasks have to be carried out in a non-obtrusive fashion. In this paper we present a set of robust person-identification and tracking algorithms, based on audio and visual processing. A main characteristic of these algorithms is that they operate on far-field and un-constrained audio–visual streams, which ensure that they are non-intrusive. We also illustrate that the combination of their outputs can lead to composite multimodal tracking components, which are suitable for supporting a broad range of context-aware services. In combining audio–visual processing results, we exploit a context-modeling approach based on a graph of situations. Accordingly, we discuss the implementation of realistic prototype applications that make use of the full range of audio, visual and multimodal algorithms.     

3D Tracking and Classification System Using a Monocular Camera

This paper details a 3D tracking and recognition system using a single camera. The system is able to track and classify targets in outdoors and indoors scenarios, as long as they move (at least approximately) on a plane. The system first detects and validates targets and then tracks them in a state-space employing cylindrical models (horizontal and vertical position on the ground, their radius and height) utilising Particle Filters. The tracker fuses visual measurements that utilise the targets’ foreground and colour models. Finally the system classifies the tracked objects based on the visual metrics extracted by our algorithm. We have tested our model in an outdoor setting using humans and automobiles passing through the field of view of the camera at various speeds and distances. The results presented in this paper show the validity our approach.     

Numerical method for determining the quantization error PDF of single-bit /spl Sigma//spl Delta/ Modulators

Existing models for the quantizer of /spl Sigma//spl Delta/ modulators make assumptions on the probability density function (pdf) of the quantization error, or some other convenient signal of the modulator. In this paper, a method for the determination of this pdf for single-bit /spl Sigma//spl Delta/ modulators is presented. First, a numerical method is proposed in order to solve the simplified equation for the quantization error pdf for first-order systems considering noiseless and noisy dc input signals. Then, it is shown how most practical high-order (>2)/spl Sigma//spl Delta/ modulators, resulting from well-established design methods, can be modeled as first-order systems plus an additive noise source at the input. Hence, their quantization error pdf is analyzed using the proposed method. Simulation results are shown to be in considerable agreement with those of the proposed method.     

Improved Gaussian Mixture Models for Adaptive Foreground Segmentation

Adaptive foreground segmentation is traditionally performed using Stauffer and Grimson’s algorithm that models every pixel of the frame by a mixture of Gaussian distributions with continuously adapted parameters. In this paper we provide an enhancement of the algorithm by adding two important dynamic elements to the baseline algorithm: The learning rate can change across space and time, while the Gaussian distributions can be merged together if they become similar due to their adaptation process. We quantify the importance of our enhancements and the effect of parameter tuning using an annotated outdoors sequence.     

Audio–Visual Active Speaker Tracking in Cluttered Indoors Environments

We propose a system for detecting the active speaker in cluttered and reverberant environments where more than one person speaks and moves. Rather than using only audio information, the system utilizes audiovisual information from multiple acoustic and video sensors that feed separate audio and video tracking modules. The audio module operates using a particle filter (PF) and an information-theoretic framework to provide accurate acoustic source location under reverberant conditions. The video subsystem combines in 3-D a number of 2-D trackers based on a variation of Stauffer's adaptive background algorithm with spatiotemporal adaptation of the learning parameters and a Kalman tracker in a feedback configuration. Extensive experiments show that gains are to be expected when fusion of the separate modalities is performed to detect the active speaker.     

eWALL: An Open-Source Cloud-Based eHealth Platform for Creating Home Caring Environments for Older Adults Living with Chronic Diseases or Frailty

Independent living of older adults is one of the main challenges linked to the ageing population. Especially those living with diseases like COPD, MCI or frailty, need more support in everyday life and this is by itself a big societal challenge with impact in multiple sectors. In this paper we present eWALL, an innovative open-source eHealth platform that aims to address these challenges by means of an advanced cloud-based infrastructure. eWALL is designed in an innovative manner and achieved technical breakthroughs in eHealth platforms, while prioritizing user and market needs that are often abandoned and are the major reason for technically sound solutions that fail. We consider this as an opportunity and we aim to change the eHealth systems’ experience for older adults and break the barriers for the penetration of ICT solutions.     

Audio-visual active speaker tracking in cluttered indoors environments

We propose a system for detecting the active speaker in cluttered and reverberant environments where more than one person speaks and moves. Rather than using only audio information, the system utilizes audiovisual information from multiple acoustic and video sensors that feed separate audio and video tracking modules. The audio module operates using a particle filter (PF) and an information-theoretic framework to provide accurate acoustic source location under reverberant conditions. The video subsystem combines in 3-D a number of 2-D trackers based on a variation of Stauffer's adaptive background algorithm with spatiotemporal adaptation of the learning parameters and a Kalman tracker in a feedback configuration. Extensive experiments show that gains are to be expected when fusion of the separate modalities is performed to detect the active speaker.     

Subclass linear discriminant analysis for video-based face recognition

We present a novel subclass Linear Discriminant Analysis algorithm for feature extraction that copes with the severe pose, expression and illumination changes present in faces extracted from far-field video streams with subjects unconstrained in their motion and uncooperative to the system. Our novelty lies on the efficient automatic generation of subclasses from the gallery faces, by exploiting their different visual appearance and not constrained by their numbers per class. The proposed feature extraction algorithm is integrated in our complete face recognition system, with modules for preprocessing, classification, and decision fusion. We demonstrate the capability of the new algorithm to automatically generate discriminable subclasses and the resulting improved classification accuracy on a challenging video-based dataset, comprising low quality and resolution faces, as well as large variations in visual appearance. Our results indicate superior recognition rate compared to any systems in the CLEAR 2007 evaluation, running on that dataset.     

Faithful representation of stimuli with a population of integrate-and-fire neurons

We consider a formal model of stimulus encoding with a circuit consisting of a bank of filters and an ensemble of integrate-and-fire neurons. Such models arise in olfactory systems, vision, and hearing. We demonstrate that bandlimited stimuli can be faithfully represented with spike trains generated by the ensemble of neurons. We provide a stimulus reconstruction scheme based on the spike times of the ensemble of neurons and derive conditions for perfect recovery. The key result calls for the spike density of the neural population to be above the Nyquist rate. We also show that recovery is perfect if the number of neurons in the population is larger than a threshold value. Increasing the number of neurons to achieve a faithful representation of the sensory world is consistent with basic neurobiological thought. Finally we demonstrate that in general, the problem of faithful recovery of stimuli from the spike train of single neurons is ill posed. The stimulus can be recovered, however, from the information contained in the spike train of a population of neurons.