Data-parallel intra decoding for block-based image and video coding on massively parallel architectures

With the increasing number of processor cores available in modern computing architectures, task or data parallelism is required to maximally exploit the available hardware and achieve optimal processing speed. Current state-of-the-art data-parallel processing methods for decoding image and video bitstreams are limited in parallelism by dependencies introduced by the coding tools and the number of synchronization points introduced by these dependencies, only allowing task or coarse-grain data parallelism. In particular, entropy decoding and data prediction are bottleneck coding tools for parallel image and video decoding. We propose a new data-parallel processing scheme for block-based intra sample and coefficient prediction that allows fine-grain parallelism and is suitable for integration in current and future state-of-the-art image and video codecs. Our prediction scheme enables maximum concurrency, independent of slice or tile configuration, while minimizing synchronization points. This paper describes our data-parallel processing scheme for one- and two-dimensional prediction and investigates its application to block-based image and video codecs using JPEG XR and H.264/AVC Intra as a starting point. We show how our scheme enables faster decoding than the state-of-the-art wavefront method with speedup factors of up to 21.5 and 7.9 for JPEG XR and H.264/AVC Intra coding tools respectively. Using the H.264/AVC Intra coding tool, we discuss the requirements of the algorithm and the impact on decoded image quality when these requirements are not met. Finally, we discuss the impact on coding rate in order to allow for optimal parallel intra decoding.     

Silhouette-based multi-sensor smoke detection

Fire is one of the leading hazards affecting everyday life around the world. The sooner the fire is detected, the better the chances are for survival. Today’s fire alarm systems, such as video-based smoke detectors, however, still pose many problems. In order to accomplish more accurate video-based smoke detection and to reduce false alarms, this paper proposes a multi-sensor smoke detector which takes advantage of the different kinds of information represented by visual and thermal imaging sensors. The detector analyzes the silhouette coverage of moving objects in visual and long-wave infrared registered (~aligned) images. The registration is performed using a contour mapping algorithm which detects the rotation, scale and translation between moving objects in the multi-spectral images. The geometric parameters found at this stage are then further used to coarsely map the silhouette images and coverage between them is calculated. Since smoke is invisible in long-wave infrared its silhouette will, contrarily to ordinary moving objects, only be detected in visual images. As such, the coverage of thermal and visual silhouettes will start to decrease in case of smoke. Due to the dynamic character of the smoke, the visual silhouette will also show a high degree of disorder. By focusing on both silhouette behaviors, the system is able to accurately detect the smoke. Experiments on smoke and non-smoke multi-sensor sequences indicate that the automated smoke detection algorithm is able to coarsely map the multi-sensor images. Furthermore, using the low-cost silhouette analysis, a fast warning, with a low number of false alarms, can be given.     

Multi-modal time-of-flight based fire detection

This paper proposes two novel time-of-flight based fire detection methods for indoor and outdoor fire detection. The indoor detector is based on the depth and amplitude image of a time-of-flight camera. Using this multi-modal information, flames can be detected very accurately by fast changing depth and amplitude disorder detection. In order to detect the fast changing depth, depth differences between consecutive frames are accumulated over time. Regions which have multiple pixels with a high accumulated depth difference are labeled as candidate flame regions. Simultaneously, the amplitude disorder is also investigated. Regions with high accumulative amplitude differences and high values in all detail images of the amplitude image its discrete wavelet transform, are also labeled as candidate flame regions. Finally, if one of the depth and amplitude candidate flame regions overlap, fire alarm is given. The outdoor detector, on the other hand, only differs from the indoor detector in one of its multi-modal inputs. As depth maps are unreliable in outdoor environments, the outdoor detector uses a visual flame detector instead of the fast changing depth detection. Experiments show that the proposed detectors have an average flame detection rate of 94% with no false positive detections.     

FireCube: A multi-view localization framework for 3D fire analysis

Effective response to fire requires accurate and timely information of its evolution. In order to accomplish this valuable fire analysis step, this work fuses low-cost video fire detection results of multiple cameras using a novel multi-view localization framework. As such, valuable fire characteristics are detected at the early stage of the fire. The framework merges the single-view detection results of the multiple cameras by homographic projection onto multiple horizontal and vertical planes, which slice the scene. The crossings of these slices create a 3D grid of virtual sensor points, called the FireCube. Using this grid and subsequent spatial and temporal 3D clean-up filters, information about the location of the fire, its size and its direction of propagation can be instantly extracted from the video data. The novel aspect in the proposed framework is the 3D grid creation, which is a 3D extension of multiple plane homography. Also the use of spatial and temporal 3D filters, which extend existing 2D filter concepts, provides a more reliable fire analysis. Experimental results indicate that the proposed multi-view fire localization framework is able to accurately detect and localize the fire. Two cameras are already sufficient to achieve a dimension accuracy of 90% and a position accuracy of 98%. By further increasing the number of cameras it is even possible to achieve a dimension accuracy of 96% and a position accuracy of 99%. Furthermore, the experiments show that increasing the number of cameras to monitor the scene has a positive effect on the detection rate. The gain of using four cameras instead of one is 3%.     

A new approach to combine texture compression and filtering

Texture mapping has been widely used to improve the quality of 3D rendered images. To reduce the storage and bandwidth impact of texture mapping, compression systems are commonly used. To further increase the quality of the rendered images, texture filtering is also often adopted. These two techniques are generally considered to be independent. First, a decompression step is executed to gather texture samples, which is then followed by a separate filtering step. We have investigated a system based on linear transforms that merges both phases together. This allows more efficient decompression and filtering at higher compression ratios. This paper formally presents our approach for any linear transformation, how the commonly used discrete cosine transform can be adapted to this new approach, and how this method can be implemented in real time on current-generation graphics cards using shaders. Through reuse of the existing hardware filtering, fast magnification and minification filtering is achieved. Our implementation provides fully anisotropically filtered samples four to six times faster than an implementation using two separate phases for decompression and filtering. Additionally, our transform-based compression also provides increased and variable compression ratios over standard hardware compression systems at a comparable or better quality level.