基于自适应白化的音乐节拍实时跟踪算法*

提出一种基于自适应白化的音乐节拍实时跟踪算法。在对音乐信号进行触发点检测之前,构建音乐信号短时傅里叶变换的当前频域峰值表,对各频段的频谱幅值进行自适应加权,使各个频段保持相似的动态范围,改善音乐信号起伏变化较大时触发点检测准确性低的问题,进而提高节拍实时跟踪效果。在MIREX2006标准测试库上的节拍跟踪实验表明,自适应白化的引入可整体提高P-score,特别是对于起伏变化很大的音乐信号,节拍跟踪效果的提高非常明显。     

3DFCNN: real-time action recognition using 3D deep neural networks with raw depth information

Multimedia Tools and Applications - This work describes an end-to-end approach for real-time human action recognition from raw depth image-sequences. The proposal is based on a 3D fully...     

On-line modeling for real-time 3D target tracking

Model-based object tracking can provide autonomous mobile robotic systems with real-time 6-dof pose information, for example, enabling them to rendezvous with targets from a particular desired direction. Most existing model-based trackers, however, require the geometric model of the target to be known a priori, which may pose a practical problem in real-world environments. This paper presents a novel 3D modeler capable of building an approximate model of a target object on-line. The proposed technique rapidly constructs a 3D tessellated enveloping mesh and uses projective texture mapping to further model the target object’s surface features. Separation of the target object from background clutter is achieved via customizable interest filters. The resulting real-time object-tracking system was tested extensively via simulations and experiments.     

基于边的自适应实时三维跟踪

针对缺乏纹理特征的物体,提出了一种基于边的自适应实时三维跟踪方法。在已知物体三维模型的情况下,通过基于历史运动信息的物体边缘检测与跟踪,可以有效准确地求解出摄像机的外参。基于并扩展了现有的基于边的实时跟踪算法,其主要工作体现在以下三个方面： 1）提出自适应阈值和基于历史信息估计当前帧的运动趋势的方法,从而提高边匹配算法在快速运动时的稳定性;2）提出一种基于随机抽样一致性（RANSAC）的边匹配策略,可以有效剔除误匹配的边,从而提高复杂模型的跟踪稳定性;3）利用抽取轮廓边的算法将边跟踪算法从CAD模型扩展到一般的面片模型。实验结果证明了该方法的鲁棒高效,能够满足增强现实、虚拟装配等应用需求。     

基于RTX的某光电装备图像处理实时性研究

针对现有光电跟踪设备一体化水平低、实时性的不足的现状,提出基于RTX技术的光电跟踪设备图像处理系统的并行处理方法。对原有图像处理系统的任务进行重新划分,利用共享内存来实现图像处理模块与图像存储模块间的数据交换。最后搭建系统的试验平台,对改善前后图像处理的实时性进行了比较,给出系统仿真结果。实验结果表明：利用RTX实时扩展技术大幅度提高了系统图像处理的实时性和有效性,RTX在图像处理速度上比Windows系统快300%左右,图像处理时间的上下浮动由几十个微秒下降到几个微秒。基本能够满足该光电跟踪设备对实时处理快、精度高、稳定性好等要求。     

Viewpoint image generation for head tracking 3D display using multi‐camera and approximate depth information

A simple and high image quality method for viewpoint image synthesis from multi‐camera images for a stereoscopic 3D display using head tracking is proposed. In this method, slices of images for depth layers are made using approximate depth information, the slices are linearly blended corresponding to the distance between the viewpoint and cameras at each layer, and the layers are overlaid from the perspective of viewpoint. Because the linear blending automatically compensates for depth error because of the visual effects of depth‐fused 3D (DFD), the resulting image is natural to observer's perception. Smooth motion parallax of wide depth range objects induced by viewpoint movement for left‐and‐right and front‐and‐back directions is achieved using multi‐camera images and approximate depth information. Because the calculation algorithm is very simple, it is suitable for real time 3D display applications.     

基于深度信息的指尖追踪及手势识别

提出了一个基于深度信息对手指和手部进行实时跟踪,并可用于手势识别的方案。用Kinect获取深度信息,然后生成手部的三维点云,进行过滤转换成像素矩阵;使用K-curvature算法获取指尖和手掌方位,然后通过手指之间的相关距离进行手指标定。实验结果证明该方案识别追踪效果稳定且高效,不受光照和复杂背景影响,能够同时跟踪双手共10个手指和2个掌心的动作轨迹,并用于手势识别。     

Adaptivity in 3D image processing

We present an adaptive numerical scheme for computing the nonlinear partial differential equations arising in 3D image multiscale analysis. The scheme is based on a semi-implicit scale discretization and on an adaptive finite element method in 3D-space. Successive coarsening of the computational grid is used for increasing the efficiency of the numerical procedure. L_∞-stability of the semi–discrete scheme is proved and computational results related to 3D nonlinear image filtering are discussed. Received: 15 December 1999 / Accepted: 8 June 2001     

Embedded real-time speed limit sign recognition using image processing and machine learning techniques

The number of traffic accidents in Brazil has reached alarming levels and is currently one of the leading causes of death in the country. With the number of vehicles on the roads increasing rapidly, these problems will tend to worsen. Consequently, huge investments in resources to increase road safety will be required. The vertical R-19 system for optical character recognition of regulatory traffic signs (maximum speed limits) according to Brazilian Standards developed in this work uses a camera positioned at the front of the vehicle, facing forward. This is so that images of traffic signs can be captured, enabling the use of image processing and analysis techniques for sign detection. This paper proposes the detection and recognition of speed limit signs based on a cascade of boosted classifiers working with haar-like features. The recognition of the sign detected is achieved based on the optimum-path forest classifier (OPF), support vector machines (SVM), multilayer perceptron, k-nearest neighbor (kNN), extreme learning machine, least mean squares, and least squares machine learning techniques. The SVM, OPF and kNN classifiers had average accuracies higher than 99.5 %; the OPF classifier with a linear kernel took an average time of 87 \(\upmu\)s to recognize a sign, while kNN took 11,721 \(\upmu\)s and SVM 12,595 \(\upmu\)s. This sign detection approach found and recognized successfully 11,320 road signs from a set of 12,520 images, leading to an overall accuracy of 90.41 %. Analyzing the system globally recognition accuracy was 89.19 %, as 11,167 road signs from a database with 12,520 signs were correctly recognized. The processing speed of the embedded system varied between 20 and 30 frames per second. Therefore, based on these results, the proposed system can be considered a promising tool with high commercial potential.

     

改进Contourlet变换的3D掌纹图像识别

提出一种基于改进Contourlet变换的3D掌纹图像识别方法;该方法通过形状指数将3D掌纹图像映射成灰度图像,以克服常用的均值或高斯曲率映射难于精确描述3D掌纹特征的缺点;基于此,将7/5滤波器引入Contourlet变换,并在变换域提取形状指数映射图各方向子带的均值与方差作为掌纹图像的特征信息,从而有效利用了Contourlet变换优越的方向特征表达能力,又可有效消除传统Contourlet变换各子图像存在的相关性;最后采用欧氏距离最近邻分类法,实现了测试图像的分类识别。实验结果表明,针对香港理工大学所提供的三维掌纹数据库,该方法总体识别率较PCA方法提高了2.9%,具有明显的优势。     

Stable real-time 3D tracking using online and offline information

We propose an efficient real-time solution for tracking rigid objects in 3D using a single camera that can handle large camera displacements, drastic aspect changes, and partial occlusions. While commercial products are already available for offline camera registration, robust online tracking remains an open issue because many real-time algorithms described in the literature still lack robustness and are prone to drift and jitter. To address these problems, we have formulated the tracking problem in terms of local bundle adjustment and have developed a method for establishing image correspondences that can equally well handle short and wide-baseline matching. We then can merge the information from preceding frames with that provided by a very limited number of keyframes created during a training stage, which results in a real-time tracker that does not jitter or drift and can deal with significant aspect changes.     

Sound image externalization for headphone based real-time 3D audio

3D audio effects can provide immersive auditory experience, but we often face the so-called in-head localization (IHL) problem in headphone sound reproduction. To address this problem, we propose an effective sound image externalization approach. Specifically, we consider several important factors related to sound propagation, which include image-source model based early reflections with distance decay, wall absorption and air absorption, late reverberation and other dynamic factors like head movement. We apply our sound image externalization approach to a headphone based real-time 3D audio system. Subjective listening tests show that the sound image externalization performance is significantly improved and the sound source direction is preserved as well. A/B preference test further shows that, as compared with a recent popular approach, the proposed approach is mostly preferred by the listeners.     

FPGA-based real-time 3D image preprocessing for image-guided medical interventions

Minimally invasive image-guided interventions (IGIs) are time and cost efficient, minimize unintended damage to healthy tissue, and lead to faster patient recovery. One emerging trend in IGI workflow is to use volumetric imaging modalities such as low-dose computed tomography (CT) and 3D ultrasound to provide real-time, accurate anatomical information intraoperatively. These intraoperative images, however, are often characterized by quantum (in low-dose CT) or speckle (in ultrasound) noise and must be enhanced prior to any advanced image processing. Anisotropic diffusion filtering and median filtering have been shown to be effective in enhancing and improving the visual quality of these images. However, achieving real-time performance, as required by IGIs, using software-only implementations is challenging because of the sheer size of the images and the arithmetic complexity of the filtering operations. We present a field-programmable gate array-based reconfigurable architecture for real-time preprocessing of intraoperative 3D images. The proposed architecture provides programmable kernels for 3D anisotropic diffusion filtering and 3D median filtering within the same framework. The implementation of this architecture using an Altera Stratix-II device achieved a voxel processing rate close to 200 MHz, which enables the use of these processing techniques in the IGI workflow prior to advanced operations such as segmentation, registration, and visualization.

A real-time digitizer for stereo image processing

This paper discusses the design and implementation of a stereo image digitizer for use in digital image processing applications. The system operates in real time with respect to the standard television refresh rate and it possesses several signal sampling, routing, and transformation options which can be controlled from an external digital computer. Results obtained with a prototype unit are illustrated and compared using precision test patterns and natural scenes.     

Extraction of 3D freeform surfaces as visual landmarks for real-time tracking

This work presents a system for the generation of a free-form surface model from video sequences. Although any single centered camera can be applied in the proposed system the approach is demonstrated using fish-eye lenses because of their good properties for tracking. The system is designed to function automatically and to be flexible with respect to size and shape of the reconstructed scene. To minimize geometric assumptions a statistic fusion of dense depth maps is utilized. Special attention is payed to the necessary rectification of the spherical images and the resulting iso-disparity surfaces, which can be exploited in the fusion approach. Before dense depth estimation can be performed the cameras’ pose parameters are extracted by means of a structure-from-motion (SfM) scheme. In this respect automation of the system is achieved by thorough decision model based on robust statistics and error propagation of projective measurement uncertainties. This leads to a scene-independent set of only a few parameters. All system components are formulated in a general way, making it possible to cope with any single centered projection model, in particular with spherical cameras. In using wide field-of-view cameras the presented system is able to reliably retrieve poses and consistently reconstruct large scenes. A textured triangle mesh constructed on basis of the scene’s reconstructed depth, makes the system’s results suitable to function as reference models in a GPU driven analysis-by-synthesis framework for real-time tracking.     

Reducing the gap between Augmented Reality and 3D modeling with real-time depth imaging

Whereas 3D surface models are often used for augmented reality (e.g., for occlusion handling or model-based camera tracking), the creation and the use of such dense 3D models in augmented reality applications usually are two separated processes. The 3D surface models are often created in offline preparation steps, which makes it difficult to detect changes and to adapt the 3D model to these changes. This work presents a 3D change detection and model adjustment framework that combines AR techniques with real-time depth imaging to close the loop between dense 3D modeling and augmented reality. The proposed method detects the differences between a scene and a 3D model of the scene in real time. Then, the detected geometric differences are used to update the 3D model, thus bringing AR and 3D modeling closer together. The accuracy of the geometric difference detection depends on the depth measurement accuracy as well as on the accuracy of the intrinsic and extrinsic parameters. To evaluate the influence of these parameters, several experiments were conducted with simulated ground truth data. Furthermore, the evaluation shows the applicability of AR and depth image–based 3D modeling for model-based camera tracking.