Integrated spatial and feature image query

We present a new system for querying for images by regions and their spatial and feature attributes. The system enables the user to find the images that contain arrangements of regions similar to those diagrammed in a query image. By indexing the attributes of regions, such as sizes, locations and visual features, a wide variety of complex joint spatial and feature queries are efficiently computed. In order to demonstrate the utility of the system, we develop a process for the extracting color regions from photographic images. We demonstrate that integrated spatial and feature querying using color regions improves image search capabilities over non-spatial content-based image retrieval methods.     

基于卷积神经网络的图像检索算法研究

由于互联网+时代的到来,在线图像的数量急剧增加,基于内容的图像检索引起了很多关注。传统的检索方法由于图像表达能力不强,使得检索效率低下,不利于大规模图像检索。因此,提出一种新的基于卷积神经网络的图像检索算法。设计一种新型的端到端的卷积神经网络结构,同时学习基于概率的语义信息相似性和图像特征相似性;引入主成分分析方法,对深层特征进行降维的同时降低信息的损失;通过距离函数计算目标图像与数据库图像的距离,实现检索。在Image Net-1000和Oxford 5K数据集上的实验结果表明,该方法能够有效地增强图像特征的表达能力,提高检索性能,优于对比方法。     

NeTra: A toolbox for navigating large image databases

We present here an implementation of NeTra, a prototype image retrieval system that uses color, texture, shape and spatial location information in segmented image regions to search and retrieve similar regions from the database. A distinguishing aspect of this system is its incorporation of a robust automated image segmentation algorithm that allows object- or region-based search. Image segmentation significantly improves the quality of image retrieval when images contain multiple complex objects. Images are segmented into homogeneous regions at the time of ingest into the database, and image attributes that represent each of these regions are computed. In addition to image segmentation, other important components of the system include an efficient color representation, and indexing of color, texture, and shape features for fast search and retrieval. This representation allows the user to compose interesting queries such as “retrieve all images that contain regions that have the color of object A, texture of object B, shape of object C, and lie in the upper of the image”, where the individual objects could be regions belonging to different images. A Java-based web implementation of NeTra is available at http://vivaldi.ece.ucsb.edu/Netra.     

Browsing and placement of multi-resolution images on parallel disks

Abstract. With rapid advances in computer and communication technologies, there is an increasing demand to build and maintain large image repositories. To reduce the demands on I/O and network resources, multi-resolution representations are being proposed for the storage organization of images. Image decomposition techniques such as wavelets can be used to provide these multi-resolution images. The original image is represented by several coefficients, one of them with visual similarity to the original image, but at a lower resolution. These visually similar coefficients can be thought of as thumbnails or icons of the original image. This paper addresses the problem of storing these multi-resolution coefficients on disks so that thumbnail browsing as well as image reconstruction can be performed efficiently. Several strategies are evaluated to store the image coefficients on parallel disks. These strategies can be classified into two broad classes, depending on whether the access pattern of the images is used in the placement. Disk simulation is used to evaluate the performance of these strategies. Simulation results are validated with results from experiments with real Disks, and are found to be in good qualitative agreement. The results indicate that significant performance improvements can be achieved with as few as four disks by placing image coefficients based upon browsing access patterns. Work supported by a research grant from NSF/ARPA/NASA IRI9411330 and NSF instrumentation grant CDA-9421978 and NSF Career grant No. IIS-9985019, and NSF grant 0010044-CCR.     

Dynamic vp-tree indexing for n-nearest neighbor search given pair-wise distances

Abstract. For some multimedia applications, it has been found that domain objects cannot be represented as feature vectors in a multidimensional space. Instead, pair-wise distances between data objects are the only input. To support content-based retrieval, one approach maps each object to a k-dimensional (k-d) point and tries to preserve the distances among the points. Then, existing spatial access index methods such as the R-trees and KD-trees can support fast searching on the resulting k-d points. However, information loss is inevitable with such an approach since the distances between data objects can only be preserved to a certain extent. Here we investigate the use of a distance-based indexing method. In particular, we apply the vantage point tree (vp-tree) method. There are two important problems for the vp-tree method that warrant further investigation, the n-nearest neighbors search and the updating mechanisms. We study an n-nearest neighbors search algorithm for the vp-tree, which is shown by experiments to scale up well with the size of the dataset and the desired number of nearest neighbors, n. Experiments also show that the searching in the vp-tree is more efficient than that for the -tree and the M-tree. Next, we propose solutions for the update problem for the vp-tree, and show by experiments that the algorithms are efficient and effective. Finally, we investigate the problem of selecting vantage-point, propose a few alternative methods, and study their impact on the number of distance computation. Received June 9, 1998 / Accepted January 31, 2000     

基于 PCA 和 LDA 的方言辨识

针对PCA没有有效利用样本的类别信息而导致方言识别率低的问题,采用PCA和LDA组合方法进行特征提取。首先用PCA对普通话、上海话、广东话和闽南话四种方言进行降维,然后在降维后的空间中用LDA进一步特征提取,最后将该特征向量送入BP神经网络进行辨识。仿真实验结果表明,基于PCA和LDA的方言识别的平均识别率高达85%。     

利用PCA进行深度学习图像特征提取后的降维研究

深度学习是当前人工智能领域广泛使用的一种机器学习方法.深度学习对数据的高度依赖性使得数据需要处理的维度剧增,极大地影响了计算效率和数据分类性能.本文以数据降维为研究目标,对深度学习中的各种数据降维方法进行分析.在此基础上,以Caltech 101图像数据集为实验对象,采用VGG-16深度卷积神经网络进行图像的特征提取,以PCA主成分分析方法为例来实现高维图像特征数据的降维处理.在实验阶段,采用欧氏距离作为相似性度量来检验经过降维处理后的精度指标.实验证明：当提取VGG-16神经网络fc3层的4096维特征后,使用PCA法将数据维度降至64维,依然能够保持较高的特征信息.     

唇读中基于像素的特征提取方法的研究

针对单独视觉通道唇读中的基于像素的特征提取问题,提出一个级联的特征提取策略。首先对图像采用相应的变换,然后对变换结果降维,最后进行特征归一化。基于对几种变换方法的比较与分析,提出利用PCA对DCT和Gabor小波变换结果降维的DCT-PCA和Gabor-PCA方法,与传统人工选择变换系数的方法相比识别率提高了约10%。     

A model-based hand gesture recognition system

This paper introduces a model-based hand gesture recognition system, which consists of three phases: feature extraction, training, and recognition. In the feature extraction phase, a hybrid technique combines the spatial (edge) and the temporal (motion) information of each frame to extract the feature images. Then, in the training phase, we use the principal component analysis (PCA) to characterize spatial shape variations and the hidden Markov models (HMM) to describe the temporal shape variations. A modified Hausdorff distance measurement is also applied to measure the similarity between the feature images and the pre-stored PCA models. The similarity measures are referred to as the possible observations for each frame. Finally, in recognition phase, with the pre-trained PCA models and HMM, we can generate the observation patterns from the input sequences, and then apply the Viterbi algorithm to identify the gesture. In the experiments, we prove that our method can recognize 18 different continuous gestures effectively. Received: 19 May 1999 / Accepted: 4 September 2000     

多路径卷积神经网络的轮廓感知

目的 引入视觉信息流的整体和局部处理机制,提出了一种多路径卷积神经网络的轮廓感知新方法。方法 利用高斯金字塔尺度分解获得低分辨率子图,用来表征视觉信息中的整体轮廓;通过2维高斯导函数模拟经典感受野的方向选择性,获得描述细节特征的边界响应子图;构建多路径卷积神经网络,利用具有稀疏编码特性的子网络（Sparse-Net）实现对整体轮廓的快速检测;利用具有冗余度增强编码特性的子网络（Redundancy-Net）实现对局部细节特征提取;对上述多路径卷积神经网络响应进行融合编码,以实现轮廓响应的整体感知和局部检测融合,获取轮廓的精细化感知结果。结果 以美国伯克利大学计算机视觉组提供的数据集BSDS500图库为实验对象,在GTX1080Ti环境下本文Sparse-Net对整体轮廓的检测速度达到42幅/s,为HFL方法1.2幅/s的35倍;而Sparse-Net和Redundancy-Net融合后的检测指标数据集尺度上最优（ODS）、图片尺度上最优（OIS）、平均精度（AP）分别为0.806、0.824、0.846,优于HED （holistically-nested edge detection）方法和RCF （richer convolution features for edge detection）方法,结果表明本文方法能有效突出主体轮廓并抑制纹理背景。结论 多路径卷积神经网络的轮廓感知应用,将有助于进一步理解视觉感知机制,并对减弱卷积神经网络的黑盒特性有着重要的意义。     

基于卷积神经网络及改进支持向量机的行人检测

针对自动驾驶实际道路场景复杂导致行人误检率高的问题,提出一种基于卷积神经网络及改进支持向量机的行人检测方法。利用聚合通道特征快速获取图像候选区域,将归一化后的候选区域图像输入卷积神经网络对其进行深度特征提取;利用主成分分析法将卷积神经网络末端所得到的特征向量进行降维处理,减少其冗余特征信息以获得精确的行人特征描述;将行人特征送至优化后的支持向量机完成分类。考虑支持向量机在分类过程中存在核函数参数选择困难的问题,利用改进后的蚁群算法对其进行优化选择,获得最优支持向量机参数以提高分类精度。实验结果表明,不同场景下的行人平均检测精确度达到92%,误检率大幅下降且具有较好的实时性。     

基于改进RCE和RBF神经网络的静态手势识别

针对手势识别的手区域分割、手势特征提取和手势分类的三个过程,提出了一种新的静态手势识别方法。改进了传统的RCE神经网络用于手区域的分割,具有更高的运行速度和更强的抗噪能力。依Freeman链码方向提取手的边缘到掌心的距离作为手势的特征向量。将上一步得到的手势特征向量作为RBF神经网络的输入,进行网络的训练和分类。实验验证了该方法的有效性和可行性,并用其实现了人和仿人机器人的剪刀石头布的猜拳游戏。     

Stereoscopic image discomfort prediction using dual-stream multi-level interactive network

Existing stereoscopic image discomfort prediction methods may fail to work well because they are difficult to extract discomfort features from stereoscopic image’s statistical information since the mechanism of human binocular vision is very complex. In this work, we propose a dual-stream multi-level interactive network that is completely end-to-end trainable for stereoscopic image discomfort prediction. This method first extracts multi-level fusion and difference features from stereoscopic images through a multi-level interaction network. Then, the low-, medium- and high-level feature maps are concatenated to simulate the complicated visual interaction mechanism of the human visual system (HVS). Finally, two fully connected layers are used as a non-linear regression function that maps the feature vectors to stereoscopic image discomfort scores. Extensive experiments demonstrate that our approach performs favorably against the existing prediction models on the IEEE-SA dataset and NBU-S3D dataset.     

Semantic analysis of real-world images using support vector machine

Digital cameras and thus digital images are now ubiquitous. How to efficiently manage a large amount of images has become important. The semantic analysis of images is an important issue in multimedia processing. Region-based image retrieval systems attempt to reduce the gap between high-level semantics and low-level features by representing images at the object level. Recently, the support vector machine (SVM) has been proposed to solve the classification problem. It can generate a hyperplane to separate two sets of features and provides good generalization performance. In this paper, we propose a novel method which integrates principal component analysis (PCA) and SVM neural networks for analyzing the semantic content of natural images, in which principal component analysis (PCA) is applied to reduce the dimension of features. Experimental results show that the proposed method is capable of analyzing the components of photographs into semantic categories with high accuracy, resulting in photographic analysis that is similar to human perception. The performance of the proposed method is better than that of the traditional radial basis function (RBF) neural network.     

多尺度密集时序卷积网络的单幅图像去雨方法

雨滴会降低户外拍摄图像质量,影响图像视觉效果及后续图像分析工作。针对目前去雨算法存在颜色失真、去雨过度化等问题,为了提高计算机视觉算法在中、大雨天气下的准确性,提出多尺度DenseTimeNet(密集时间序列卷积神经网络)的单幅图像去雨方法。该网络由多个尺度DenseTimeNetBlock(密集时序卷积网络密集块)组成,通过卷积下采样技术得到不同尺度下雨线特征信息与降低图像维度后利用时域卷积寻找的时间维度特征信息。在不同维度下学习雨景图和无雨图之间的映射关系,网络主体由密集卷积块和残差网络组成,可加速算法收敛速度,更深度学习图像纹理特征,使特征信息在网络结构进行深度传播,可以更好地复原残损图像。在不同方向,不同大小的雨滴图像上对所提方法进行验证,实验结果表明,该方法相较于现有算法,图像去雨效果良好。     

提高小样本高光谱图像分类性能的变维卷积神经网络

目的 为了解决基于卷积神经网络的算法对高光谱图像小样本分类精度较低、模型结构复杂和计算量大的问题,提出了一种变维卷积神经网络。方法 变维卷积神经网络对高光谱分类过程可根据内部特征图维度的变化分为空—谱信息融合、降维、混合特征提取与空—谱联合分类的过程。这种变维结构通过改变特征映射的维度,简化了网络结构并减少了计算量,并通过对空—谱信息的充分提取提高了卷积神经网络对小样本高光谱图像分类的精度。结果 实验分为变维卷积神经网络的性能分析实验与分类性能对比实验,所用的数据集为Indian Pines和Pavia University Scene数据集。通过实验可知,变维卷积神经网络对高光谱小样本可取得较高的分类精度,在Indian Pines和Pavia University Scene数据集上的总体分类精度分别为87.87%和98.18%,与其他分类算法对比有较明显的性能优势。结论 实验结果表明,合理的参数优化可有效提高变维卷积神经网络的分类精度,这种变维模型可较大程度提高对高光谱图像中小样本数据的分类性能,并可进一步推广到其他与高光谱图像相关的深度学习分类模型中。     

基于索引和相关反馈的图像检索研究

在基于图像内容的图像检索(CBIR)系统中,搜索引擎检索图像类似于按照相似标准来查询图像,它应该有足够快的速度并且有较高的检索准确率.索引用来提高系统响应,而相关反馈用于帮助提高检索准确率.在本文中,主要说明基于人感知的相似性度量,以及讨论综合相关反馈的索引方案.该索引方案通过分析特征熵而得出的主从键,而相关反馈是根据Mann-Whitnev检验而提出的,该检验通常用来识别来自同一搜索集中相关图像和不相关图像之间不同特征,并利用不同特征的特点提高检索性能.相关反馈方案针对两不同相似标准来执行,检验判定了这个方法的有效性.最后,把索引机制和相关反馈机制结合起来建立搜索引擎.     

Recognition of handprinted numerals in VISA® card application forms

An optical character recognition (OCR) framework is developed and applied to handprinted numeric fields recognition. The numeric fields were extracted from binary images of VISA? credit card application forms. The images include personal identity numbers and telephone numbers. The proposed OCR framework is a cascaded neural networks. The first stage is a self-organizing feature map algorithm. The second stage maps distance values into allograph membership values using a gradient descent learning algorithm. The third stage is a multi-layer feedforward network. In this paper, we present experimental results which demonstrate the ability to read handprinted numeric fields. Experiments were performed on a test data set from the CCL/ITRI database which consists of over 90,390 handwritten numeric digits.