Self-generating prototypes for pattern classification

Prototype classifiers are a type of pattern classifiers, whereby a number of prototypes are designed for each class so as they act as representatives of the patterns of the class. Prototype classifiers are considered among the simplest and best performers in classification problems. However, they need careful positioning of prototypes to capture the distribution of each class region and/or to define the class boundaries. Standard methods, such as learning vector quantization (LVQ), are sensitive to the initial choice of the number and the locations of the prototypes and the learning rate. In this article, a new prototype classification method is proposed, namely self-generating prototypes (SGP). The main advantage of this method is that both the number of prototypes and their locations are learned from the training set without much human intervention. The proposed method is compared with other prototype classifiers such as LVQ, self-generating neural tree (SGNT) and K-nearest neighbor (K-NN) as well as Gaussian mixture model (GMM) classifiers. In our experiments, SGP achieved the best performance in many measures of performance, such as training speed, and test or classification speed. Concerning number of prototypes, and test classification accuracy, it was considerably better than the other methods, but about equal on average to the GMM classifiers. We also implemented the SGP method on the well-known STATLOG benchmark, and it beat all other 21 methods (prototype methods and non-prototype methods) in classification accuracy.     

一种改进的支持向量机NN-SVM

支持向量机(SVM)是一种较新的机器学习方法，它利用靠近边界的少数向量构造一个最优分类超平面。在训练分类器时，SVM的着眼点在于两类的交界部分，那些混杂在另一类中的点往往无助于提高分类器的性能，反而会大大增加训练器的计算负担，同时它们的存在还可能造成过学习，使泛化能力减弱．为了改善支持向量机的泛化能力，该文在其基础上提出了一种改进的SVM—NN-SVM：它先对训练集进行修剪，根据每个样本与其最近邻类标的异同决定其取舍，然后再用SVM训练得到分类器．实验表明，NN-SVM相比SVM在分类正确率、分类速度以及适用的样本规模上都表现出了一定的优越性．     

Extraction rice-planted areas by RADARSAT data using neural networks

A classification technique using the neural networks has recently been developed. We apply a neural network of learning vector quantization (LVQ) to classify remote-sensing data, including microwave and optical sensors, for the estimation of a rice-planted area. The method has the capability of nonlinear discrimination, and the classification function is determined by learning. The satellite data were observed before and after planting rice in 1999. Three sets of RADARSAT and one set of SPOT/HRV data were used in Higashi–Hiroshima, Japan. Three RADARSAT images from April to June were used for this study. The LVQ classification was applied the RADARSAT and SPOT to evaluate the estimate of the area of planted-rice. The results show that the true production rate of the rice-planted area estimation of RADASAT by LVQ was approximately 60% compared with that of SPOT by LVQ. It is shown that the present method is much better than the SAR image classification by the maximum likelihood method.     

事件检测支持向量机模型与神经网络模型比较

针对交通领域中的事件检测(无事件模式和有事件模式)模式识别问题,描述了支持向量机(SVM)的基本方法,建立了基于线性(linearfunction)、多项式(polynomialfunction)和径向基(radialbasisfunction)3种核函数的事件检测SVM模型,并与PNN、MLF模型进行了理论比较。采用I-880线圈数据集和事件数据集建立并验证SVM、PNN和MLF模型,结果发现:无论对于向北、向南或混合方向的事件检测,SVM模型的检测率(DR)、误报率(FAR)和平均检测时间(MTTD)指标均比MLF模型好;PNN模型的DR比SVM(P)模型的高,但FAR和MTTD指标不比SVM(P)模型好;在3个SVM模型中,SVM(P)检测效果最好,SVM(L)最差。SVM算法与神经网络算法相比具有避免局部最小,实现全局最优化,更好的泛化效果的优点,是高速公路事件检测的一种很有潜力的算法。     

Relationship Between Support Vector Set and Kernel Functions in SVM

Based on a constructive learning approach,covering algorithms,we investigate the relationship between support vector sets and kernel functions in support vector machines (SVM).An interesting result is obtained.That is,in the linearly non-separable case,any sample of a given sample set K can become a support vector under a certain kernel function.The result shows that when the sample set K is linearly non-separable,although the chosen kernel function satisfies Mercer‘s condition its corresponding support vector set is not necessarily the subset of K that plays a crucial role in classifying K.For a given sample set,what is the subset that plays the crucial role in classification?In order to explore the problem,a new concept,boundary or boundary points,is defined and its properties are discussed.Given a sample set K,we show that the decision functions for classifying the boundary points of K are the same as that for classifying the K itself.And the boundary points of K only depend on K and the structure of the space at which k is located and independent of the chosen approach for finding the boundary.Therefore,the boundary point set may become the subset of K that plays a crucial role in classification.These results are of importance to understand the principle of the support vector machine(SVM) and to develop new learning algorithms.     

应用多索引加法量化编码的近邻检索算法

目的 海量图像检索技术是计算机视觉领域研究热点之一,一个基本的思路是对数据库中所有图像提取特征,然后定义特征相似性度量,进行近邻检索。海量图像检索技术,关键的是设计满足存储需求和效率的近邻检索算法。为了提高图像视觉特征的近似表示精度和降低图像视觉特征的存储空间需求,提出了一种多索引加法量化方法。方法 由于线性搜索算法复杂度高,而且为了满足检索的实时性,需把图像描述符存储在内存中,不能满足大规模检索系统的需求。基于非线性检索的优越性,本文对非穷尽搜索的多索引结构和量化编码进行了探索新研究。利用多索引结构将原始数据空间划分成多个子空间,把每个子空间数据项分配到不同的倒排列表中,然后使用压缩编码的加法量化方法编码倒排列表中的残差数据项,进一步减少对原始空间的量化损失。在近邻检索时采用非穷尽搜索的策略,只在少数倒排列表中检索近邻项,可以大大减少检索时间成本,而且检索过程中不用存储原始数据,只需存储数据集中每个数据项在加法量化码书中的码字索引,大大减少内存消耗。结果 为了验证算法的有效性,在3个数据集SIFT、GIST、MNIST上进行测试,召回率相比近几年算法提升4%~15%,平均查准率提高12%左右,检索时间与最快的算法持平。结论 本文提出的多索引加法量化编码算法,有效改善了图像视觉特征的近似表示精度和存储空间需求,并提升了在大规模数据集的检索准确率和召回率。本文算法主要针对特征进行近邻检索,适用于海量图像以及其他多媒体数据的近邻检索。     

Learning effective color features for content based image retrieval in dermatology

We investigate the extraction of effective color features for a content-based image retrieval (CBIR) application in dermatology. Effectiveness is measured by the rate of correct retrieval of images from four color classes of skin lesions. We employ and compare two different methods to learn favorable feature representations for this special application: limited rank matrix learning vector quantization (LiRaM LVQ) and a Large Margin Nearest Neighbor (LMNN) approach. Both methods use labeled training data and provide a discriminant linear transformation of the original features, potentially to a lower dimensional space. The extracted color features are used to retrieve images from a database by a k-nearest neighbor search. We perform a comparison of retrieval rates achieved with extracted and original features for eight different standard color spaces. We achieved significant improvements in every examined color space. The increase of the mean correct retrieval rate lies between 10% and 27% in the range of k=1-25 retrieved images, and the correct retrieval rate lies between 84% and 64%. We present explicit combinations of RGB and CIE-Lab color features corresponding to healthy and lesion skin. LiRaM LVQ and the computationally more expensive LMNN give comparable results for large values of the method parameter κ of LMNN (κ≥25) while LiRaM LVQ outperforms LMNN for smaller values of κ. We conclude that feature extraction by LiRaM LVQ leads to considerable improvement in color-based retrieval of dermatologic images.     

Initialization insensitive LVQ algorithm based on cost-function adaptation

A learning vector quantization (LVQ) algorithm called harmonic to minimum LVQ algorithm (H2M-LVQ)¹ is presented to tackle the initialization sensitiveness problem associated with the original generalized LVQ (GLVQ) algorithm. Experimental results show superior performance of the H2M-LVQ algorithm over the GLVQ and one of its variants on several datasets.     

Fast agglomerative clustering using information of k-nearest neighbors

In this paper, we develop a method to lower the computational complexity of pairwise nearest neighbor (PNN) algorithm. Our approach determines a set of candidate clusters being updated after each cluster merge. If the updating process is required for some of these clusters, k-nearest neighbors are found for them. The number of distance calculations for our method is O(N²), where N is the number of data points. To further reduce the computational complexity of the proposed algorithm, some available fast search approaches are used. Compared to available approaches, our proposed algorithm can reduce the computing time and number of distance calculations significantly. Compared to FPNN, our method can reduce the computing time by a factor of about 26.8 for the data set from a real image. Compared with PMLFPNN, our approach can reduce the computing time by a factor of about 3.8 for the same data set.     

Divergence-based classification in learning vector quantization

We discuss the use of divergences in dissimilarity-based classification. Divergences can be employed whenever vectorial data consists of non-negative, potentially normalized features. This is, for instance, the case in spectral data or histograms. In particular, we introduce and study divergence based learning vector quantization (DLVQ). We derive cost function based DLVQ schemes for the family of γ‐divergences which includes the well-known Kullback-Leibler divergence and the so-called Cauchy-Schwarz divergence as special cases. The corresponding training schemes are applied to two different real world data sets. The first one, a benchmark data set (Wisconsin Breast Cancer) is available in the public domain. In the second problem, color histograms of leaf images are used to detect the presence of cassava mosaic disease in cassava plants. We compare the use of standard Euclidean distances with DLVQ for different parameter settings. We show that DLVQ can yield superior classification accuracies and Receiver Operating Characteristics.     

一种基于改进CP网络与HMM相结合的混合音素识别方法

提出了一种基于改进对偶传播（ＣＰ）神经网络与隐驰尔可夫模型（ＨＭＭ）相结合的混合音素识别方法．这一方法的特点是用一个具有有指导学习矢量量化（ＬＶＱ）和动态节点分配等特性的改进的ＣＰ网络生成离散ＨＭＭ音素识别系统中的码书。因此,用这一方法构造的混合音素识别系统中的码书实际上是一个由有指导ＬＶＱ算法训练的具有很强分类能力的高性能分类器,这就意味着在用ＨＭＭ对语音信号进行建模之前,由码书产生的观测序列中     

Soft nearest prototype classification

We propose a new method for the construction of nearest prototype classifiers which is based on a Gaussian mixture ansatz and which can be interpreted as an annealed version of learning vector quantization (LVQ). The algorithm performs a gradient descent on a cost-function minimizing the classification error on the training set. We investigate the properties of the algorithm and assess its performance for several toy data sets and for an optical letter classification task. Results show 1) that annealing in the dispersion parameter of the Gaussian kernels improves classification accuracy; 2) that classification results are better than those obtained with standard learning vector quantization (LVQ 2.1, LVQ 3) for equal numbers of prototypes; and 3) that annealing of the width parameter improved the classification capability. Additionally, the principled approach provides an explanation of a number of features of the (heuristic) LVQ methods.     

基于矢量量化的快速图像检索

传统索引方法对高维数据存在"维数灾难"的困难.而对数据分布的精确描述及对数据空间的有效划分是高维索引机制中的关键问题.提出一种基于矢量量化的索引方法.该方法使用高斯混合模型描述数据的整体分布,并训练优化的矢量量化器划分数据空间.高斯混合模型能更好地描述真实图像库的数据分布;而矢量量化的划分方法可以充分利用维之间的统计相关性,能够对数据向量构造出更加精确的近似表示,从而提高索引结构的过滤效率并减少需要访问的数据向量.在大容量真实图像库上的实验表明,该方法显著减少了支配检索时间的I/O开销,提高了索引性能.     

Self-Organizing Maps and Learning Vector Quantization for Feature Sequences

The Self-Organizing Map (SOM) and Learning Vector Quantization (LVQ) algorithms are constructed in this work for variable-length and warped feature sequences. The novelty is to associate an entire feature vector sequence, instead of a single feature vector, as a model with each SOM node. Dynamic time warping is used to obtain time-normalized distances between sequences with different lengths. Starting with random initialization, ordered feature sequence maps then ensue, and Learning Vector Quantization can be used to fine tune the prototype sequences for optimal class separation. The resulting SOM models, the prototype sequences, can then be used for the recognition as well as synthesis of patterns. Good results have been obtained in speaker-independent speech recognition.     

Prototype reduction schemes applicable for non-stationary data sets

All of the prototype reduction schemes (PRS) which have been reported in the literature, process time-invariant data to yield a subset of prototypes that are useful in nearest-neighbor-like classification. Although these methods have been proven to be powerful, they suffer from a major disadvantage when they are utilized for applications involving non-stationary data, namely, time varying samples, typical of video and multimedia applications. In this paper, we suggest two PRS mechanisms which, in turn, are suitable for two distinct models of non-stationarity. In the first model, the data points obtained at discrete time steps, are individually assumed to be perturbed in the feature space, because of noise in the measurements or features. As opposed to this, in the second model, we assume that, at discrete time steps, new data points are available, and that these themselves are generated due to a non-stationarity in the parameters of the feature space. In both of these cases, rather than process all the data as a whole set using a PRS, we propose that the information gleaned from a previous PRS computation be enhanced to yield the prototypes for the current data set using an LVQ-3 type “fine tuning”. The results are, to our knowledge, the first reported PRS results for non-stationary data, and can be summarized as follows: if the system obeys the first model of non-stationarity, the improved accuracy is as high as 90.98% for artificial data “Non_normal 2”, and as high as 97.62% for the real-life data set, “Arrhythmia”. As opposed to this, if the system obeys the second model of non-stationarity, the improved accuracy is as high as 76.30% for the artificial data, and as high as 97.40% for this real-life data set. These are, in our opinion, very impressive, considering that the data sets are truly time-varying.     

视频图像高压缩比编码算法研究

提出了一种基于DCT变换的矢量自适应分类的全局矢量量化编码算法。为降低码矢的维数和计算复杂度，提高搜索速度和压缩比，将变换的DCT矢量自适应分类为平滑类、边缘类和纹理类，根据矢量的类别构造不同长度的变换矢量和根据矢量的类别分别采用改进的全局矢量量化算法进行相应的码书设计。为提高光照变化时相邻帧间矢量运动补偿的匹配率，在矢量构造中将DC系数单独进行编码。实验结果表明：该算法在信噪比和压缩比方面具有良好的视频压缩性能，比较适合于智能视频监控系统以及水下视频等光照随时间有较大变化的场合。     

Rough set based 1-v-1 and 1-v-r approaches to support vector machine multi-classification

Support vector machines (SVMs) are essentially binary classifiers. To improve their applicability, several methods have been suggested for extending SVMs for multi-classification, including one-versus-one (1-v-1), one-versus-rest (1-v-r) and DAGSVM. In this paper, we first describe how binary classification with SVMs can be interpreted using rough sets. A rough set approach to SVM classification removes the necessity of exact classification and is especially useful when dealing with noisy data. Next, by utilizing the boundary region in rough sets, we suggest two new approaches, extensions of 1-v-r and 1-v-1, to SVM multi-classification that allow for an error rate. We explicitly demonstrate how our extended 1-v-r may shorten the training time of the conventional 1-v-r approach. In addition, we show that our 1-v-1 approach may have reduced storage requirements compared to the conventional 1-v-1 and DAGSVM techniques. Our techniques also provide better semantic interpretations of the classification process. The theoretical conclusions are supported by experimental findings involving a synthetic dataset.     

基于支持向量机的计算机键盘用户身份验真

口令认证因为简便易实现而被大多数计算机系统所采用，但容易被盗用，存在着严重的安全隐患，而利用对用户的键入特性的识别，可以大大加强口令认证的可靠性，在对国内外众多学者所做工作研究的基础上，鉴于支持向量机在进行模式识别对所具有的优良性能，提出利用支持向量机进行键入特性验真，并通过实验将其与BP，RBF，PNN和LVQ四种神经网络模型进行比较，证实采用SVM进行键入特性验真的有效性，因而其具有广阔的应用前景。     

改进径向基函数神经网及其在手写体字符识别中的应用

提出一种基于半模型矢量量化（ＳＦＶＱ）技术的改进径向基函数神经网（ＩＲＢＦＮＮ）分类器,并且用于无约束手写体数字的识别。作者在模糊聚类和矢量量化的基础上利用半模糊的思想提出了半模糊矢量量化算法,并在其中加入了有监督的控制,从而使系统在聚类过程中可以确定比较合适的类别数并使聚类结果能更好地反映训练集的概率分布。以半模糊矢量量化作为预处理的改进ＲＢＦ网,应用了多尺度补偿等办法,能够充分利用训练样本集的