Adaptive quasiconformal kernel nearest neighbor classification

Nearest neighbor classification assumes locally constant class conditional probabilities. This assumption becomes invalid in high dimensions due to the curse-of-dimensionality. Severe bias can be introduced under these conditions when using the nearest neighbor rule. We propose an adaptive nearest neighbor classification method to try to minimize bias. We use quasiconformal transformed kernels to compute neighborhoods over which the class probabilities tend to be more homogeneous. As a result, better classification performance can be expected. The efficacy of our method is validated and compared against other competing techniques using a variety of data sets.     

LDA/SVM driven nearest neighbor classification

Nearest neighbor (NN) classification relies on the assumption that class conditional probabilities are locally constant. This assumption becomes false in high dimensions with finite samples due to the curse of dimensionality. The NN rule introduces severe bias under these conditions. We propose a locally adaptive neighborhood morphing classification method to try to minimize bias. We use local support vector machine learning to estimate an effective metric for producing neighborhoods that are elongated along less discriminant feature dimensions and constricted along most discriminant ones. As a result, the class conditional probabilities can be expected to be approximately constant in the modified neighborhoods, whereby better classification performance can be achieved. The efficacy of our method is validated and compared against other competing techniques using a number of datasets.     

Locally adaptive metric nearest-neighbor classification

Nearest-neighbor classification assumes locally constant class conditional probabilities. This assumption becomes invalid in high dimensions with finite samples due to the curse of dimensionality. Severe bias can be introduced under these conditions when using the nearest-neighbor rule. We propose a locally adaptive nearest-neighbor classification method to try to minimize bias. We use a chi-squared distance analysis to compute a flexible metric for producing neighborhoods that are highly adaptive to query locations. Neighborhoods are elongated along less relevant feature dimensions and constricted along most influential ones. As a result, the class conditional probabilities are smoother in the modified neighborhoods, whereby better classification performance can be achieved. The efficacy of our method is validated and compared against other techniques using both simulated and real-world data     

Discriminant adaptive nearest neighbor classification

Nearest neighbour classification expects the class conditional probabilities to be locally constant, and suffers from bias in high dimensions. We propose a locally adaptive form of nearest neighbour classification to try to ameliorate this curse of dimensionality. We use a local linear discriminant analysis to estimate an effective metric for computing neighbourhoods. We determine the local decision boundaries from centroid information, and then shrink neighbourhoods in directions orthogonal to these local decision boundaries, and elongate them parallel to the boundaries. Thereafter, any neighbourhood-based classifier can be employed, using the modified neighbourhoods. The posterior probabilities tend to be more homogeneous in the modified neighbourhoods. We also propose a method for global dimension reduction, that combines local dimension information. In a number of examples, the methods demonstrate the potential for substantial improvements over nearest neighbour classification     

用于文本分类的改进KNN算法

最近邻分类器是假定局部的类条件概率不变,而这个假定在高维特征空间中无效。因此在高维特征空间中使用k最近邻分类器,不对特征权重进行修正就会引起严重的偏差。本文采用灵敏度法,利用前馈神经网络获得初始特征权重并进行二次降维。在初始权重下,根据样本间相似度采用SS树方法将训练样本划分成若干小区域,以此寻找待分类样本的近似k₀个最近邻,并根据近似k₀个最近邻和Chi-square距离原理计算新权重,搜索出新的k个最近邻。此方法在付出较小时间代价的情况下,在文本分离中可获得较好的分类精度的提高。     

一种新的自适应尺度近邻分类器

基于多类别监督学习，提出了一种局部自适应最近邻分类器。此方法使用椭球聚类学习方法估计有效尺度，用于拉长特征不明显的维，并限制特征重要的维。在修正的领域中，类条件概率按预期近似为常数，从而得到更好的分类性能。实验结果显示，对多类问题，这是一种有效且鲁棒的分类方法。     

Improving nearest neighbor classification with cam weighted distance

Nearest neighbor (NN) classification assumes locally constant class conditional probabilities, and suffers from bias in high dimensions with a small sample set. In this paper, we propose a novel cam weighted distance to ameliorate the curse of dimensionality. Different from the existing neighborhood-based methods which only analyze a small space emanating from the query sample, the proposed nearest neighbor classification using the cam weighted distance (CamNN) optimizes the distance measure based on the analysis of inter-prototype relationship. Our motivation comes from the observation that the prototypes are not isolated. Prototypes with different surroundings should have different effects in the classification. The proposed cam weighted distance is orientation and scale adaptive to take advantage of the relevant information of inter-prototype relationship, so that a better classification performance can be achieved. Experiments show that CamNN significantly outperforms one nearest neighbor classification (1-NN) and k-nearest neighbor classification (k-NN) in most benchmarks, while its computational complexity is comparable with that of 1-NN classification.     

基于局部自适应逼近的半监督反馈算法

将鉴别信息引入到距离测度中,利用这个新的局部距离测度代替欧氏距离构建k-近邻,提出一种新的局部线性近邻扩展算法.将此用于图像检索的相关反馈机制,产生基于局部自适应逼近的半监督反馈算法FLANNP(feedback locally adaptive nearest neighbor propagation).该方法首先由支持向量机构建的判别函数来确定最优判别方向,基于此方向产生一个局部自适应距离算法,进而确定数据点间的权重.最后,标签信息由全局一致性假设,通过局部最近邻,从有标签数据点开始进行全局扩散标注.该方法使用有鉴别信息的距离测度,提高了图像检索的准确度.     

A simple and fast representation-based face recognition method

In this paper, we propose a very simple and fast face recognition method and present its potential rationale. This method first selects only the nearest training sample, of the test sample, from every class and then expresses the test sample as a linear combination of all the selected training samples. Using the expression result, the proposed method can classify the testing sample with a high accuracy. The proposed method can classify more accurately than the nearest neighbor classification method (NNCM). The face recognition experiments show that the classification accuracy obtained using our method is usually 2–10% greater than that obtained using NNCM. Moreover, though the proposed method exploits only one training sample per class to perform classification, it might obtain a better performance than the nearest feature space method proposed in Chien and Wu (IEEE Trans Pattern Anal Machine Intell 24:1644–1649, 2002), which depends on all the training samples to classify the test sample. Our analysis shows that the proposed method achieves this by modifying the neighbor relationships between the test sample and training samples, determined by the Euclidean metric.     

基于自适应距离度量的分类器设计方法

通过对欧氏距离度量的分析,提出了自适应距离度量.首先利用训练样本建立自适应距离度量模型,该模型保证了训练样本到相同模式类的距离最近,到不同模式类的距离最远,根据该模型建立目标函数,求解目标函数,得到最优权重.基于最小距离分类器和K近邻分类器,采用UCI标准数据库中部分数据,对提出的自适应距离度量和欧氏距离度量进行了实验比较,实验结果表明自适应距离度量更有效.     

Fisher鉴别特征的最近邻凸包分类

基于Fisher准则的特征提取方法是模式识别技术的重要分支，其中，Foley-Sammon变换和具有统计不相关性的最佳鉴别变换是这一技术典型代表，本文将它们与一种新型分类器一最近邻凸包分类器相结合，从而实现Fisher鉴别特征的有效分类。最近邻凸包分类器是一类以测试样本点到各类训练集生成类别凸包的距离为分类判别依据的模式分类新方法，具有非线性性，无参性，多类别适用性等特点。实验证实了本文方法的有效性。     

一种融合语义距离的最近邻图像标注方法

传统的基于最近邻的图像标注方法效果不佳,主要原因在于提取图像视觉特征时,损失了很多有价值的信息.提出了一种改进的最近邻分类模型.首先利用距离测度学习方法,引入图像的语义类别信息进行训练,生成新的语义距离;然后利用该距离对每一类图像进行聚类,生成多个类内的聚类中心;最后通过计算图像到各个聚类中心的语义距离来构建最近邻分类模型.在构建最近邻分类模型的整个过程中,都使用训练得到的语义距离来计算,这可以有效减少相同图像类内的变动和不同图像类之间的相似所造成的语义鸿沟.在ImageCLEF2012图像标注数据库上进行了实验,将本方法与传统分类模型和最新的方法进行了比较,验证了本方法的有效性.     

Fast minimization of structural risk by nearest neighbor rule

In this paper, we present a novel nearest neighbor rule-based implementation of the structural risk minimization principle to address a generic classification problem. We propose a fast reference set thinning algorithm on the training data set similar to a support vector machine (SVM) approach. We then show that the nearest neighbor rule based on the reduced set implements the structural risk minimization principle, in a manner which does not involve selection of a convenient feature space. Simulation results on real data indicate that this method significantly reduces the computational cost of the conventional SVMs, and achieves a nearly comparable test error performance.     

利用支持向量机识别汽车颜色

大类别数分类时支持向量机(SVM)数量较多，文中通过类别合并和特征空间分解，结合决策树判别方法．对SVM数量进行优化，提出了一种基于优化SVM的汽车颜色识别方法．该方法与最近邻分类方法相比，无论是在速度上还是识别正确率上都得到了提高．实验结果表明，该方法是一种快速且正确率较高的多类别分类方法，可以满足实时识别的要求．     

Explaining Classifications For Individual Instances

We present a method for explaining predictions for individual instances. The presented approach is general and can be used with all classification models that output probabilities. It is based on the decomposition of a model's predictions on individual contributions of each attribute. Our method works for the so-called black box models such as support vector machines, neural networks, and nearest neighbor algorithms, as well as for ensemble methods such as boosting and random forests. We demonstrate that the generated explanations closely follow the learned models and present a visualization technique that shows the utility of our approach and enables the comparison of different prediction methods.     

邻域自适应SMOTE算法研究

针对SMOTE(synthetic minority over-sampling technique)等基于近邻值的传统过采样算法在处理类不平衡数据时近邻参数不能根据少数类样本的分布及时调整的问题,提出邻域自适应SMOTE算法AdaN_SMOTE.为使合成数据保留少数类的原始分布,跟踪精度下降点确定每个少数类数据的近邻值,并根据噪声、小析取项或复杂的形状及时调整近邻值的大小;合成数据保留了少数类的原始分布,算法分类性能更佳.在KE E L数据集上进行实验对比验证,结果表明AdaN_SMOTE分类性能优于其他基于近邻值的过采样方法,且在有噪声的数据集中更有效.     

基于最近邻的遥感影像单类信息提取

遥感影像单类信息提取是一种特殊的分类,旨在训练和提取单一兴趣类别。研究了基于最近邻分类器的单类信息提取方法,包括类别划分和样本选择问题。首先分析论证了最近邻方法提取单类信息只与所选择的样本相关,而与类别划分无关,因此可以将单类信息提 取作为二类分类问题进行处理。然后在二类分类问题中,根据空间和特征邻近性选择非兴趣类别的部分训练样本,简化了分类过程。实验结果表明,所提出的方法可以有效实现遥感影像单类信息的提取。     

Adaptive binary tree for fast SVM multiclass classification

This paper presents an adaptive binary tree (ABT) to reduce the test computational complexity of multiclass support vector machine (SVM). It achieves a fast classification by: (1) reducing the number of binary SVMs for one classification by using separating planes of some binary SVMs to discriminate other binary problems; (2) selecting the binary SVMs with the fewest average number of support vectors (SVs). The average number of SVs is proposed to denote the computational complexity to exclude one class. Compared with five well-known methods, experiments on many benchmark data sets demonstrate our method can speed up the test phase while remain the high accuracy of SVMs.     

Naive Bayes text classifiers: a locally weighted learning approach

Due to being fast, easy to implement and relatively effective, some state-of-the-art naive Bayes text classifiers with the strong assumption of conditional independence among attributes, such as multinomial naive Bayes, complement naive Bayes and the one-versus-all-but-one model, have received a great deal of attention from researchers in the domain of text classification. In this article, we revisit these naive Bayes text classifiers and empirically compare their classification performance on a large number of widely used text classification benchmark datasets. Then, we propose a locally weighted learning approach to these naive Bayes text classifiers. We call our new approach locally weighted naive Bayes text classifiers (LWNBTC). LWNBTC weakens the attribute conditional independence assumption made by these naive Bayes text classifiers by applying the locally weighted learning approach. The experimental results show that our locally weighted versions significantly outperform these state-of-the-art naive Bayes text classifiers in terms of classification accuracy.     

Attribute weighted Naive Bayes classifier using a local optimization

The Naive Bayes classifier is a popular classification technique for data mining and machine learning. It has been shown to be very effective on a variety of data classification problems. However, the strong assumption that all attributes are conditionally independent given the class is often violated in real-world applications. Numerous methods have been proposed in order to improve the performance of the Naive Bayes classifier by alleviating the attribute independence assumption. However, violation of the independence assumption can increase the expected error. Another alternative is assigning the weights for attributes. In this paper, we propose a novel attribute weighted Naive Bayes classifier by considering weights to the conditional probabilities. An objective function is modeled and taken into account, which is based on the structure of the Naive Bayes classifier and the attribute weights. The optimal weights are determined by a local optimization method using the quasisecant method. In the proposed approach, the Naive Bayes classifier is taken as a starting point. We report the results of numerical experiments on several real-world data sets in binary classification, which show the efficiency of the proposed method.