基于Bhattacharyya距离准则的核空间特征提取算法

提出了一种新的以Bhattacharyya距离为准则的核空间特征提取算法．该算法的核心思想是把样本非线性映射到高维核空间．在核空间中寻找一组最优特征向量，然后把样本线性映射到低维特征空间，使类别间的Bhattacharyya距离最大。从而保证Bayes分类误差上界最小．采用核函数技术，把特征提取问题转化为一个QP(Quadratic Programming)优化问题．保证了算法的全局收敛性和快速性．此算法具有两个优点：(1)该算法提取的特征对数据分类来说更有效；(2)对于给定的模式分类问题，算法可以预测出在不损失分类精度情况下所必须的特征向量数目的上界，并能够提取出分类有效特征．实验结果表明，该算法的性能与理论分析的结论相吻合，优于目前常用的特征提取算法．

Feature Extraction in Kernel Space Using Bhattacharyya Distance as Criterion Function

Authors propose a novel approach to feature extraction for classification in kernel space using Bhattacharyya distance, determining the upper-bond of Bayes error, as criterion function, which is called BKFE. The key idea of BKFE is that the data are nonlinearly mapped into high dimensional kernel space at first. Then we can find a set of discriminantly informative features in kernel space to linearly map the data into low dimensional feature space, where the Bhattacharyya distances between classes are maximized. First authors draw on kernel theory and nonlinear optimization technique to develop BKFE for binary classification problem, and solves the feature extraction problem by quadratic programming method, which endows BKFE with fast and global convergence. Then authors extend BKFE to multi-class classification problem. After defining the kernel space feature extraction matrix (KFEM), they discuss theoretically in some depth the relationship exiting between KFEM and intrinsic discriminant dimension of a classification problem, gives Theorem 1 for this relationship. According to theorem 1, the rank of KFEM is equal to the intrinsic discriminant dimension, and its eigenvectors with non-zero eigenvalue are discriminantly informative features. Because of high dimension, the rank and eigenvectors of KFEM are difficult to calculate directly. In order to overcome the difficulty, authors construct a new low dimension kernel matrix, which has the same rank as KFEM, and extract discriminantly informative features by calculating the eigenvectors of the new kernel matrix. In particular, the upper-bond of intrinsic discriminant dimension for a classification problem is M(M-1)/2 (M is class number). It is important because it indicates how many features should be extracted for a given classification problem. Compared with KPCA (Kernel Principal Components Analysis), KFD (Kernel Fisher Discriminate), and FD (Fisher Discriminate), BKFE has two desirable advantages: (1) features extracted by BKFE are more effective for classification; (2) it predicts the upper-bound of the number of necessary features to achieve the same classification accuracy as in the original space for a given pattern recognition problem. Experimental results show that BKFE can provide more informative features for pattern classification than others.