Kernel class-wise locality preserving projection

In the recent years, the pattern recognition community paid more attention to a new kind of feature extraction method, the manifold learning methods, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among them, locality preserving projection (LPP) is one of the most promising feature extraction techniques. However, when LPP is applied to the classification tasks, it shows some limitations, such as the ignorance of the label information. In this paper, we propose a novel local structure based feature extraction method, called class-wise locality preserving projection (CLPP). CLPP utilizes class information to guide the procedure of feature extraction. In CLPP, the local structure of the original data is constructed according to a certain kind of similarity between data points, which takes special consideration of both the local information and the class information. The kernelized (nonlinear) counterpart of this linear feature extractor is also established in the paper. Moreover, a kernel version of CLPP namely Kernel CLPP (KCLPP) is developed through applying the kernel trick to CLPP to increase its performance on nonlinear feature extraction. Experiments on ORL face database and YALE face database are performed to test and evaluate the proposed algorithm.     

Discriminant Analysis with Local Gaussian Similarity Preserving for Feature Extraction

In this paper, we propose a novel discriminant analysis with local Gaussian similarity preserving (DA-LGSP) method for feature extraction. DA-LGSP can be viewed as a linear approximation of manifold learning based approach which seeks to find a linear projection that maximizes the between-class dissimilarities under the constraint of locality preserving. The local geometry of each point is preserved by the Gaussian coefficients of its neighbors, meanwhile the between-class dissimilarities are represented by Euclidean distances. Experiments are conducted on USPA data, COIL-20 dataset, ORL dataset and FERET dataset. The performance of the proposed method demonstrates that DA-LGSP is effective in feature extraction.     

基于大间距准则的不相关保局投影分析

局部保持投影(Locality preserving projections,LPP)算法只保持了目标在投影后的邻域局部信息,为了更好地刻画数据的流形结构, 引入了类内和类间局部散度矩阵,给出了一种基于有效且稳定的大间距准则(Maximum margin criterion,MMC)的不相关保局投影分析方法.该方法在最大化散度矩阵迹差时,引入尺度因子α,对类内和类间局部散度矩阵进行加权,以便找到更适合分类的子空间并且可避免小样本问题; 更重要的是,大间距准则下提取的判别特征集一般情况下是统计相关的,造成了特征信息的冗余, 因此,通过增加一个不相关约束条件,利用推导出的公式提取不相关判别特征集, 这样做, 对正确识别更为有利.在Yale人脸库、PIE人脸库和MNIST手写数字库上的测试结果表明,本文方法有效且稳定, 与LPP、LDA (Linear discriminant analysis)和LPMIP(Locality-preserved maximum information projection)方法等相比,具有更高的正确识别率.     

基于全局不相关的多流形学习

为提升人脸识别算法的鲁棒性,减少判别信息的冗余度,提出基于全局不相关的多流形判别学习算法(UFDML)。使用特征空间到特征空间的距离,学习样本局部判别信息,提出全局不相关约束,使提取的判别特征是统计不相关的。在Yale,AR,ORL人脸库上的实验结果表明,与LPP(局部保持投影)、LDA(线性判别分析)、UDP(非监督判别投影)等人脸识别算法相比,所提算法的平均识别率高于其它算法,验证了其有效性。     

基于大间距准则和图像矩阵双向投影的人脸特征提取方法

传统基于降维技术的人脸特征提取需要将图像转换成更高维的向量, 从而加剧维数灾难问题, 对于采用Fisher优化准则的特征提取, 这也会使小样本问题更加突出. 基于图像的矩阵表示, 本文提出了一种新的基于大间距准则和矩阵双向投影技术的人脸特征提取方法(Maximum margin criterion and image matrix bidirectional projection, MMC-MBP). 该方法一方面在计算散度矩阵时引入了能保持数据局部性的Laplacian矩阵, 以保持数据的流形结构, 从而提高识别正确率; 另一方面采用了有效且稳定的大间距的优化准则即最大化矩阵迹差, 能克服利用Fisher准则所带来的小样本问题; 更重要的, MMC-MBP方法给出了求解最优双向投影矩阵的迭代计算过程, 该迭代求解过程能保证目标函数的单调递增性、收敛性以及投影矩阵的收敛性, 从而成功解决了传统基于张量(矩阵)投影技术的特征提取方法特征维数过高或者无收敛解的问题. 最后广泛而系统的人脸识别实验表明, MMC-MBP的迭代求解过程能很快收敛, 且相比Eigenfaces, Fisherfaces, Laplacianfaces等脸识别方法, 具有更高的识别正确率, 是一种有效的人脸特征提取方法.     

Learning a locality discriminating projection for classification

This paper introduces a new algorithm called locality discriminating projection (LDP) for subspace learning, which provides a new scheme for discriminant analysis by considering both the manifold structure and the prior class information. In the LDP algorithm, the overlap among the class-specific manifolds is approximated by an invader graph, and a locality discriminant criterion is proposed to find the projections that best preserve the within-class local structures while decrease the between-class overlap. The feasibility of the LDP algorithm has been successfully tested in text data and visual recognition experiments. Experiment results show it is an effective technique for data modeling and classification comparing to linear discriminant analysis, locality preserving projection, and marginal Fisher analysis.     

Locality preserving embedding for face and handwriting digital recognition

Most supervised manifold learning-based methods preserve the original neighbor relationships to pursue the discriminating power. Thus, structure information of the data distributions might be neglected and destroyed in low-dimensional space in a certain sense. In this paper, a novel supervised method, called locality preserving embedding (LPE), is proposed to feature extraction and dimensionality reduction. LPE can give a low-dimensional embedding for discriminative multi-class sub-manifolds and preserves principal structure information of the local sub-manifolds. In LPE framework, supervised and unsupervised ideas are combined together to learn the optimal discriminant projections. On the one hand, the class information is taken into account to characterize the compactness of local sub-manifolds and the separability of different sub-manifolds. On the other hand, at the same time, all the samples in the local neighborhood are used to characterize the original data distributions and preserve the structure in low-dimensional subspace. The most significant difference from existing methods is that LPE takes the distribution directions of local neighbor data into account and preserves them in low-dimensional subspace instead of only preserving the each local sub-manifold’s original neighbor relationships. Therefore, LPE optimally preserves both the local sub-manifold’s original neighborhood relationships and the distribution direction of local neighbor data to separate different sub-manifolds as far as possible. The criterion, similar to the classical Fisher criterion, is a Rayleigh quotient in form, and the optimal linear projections are obtained by solving a generalized Eigen equation. Furthermore, the framework can be directly used in semi-supervised learning, and the semi-supervised LPE and semi-supervised kernel LPE are given. The proposed LPE is applied to face recognition (on the ORL and Yale face databases) and handwriting digital recognition (on the USPS database). The experimental results show that LPE consistently outperforms classical linear methods, e.g., principal component analysis and linear discriminant analysis, and the recent manifold learning-based methods, e.g., marginal Fisher analysis and constrained maximum variance mapping.     

基于判别正则化的局部保留投影

局部保留投影（Locality preserving projections,LPP）是一种常用的线性化流形学习方法,其通过线性嵌入来保留基于图所描述的流形数据本质结构特征,因此LPP对图的依赖性强,且在嵌入过程中缺少对图描述的进一步分析和挖掘。当图对数据本质结构特征描述不恰当时,LPP在嵌入过程中不易实现流形数据本质结构的有效提取。为了解决这个问题,本文在给定流形数据图描述的条件下,通过引入局部相似度阈值进行局部判别分析,并据此建立判别正则化局部保留投影（简称DRLPP）。该方法能够在现有图描述的条件下,有效突出不同流形结构在线性嵌入空间中的可分性。在人造合成数据集和实际标准数据集上对DRLPP以及相关算法进行对比实验,实验结果证明了DRLPP的有效性。     

融合局部结构和差异信息的监督特征提取算法

针对监督局部保持投影(Supervised locality preserving projection, SLPP)存在过学习和不能较好地保持图像空间的差异信息等问题,造成算法性能不够好, 提出了一种新的基于流形学习的监督特征提取方法(Supervised local structure and diversity projection, S-LSDP). S-LSDP从信息统计量角度引入差异信息,并给出度量差异信息大小的准则(差异离散度) 及明确的物理含义;然后通过最小化局部离散度和最大化差异离散度准则提取投影方向. 投影后的特征既能有效地保持图像之间的局部结构属性,又能较好地保持图像之间的差异信息, 而且避免了过学习问题.在UMIST, Yale, PIE和AR数据库上的实验结果表明了该算法的有效性.     

Locality preserving and global discriminant projection with prior information

Existing supervised and semi-supervised dimensionality reduction methods utilize training data only with class labels being associated to the data samples for classification. In this paper, we present a new algorithm called locality preserving and global discriminant projection with prior information (LPGDP) for dimensionality reduction and classification, by considering both the manifold structure and the prior information, where the prior information includes not only the class label but also the misclassification of marginal samples. In the LPGDP algorithm, the overlap among the class-specific manifolds is discriminated by a global class graph, and a locality preserving criterion is employed to obtain the projections that best preserve the within-class local structures. The feasibility of the LPGDP algorithm has been evaluated in face recognition, object categorization and handwritten Chinese character recognition experiments. Experiment results show the superior performance of data modeling and classification to other techniques, such as linear discriminant analysis, locality preserving projection, discriminant locality preserving projection and marginal Fisher analysis.     

Supervised optimal locality preserving projection

In the past few years, the computer vision and pattern recognition community has witnessed a rapid growth of a new kind of feature extraction method, the manifold learning methods, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among these methods, locality preserving projection (LPP) is one of the most promising feature extraction techniques. Unlike the unsupervised learning scheme of LPP, this paper follows the supervised learning scheme, i.e. it uses both local information and class information to model the similarity of the data. Based on novel similarity, we propose two feature extraction algorithms, supervised optimal locality preserving projection (SOLPP) and normalized Laplacian-based supervised optimal locality preserving projection (NL-SOLPP). Optimal here means that the extracted features via SOLPP (or NL-SOLPP) are statistically uncorrelated and orthogonal. We compare the proposed SOLPP and NL-SOLPP with LPP, orthogonal locality preserving projection (OLPP) and uncorrelated locality preserving projection (ULPP) on publicly available data sets. Experimental results show that the proposed SOLPP and NL-SOLPP achieve much higher recognition accuracy.     

大间距无监督正交特征提取算法

以主成分分析和局部保持投影为理论基础,提出了一种同时考虑数据样本的全局和局部特性的大间距无监督正交特征提取算法,算法的目标函数采用大间距准则,避免了由于矩阵求逆带来的小样本问题,同时为了进一步增强算法的识别性能,对所求取的投影矩阵进行了正交化约束,最后人脸库上的实验结果表明所提方法的有效性.     

Improved discriminant locality preserving projections for face and palmprint recognition

We propose in this paper two improved manifold learning methods called diagonal discriminant locality preserving projections (Dia-DLPP) and weighted two-dimensional discriminant locality preserving projections (W2D-DLPP) for face and palmprint recognition. Motivated by the fact that diagonal images outperform the original images for conventional two-dimensional (2D) subspace learning methods such as 2D principal component analysis (2DPCA) and 2D linear discriminant analysis (2DLDA), we first propose applying diagonal images to a recently proposed 2D discriminant locality preserving projections (2D-DLPP) algorithm, and formulate the Dia-DLPP method for feature extraction of face and palmprint images. Moreover, we show that transforming an image to a diagonal image is equivalent to assigning an appropriate weight to each pixel of the original image to emphasize its different importance for recognition, which provides the rationale and superiority of using diagonal images for 2D subspace learning. Inspired by this finding, we further propose a new discriminant weighted method to explicitly calculate the discriminative score of each pixel within a face and palmprint sample to duly emphasize its different importance, and incorporate it into 2D-DLPP to formulate the W2D-DLPP method to improve the recognition performance of 2D-DLPP and Dia-DLPP. Experimental results on the widely used FERET face and PolyU palmprint databases demonstrate the efficacy of the proposed methods.     

一种基于核的监督流形学习算法

针对流形学习算法——局部保持映射存在的参数选择及不能进行非线性特征提取的问题,提出一种基于核的监督流形学习算法.该算法作为局部保持映射算法的改进算法用样本类标识信息指导建立局部最近邻图,并在建立局部最近邻图使用无参数的相似度量.利用核方法来解决局部保持映射算法在处理线性不可分问题上的局限性问题.在两个常用数据库上验证本文算法的可行性和有效性.     

图最优化线性鉴别投影及其在图像识别中的应用

在图最优化局部保持投影(GoLPP)算法的基础上,本文充分利用数据的类别信息,提出一种新的特征抽取算法——图最优化线性鉴别投影(GoLDP)。与GoLPP类似,GoLDP的邻接图是通过最优化一个目标函数创建的;与GoLPP不同,GoLDP利用数据的类别信息创建两幅最优邻接图——最优内在图和最优惩罚图,由这两幅最优邻接图求得最优投影矩阵。FERET与YALE人脸数据库以及PolyU掌纹数据库上的实验结果证明了GoLDP算法的有效性。     

Local sparse representation projections for face recognition

How to define the sparse affinity weight matrices is still an open problem in existing manifold learning algorithm. In this paper, we propose a novel supervised learning method called local sparse representation projections (LSRP) for linear dimensionality reduction. Differing from sparsity preserving projections (SPP) and the recent manifold learning methods such as locality preserving projections (LPP), LSRP introduces the local sparse representation information into the objective function. Although there are no labels used in the local sparse representation, it still can provide better measure coefficients and significant discriminant abilities. By combining the local interclass neighborhood relationships and sparse representation information, LSRP aims to preserve the local sparse reconstructive relationships of the data and simultaneously maximize the interclass separability. Comprehensive comparison and extensive experiments show that LSRP achieves higher recognition rates than principle component analysis, linear discriminant analysis and the state-of-the-art techniques such as LPP, SPP and maximum variance projections.     

Face recognition via local preserving average neighborhood margin maximization and extreme learning machine

Average neighborhood maximum margin (ANMM) is an effective method for feature extraction in appearance-based face recognition. In this paper, we extend ANMM to locality preserving average neighborhood margin maximization (LPANMM) in order to maintain the local structure on the original data manifold in the discriminant feature space. We also combine LPANMM with extreme learning machine (ELM) as a new scheme for face recognition, we train the single-hidden layer feedforward neural network (SLFN) in the ELM classifier with the discriminant features that are extracted by LPANMM, then we use the trained ELM classifer to classify the test data. In the process of training SLFN, ELM can not only achieve the smallest training error in theory, but is also not sensitive to the initial value selection of the parameters for the SLFN. Experimental results on ORL, Yale, CMU PIE and FERET face databases demonstrate the scheme LPANMM/ELM can achieve better performance than ANMM and other traditional schemes for face recognition.     

无参数无相关最大化判别边界算法

在人脸识别算法中,无参数局部保持投影(PFLPP)是一种有效的特征提取算法, 但忽略了异类近邻样本在分类中所起的作用,并且对于近邻的处理仅利用样本与总体均值的 距离关系来判断,因此并不能有效地确定近邻关系。基于此,提出一种无参数无相关最大化 判别边界算法,有效地利用了样本的类别信息,定义了无参数同类近邻样本的相似权值与异 类近邻样本的惩罚权值,样本邻域大小可根据类内平均余弦距离和类间余弦距离自适应确定, 为了进一步增强算法的性能,给出了具有不相关性的目标函数。UMIST 和 AR 人脸库上的实 验结果表明,该算法相对于不相关保局投影分析算法和 PFLPP 算法,具有运算量低、识别性 能高的优势。     

基于边界判别投影的数据降维

为了提取具有较好判别性能的低维特征,提出了一种新的有监督的线性降维算法——边界判别投影,即,最小化同类样本间的最大距离,最大化异类样本间的最小距离,同时保持数据流形的几何形状.与经典的基于边界定义的算法相比,边界判别投影可以较好地保持数据流形的几何结构和判别结构等全局特性,可避免小样本问题,具有较低的计算复杂度,可应用于超高维的大数据降维.人脸数据集上的实验结果表明,边界判别分析是一种有效的降维算法,可应用于大数据上的特征提取.     

一种用于特征提取的保局判别分析算法

针对保局投影(LPP)为无监督算法的局限,提出了一种新的监督版的LPP,即保局判别分析(LPDA)算法。LPDA吸收了流形学习算法与最大边界准则(MMC)的共同特点,可以将高维的人脸数据投影到低维子空间,具有能处理新样本与无小样本问题的优点。与现有的多种经典相关方法相比,从Yale, UMIST及MIT 3个人脸数据库的实验结果表明,提出的LPDA算法在降维的同时提取了用于人脸识别的更有效的特征,人脸图像识别性能较好,具有较强的判别分析能力。