Kernel class-wise locality preserving projection

In the recent years, the pattern recognition community paid more attention to a new kind of feature extraction method, the manifold learning methods, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among them, locality preserving projection (LPP) is one of the most promising feature extraction techniques. However, when LPP is applied to the classification tasks, it shows some limitations, such as the ignorance of the label information. In this paper, we propose a novel local structure based feature extraction method, called class-wise locality preserving projection (CLPP). CLPP utilizes class information to guide the procedure of feature extraction. In CLPP, the local structure of the original data is constructed according to a certain kind of similarity between data points, which takes special consideration of both the local information and the class information. The kernelized (nonlinear) counterpart of this linear feature extractor is also established in the paper. Moreover, a kernel version of CLPP namely Kernel CLPP (KCLPP) is developed through applying the kernel trick to CLPP to increase its performance on nonlinear feature extraction. Experiments on ORL face database and YALE face database are performed to test and evaluate the proposed algorithm.     

Supervised optimal locality preserving projection

In the past few years, the computer vision and pattern recognition community has witnessed a rapid growth of a new kind of feature extraction method, the manifold learning methods, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among these methods, locality preserving projection (LPP) is one of the most promising feature extraction techniques. Unlike the unsupervised learning scheme of LPP, this paper follows the supervised learning scheme, i.e. it uses both local information and class information to model the similarity of the data. Based on novel similarity, we propose two feature extraction algorithms, supervised optimal locality preserving projection (SOLPP) and normalized Laplacian-based supervised optimal locality preserving projection (NL-SOLPP). Optimal here means that the extracted features via SOLPP (or NL-SOLPP) are statistically uncorrelated and orthogonal. We compare the proposed SOLPP and NL-SOLPP with LPP, orthogonal locality preserving projection (OLPP) and uncorrelated locality preserving projection (ULPP) on publicly available data sets. Experimental results show that the proposed SOLPP and NL-SOLPP achieve much higher recognition accuracy.     

Supervised Discriminant Projection with Its Application to Face Recognition

In the past few decades, many face recognition methods have been developed. Among these methods, subspace analysis is an effective approach for face recognition. Unsupervised discriminant projection (UDP) finds an embedding subspace that preserves local structure information, and uncovers and separates embedding corresponding to different manifolds. Though UDP has been applied in many fields, it has limits to solve the classification tasks, such as the ignorance of the class information. Thus, a novel subspace method, called supervised discriminant projection (SDP), is proposed for face recognition in this paper. In our method, the class information was utilized in the procedure of feature extraction. In SDP, the local structure of the original data is constructed according to a certain kind of similarity between data points, which takes special consideration of both the local information and class information. We test the performance of the proposed method SDP on three popular face image databases (i.e. AR database, Yale database, and a subset of FERET database). Experimental results show that the proposed method is effective.     

基于类别信息的监督局部保持投影方法

局部保持投影算法(LPP)是拉普拉斯映射(LE)的线性近似,但LPP作为一种无监督方法,并没有有效利用已有的类别信息提高分类效率。为此提出一种基于类别信息的监督局部保持投影方法(SLPP-LI)。在学习投影矩阵时,SLPP-LI综合利用了流形的几何结构和已有训练点的类别信息,通过调整控制参数的取值,有效地利用已知的低维信息,并且直接求解线性方程获得高维数据的低维模型。通过在多个人脸数据库和手写数字库上的对比实验,表明了SLPP-LI对于高维数据的初始维数以及训练数据的数目并不敏感,〖BP(〗同类问题中与相应的对比算法相比〖BP)〗与主分量分析法(PCA)、LPP、正交LPP(OLPP)、有监督的LPP(SLPP)相比,均具有较高的识别率,充分说明SLPP-LI算法能够有效处理分类问题。     

Learning a locality discriminating projection for classification

This paper introduces a new algorithm called locality discriminating projection (LDP) for subspace learning, which provides a new scheme for discriminant analysis by considering both the manifold structure and the prior class information. In the LDP algorithm, the overlap among the class-specific manifolds is approximated by an invader graph, and a locality discriminant criterion is proposed to find the projections that best preserve the within-class local structures while decrease the between-class overlap. The feasibility of the LDP algorithm has been successfully tested in text data and visual recognition experiments. Experiment results show it is an effective technique for data modeling and classification comparing to linear discriminant analysis, locality preserving projection, and marginal Fisher analysis.     

基于大间距准则的不相关保局投影分析

局部保持投影(Locality preserving projections,LPP)算法只保持了目标在投影后的邻域局部信息,为了更好地刻画数据的流形结构, 引入了类内和类间局部散度矩阵,给出了一种基于有效且稳定的大间距准则(Maximum margin criterion,MMC)的不相关保局投影分析方法.该方法在最大化散度矩阵迹差时,引入尺度因子α,对类内和类间局部散度矩阵进行加权,以便找到更适合分类的子空间并且可避免小样本问题; 更重要的是,大间距准则下提取的判别特征集一般情况下是统计相关的,造成了特征信息的冗余, 因此,通过增加一个不相关约束条件,利用推导出的公式提取不相关判别特征集, 这样做, 对正确识别更为有利.在Yale人脸库、PIE人脸库和MNIST手写数字库上的测试结果表明,本文方法有效且稳定, 与LPP、LDA (Linear discriminant analysis)和LPMIP(Locality-preserved maximum information projection)方法等相比,具有更高的正确识别率.     

Feature extraction using constrained maximum variance mapping

In this paper, an efficient feature extraction method named as constrained maximum variance mapping (CMVM) is developed. The proposed algorithm can be viewed as a linear approximation of multi-manifolds learning based approach, which takes the local geometry and manifold labels into account. The CMVM and the original manifold learning based approaches have a point in common that the locality is preserved. Moreover, the CMVM is globally maximizing the distances between different manifolds. After the local scatters have been characterized, the proposed method focuses on developing a linear transformation that can maximize the dissimilarities between all the manifolds under the constraint of locality preserving. Compared to most of the up-to-date manifold learning based methods, this trick makes contribution to pattern classification from two aspects. On the one hand, the local structure in each manifold is still kept; on the other hand, the discriminant information between manifolds can be explored. Finally, FERET face database, CMU PIE face database and USPS handwriting data are all taken to examine the effectiveness and efficiency of the proposed method. Experimental results validate that the proposed approach is superior to other feature extraction methods, such as linear discriminant analysis (LDA), locality preserving projection (LPP), unsupervised discriminant projection (UDP) and maximum variance projection (MVP).     

一种基于核的监督流形学习算法

针对流形学习算法——局部保持映射存在的参数选择及不能进行非线性特征提取的问题,提出一种基于核的监督流形学习算法.该算法作为局部保持映射算法的改进算法用样本类标识信息指导建立局部最近邻图,并在建立局部最近邻图使用无参数的相似度量.利用核方法来解决局部保持映射算法在处理线性不可分问题上的局限性问题.在两个常用数据库上验证本文算法的可行性和有效性.     

全局判别与局部稀疏保持HSI半监督特征提取

针对高光谱图像存在“维数灾难”的问题，提出一种全局判别与局部稀疏保持的高光谱图像半监督特征提取算法（GLSSFE）。该算法通过LDA算法的散度矩阵保存有类标样本的全局类内判别信息和全局类间判别信息，结合利用半监督PCA算法对有类标和无类标样本进行主成分分析，保存样本的全局结构；利用稀疏表示优化模型自适应揭示样本数据间的非线性结构，将局部类间判别权值和局部类内判别权值嵌入半监督LPP算法保留样本数据的局部结构，从而最大化同类样本的相似性和异类样本的差异性。通过1-NN和SVM两个分类器分别对Indian Pines和Pavia University两个公共高光谱图像数据集进行分类，验证所提特征提取方法的有效性。实验结果表明，该GLSSFE算法最高总体分类精度分别达到89.10%和92.09%，优于现有的特征提取算法，能有效地挖掘高光谱图像的全局特征和局部特征，极大地提升高光谱图像的地物分类效果。     

Locality-preserved maximum information projection.

Dimensionality reduction is usually involved in the domains of artificial intelligence and machine learning. Linear projection of features is of particular interest for dimensionality reduction since it is simple to calculate and analytically analyze. In this paper, we propose an essentially linear projection technique, called locality-preserved maximum information projection (LPMIP), to identify the underlying manifold structure of a data set. LPMIP considers both the within-locality and the between-locality in the processing of manifold learning. Equivalently, the goal of LPMIP is to preserve the local structure while maximize the out-of-locality (global) information of the samples simultaneously. Different from principal component analysis (PCA) that aims to preserve the global information and locality-preserving projections (LPPs) that is in favor of preserving the local structure of the data set, LPMIP seeks a tradeoff between the global and local structures, which is adjusted by a parameter alpha, so as to find a subspace that detects the intrinsic manifold structure for classification tasks. Computationally, by constructing the adjacency matrix, LPMIP is formulated as an eigenvalue problem. LPMIP yields orthogonal basis functions, and completely avoids the singularity problem as it exists in LPP. Further, we develop an efficient and stable LPMIP/QR algorithm for implementing LPMIP, especially, on high-dimensional data set. Theoretical analysis shows that conventional linear projection methods such as (weighted) PCA, maximum margin criterion (MMC), linear discriminant analysis (LDA), and LPP could be derived from the LPMIP framework by setting different graph models and constraints. Extensive experiments on face, digit, and facial expression recognition show the effectiveness of the proposed LPMIP method.     

Locality preserving multimodal discriminative learning for supervised feature selection

Feature selection has been an important preprocessing step in high-dimensional data analysis and pattern recognition. In this paper, we propose a locality preserving multimodal discriminative learning method called LPMDL for supervised feature selection, which arises by solving two standard eigenvalue problems and seeks to find a pair of optimal transformations for two sets of multivariate data in different classes. This topic can optimally discover the local structure information of the given data hided in the original space and aims at structuring an effective low-dimensional embedding space, under which LPMDL keeps nearby data pairs in the same class close and between-class data pairs apart, and the projections of the original data in different classes can be appropriately separated from each other. LPMDL can be performed either in the input space or the reproducing kernel Hilbert space which gives rise to the kernelized version of LPMDL. We also evaluate the feasibility and efficiency of the LPMDL approach by conducting extensive data visualization and classification tasks. Experimental results on a broad range of data sets show LPMDL tends to capture the intrinsic structure characteristics of the samples data due to the effective representation of the points and achieves similar or even better performance than the conventional PCA, NPE, LPP and LFDA methods.     

面向酉子空间的二维判别保局投影的人脸识别*

保局投影算法（LPP）在人脸识别中具有较好的识别性能,但它是一种非监督学习,并且在具体实现时需要把图像转换为向量,破坏了图像的像素结构,这显然不利于模式识别。针对这些问题,提出基于酉子空间的二维判别保局算法,不仅在判别保局算法的基础上增加了类别信息,而且直接在灰度矩阵上进行水平和垂直方向上的二维保局投影。该方法构造酉空间上的复向量后再运用线性判别分析提取特征。在ORL、Yale和XJTU人脸库中验证了算法的正确性和有效性,其识别率比传统的2DLDA和2DLPP等方法提高4～5个百分点。     

Locality preserving and global discriminant projection with prior information

Existing supervised and semi-supervised dimensionality reduction methods utilize training data only with class labels being associated to the data samples for classification. In this paper, we present a new algorithm called locality preserving and global discriminant projection with prior information (LPGDP) for dimensionality reduction and classification, by considering both the manifold structure and the prior information, where the prior information includes not only the class label but also the misclassification of marginal samples. In the LPGDP algorithm, the overlap among the class-specific manifolds is discriminated by a global class graph, and a locality preserving criterion is employed to obtain the projections that best preserve the within-class local structures. The feasibility of the LPGDP algorithm has been evaluated in face recognition, object categorization and handwritten Chinese character recognition experiments. Experiment results show the superior performance of data modeling and classification to other techniques, such as linear discriminant analysis, locality preserving projection, discriminant locality preserving projection and marginal Fisher analysis.     

局部结构保持的鉴别分析方法

局部投影保持LPP(Locality Preserving Projections)是一种局部特征提取算法,它能够有效地保留数据集的局部结构。不相关保局投影鉴别UDLPP(Uncorrelated Discriminant Locality Preserving Projections)在LPP的基础上考虑了类别信息,通过保留类内几何结构并最大化类间距离获得了良好的鉴别性能。结合UDLPP的思想,在UDLPP的基础上提出了一种局部结构保持的鉴别分析方法PCLSP(Pattern Classification based on Local Structure Preserving)。该方法结合了数据集的类别信息以及数据集的局部结构信息,通过最小化类内近邻分离度以及最大化类间近邻分离度来提高鉴别性能,从而进一步反映了数据的局部结构,提高了识别率。通过在ORL(Olivetti-Oracle Research Lab)和YALE两个标准人脸库上实验验证了该算法的有效性。     

Unsupervised double weight graphs based discriminant analysis for dimensionality reduction

ABSTRACT

Dimensionality reduction plays an important role in pattern recognition tasks. Locality preserving projection and neighbourhood preserving embedding are popular unsupervised feature extraction methods, which try to preserve a certain local structure in the low-dimensional subspace. However, only considering the local neighbour information will limit the methods to achieve higher recognition accuracy. In this paper, an unsupervised double weight graphs based discriminant analysis method (uDWG-DA) is proposed. First, uDWG-DA considers both similar and dissimilar relationships among samples by using double weight graphs. In order to explore the dissimilar information, a new partitioning strategy is proposed to divide the data set into different clusters, where samples of different clusters are dissimilar. Then, based on L_2,1 norm, uDWG-DA finds the optimal projection to not only preserve the similar local structure but also increase the separability among different clusters of the data set. Experiments on four hyperspectral images validate the advantage and feasibility of the proposed method compared with other dimensionality reduction methods.     

稀疏近似最近特征空间嵌入标签传播

针对现有的基于图的半监督学习（graph-based semi-supervised learning,简称GSSL）方法存在模型参数敏感和数据空间判别信息不充分等问题,受最近特征空间嵌入和数据稀疏表示思想的启发,提出一种稀疏近似最近特征空间嵌入标签传播算法SANFSP（sparse approximated nearest feature space embedding label propagation）.SANFSP首先利用特征空间嵌入投影点来稀疏表示原始数据;然后,度量原始数据和稀疏近似最近特征空间嵌入投影间的相似性;进而提出稀疏近似最近特征空间嵌入正则化项;最后,基于传统GSSL 方法的标签传播算法,实现数据标签的平滑传播.同时,还将SANFSP 算法简单拓展到out-of-sample 学习.SANFSP 算法在人造和实际数据集（如人脸识别、可视物件识别以及手写数字分类等）上取得了有效的实验结果.     

Globally maximizing, locally minimizing: unsupervised discriminant projection with applications to face and palm biometrics

This paper develops an unsupervised discriminant projection (UDP) technique for dimensionality reduction of high-dimensional data in small sample size cases. UDP can be seen as a linear approximation of a multimanifolds-based learning framework which takes into account both the local and nonlocal quantities. UDP characterizes the local scatter as well as the nonlocal scatter, seeking to find a projection that simultaneously maximizes the nonlocal scatter and minimizes the local scatter. This characteristic makes UDP more intuitive and more powerful than the most up-to-date method, locality preserving projection (LPP), which considers only the local scatter for clustering or classification tasks. The proposed method is applied to face and palm biometrics and is examined using the Yale, FERET, and AR face image databases and the PolyU palmprint database. The experimental results show that UDP consistently outperforms LPP and PCA and outperforms LDA when the training sample size per class is small. This demonstrates that UDP is a good choice for real-world biometrics applications     

基于改进结构保持数据降维方法的故障诊断研究

传统基于核主成分分析(Kernel principal component analysis,KPCA)的数据降维方法在提取有效特征信息时只考虑全局结构保持而未考虑样本间的局部近邻结构保持问题,本文提出一种改进全局结构保持算法的特征提取与降维方法.改进的特征提取与降维方法将流形学习中核局部保持投影(Kernel loc...     

基于全局不相关的多流形学习

为提升人脸识别算法的鲁棒性,减少判别信息的冗余度,提出基于全局不相关的多流形判别学习算法(UFDML)。使用特征空间到特征空间的距离,学习样本局部判别信息,提出全局不相关约束,使提取的判别特征是统计不相关的。在Yale,AR,ORL人脸库上的实验结果表明,与LPP(局部保持投影)、LDA(线性判别分析)、UDP(非监督判别投影)等人脸识别算法相比,所提算法的平均识别率高于其它算法,验证了其有效性。     

一种基于局部保持的隐变量模型

隐变量模型是一类有效的降维方法,但是由非线性核映射建立的隐变量模型不能保持数据空间的局部结构。为了克服这个缺点,文中提出一种保持数据局部结构的隐变量模型。该算法充分利用局部保持映射的保局性质,将局部保持映射的目标函数作为低维空间中数据的先验信息,对高斯过程隐变量中的低维数据进行约束,建立局部保持的隐变量。实验结果表明,相比原有的高斯过程隐变量,文中算法较好地保持数据局部结构的效果。