改进的LDA算法及秩限制问题研究

针对经典线性判别分析中存在的秩限制和小样本问题,通过改进原有的Fisher准则,提出了一种改进的线性判别分析算法ILDA,以克服秩限制问题并同时解决了小样本问题。重点研究了ILDA在解决样本类间离散度矩阵秩限制方面的有效性。在多个国际标准数据集和人工数据集上实验的结果表明ILDA算法不仅有效地突破了秩限制,达到提取更多判别特征的目的,而且具有良好的识别效果。     

改进的线性判别分析算法

线性判别分析是一种有效的特征提取方法,但其存在两个缺陷：小样本问题和秩限制问题。为了解决上述问题,提出一种改进的线性判别分析算法ILDA。该方法引进类间离散度标量和类内离散度标量,通过求解样本各维的权值达到特征提取的目的。若干标准人脸数据集和人工数据集上的实验表明ILDA在特征提取方面的有效性。     

一种改进的线性判别分析算法MLDA

线性判别分析(LDA)是模式识别方法之一,已广泛应用于模式识别、数据分析等诸多领域。线性判别分析法寻找的是有效分类的方向。而当样本维数远大于样本个数(即小样本问题)时,LDA便束手无策。为有效解决线性判别分析法的小样本问题,提出了一种改进的LDA算法——MLDA。该算法将类内离散度矩阵进行标量化处理,有效地避免了对类内离散度矩阵求逆。通过实验证明MLDA在一定程度上解决了经典LDA的小样本问题。     

多阶矩阵组合LDA及其在人脸识别中的应用

线性判别分析（LDA）是一种普遍用于特征提取的线性分类方法。但将LDA直接用于人脸识别会遇到小样本问题和秩限制问题。为了解决以上问题,提出一种基于多阶矩阵组合的LDA算法——MLDA。该算法重新定义了传统LDA中的类内离散度矩阵Sw,使传统Fisher准则具有更好的健壮性和适应性。若干人脸数据库上的比较实验证明了MLDA的有效性。     

利用标准化LDA进行人脸识别

线性判别分析（LDA）是一种较为普遍的用于特征提取的线性分类方法。提出一种基于LDA的人脸识别方法－－标准化LDA，该方法克服了传统LDA方法的缺点，重新定义了样本类间离散度矩阵，在原始定义的基础上增加一个由类间距离决定的可变权函数，使得在选择投地，能够更好地分开各个类的样本；同时，它采用一种合理而有效的方法解决矩阵奇异的问题，即保留样本类内离散度矩阵的零空间，因为这个空间包含了最具有判别能力的信息。在这个零空间里，寻找对应于样本类间离散度矩阵的较大特征值的特征向量作为最后降维的转换矩阵。实验结果显示，在人脸识别中，与传统LDA相比，该方法有更好的识别率。标准化LDA也可以用于其他图像识别问题。     

一种适用于小样本问题的基于边界的特征提取算法

特征提取技术是模式识别领域进行数据降维和强化判别信息的有效方法.线性判别分析是监督特征提取方法的典型代表,获得广泛应用,但受到小样本问题的制约.对此提出一种适用于小样本问题的基于边界的特征提取算法.算法利用高维数据小样本情况下线性可分概率增加以及其低维投影趋于正态分布的特点,定义了新的类别边界,不但考虑了由线性判别分析提出的类内、类间离散度,也兼顾各类别的方差差异性.通过极大化该边界获得最优投影向量,同时避免因类内离散度矩阵奇异导致的小样本问题.进一步将算法推广到多类问题.高光谱数据特征提取与分类实验表明,算法在小样本情况下对于两类和多类问题均具有良好的推广性能,优于多种线性判别分析的改进算法,并且在样本较多时也取得了满意结果.     

基于对称线性判别分析算法的人脸识别

小样本问题的存在使得类内离散度矩阵为奇异阵,因此求解线性判别分析(LDA)算法的广义特征方程存在病态奇异问题。为解决此问题,在已有算法的基础上,引入镜像图像来扩大样本容量,并采用零空间的方法求得Fisher准则函数的最优解。通过在ORL和Yale标准人脸库上的实验结果表明,人脸识别效果优于传统LDA方法、独立成分分析(ICA)方法以及二维对称主成分分析(2DSPCA)方法。     

间距判别投影及其在表情识别中的应用

针对全局降维方法判别信息不足,局部降维方法对邻域关系的判定存在缺陷的问题,提出一种新的基于间距的降维方法——间距判别投影（MDP）。首先,根据类的中心均值的异类近邻关系定义描述类边缘的边界向量;在这个基础上,MDP重新定义类间离散度矩阵,同时,使用全局的方法构造类内离散度矩阵;然后,MDP借鉴判别分析思想建立衡量类间距的准则,并通过类间距最大化增强样本在投影空间中的可分性。对MDP在人脸表情数据库JAFFE和Extended Cohn-Kanade上进行表情识别实验,并且跟传统的降维方法主成分分析（PCA）、最大间距准则（MMC）和边界Fisher分析（MFA）进行对比,实验结果表明,所提算法能够有效提取更具区分性的低维特征,比其他几种方法分类精度更高。     

文本分类中信息增益算法的改进

分析了信息增益方法的不足,并将类内离散度、类间离散度和权重协调因子应用到信息增益算法上,提出了一种改进的信息增益算法.实验表明,该方法在分类效果上与经典算法相比有一定的提高.     

用于人脸识别的两类主成分分析融合

分析了基于总体离散度矩阵和总类间离散度矩阵的主成分分析的原理。利用两种方法分别提取人脸特征并进行识别。对两种方法获得的结果进行了特征层融合和决策层融合,基于ORL人脸数据库的实验表明该方法的识别性能优于单一的主成分分析方法。     

Incremental linear discriminant analysis for face recognition.

Dimensionality reduction methods have been successfully employed for face recognition. Among the various dimensionality reduction algorithms, linear (Fisher) discriminant analysis (LDA) is one of the popular supervised dimensionality reduction methods, and many LDA-based face recognition algorithms/systems have been reported in the last decade. However, the LDA-based face recognition systems suffer from the scalability problem. To overcome this limitation, an incremental approach is a natural solution. The main difficulty in developing the incremental LDA (ILDA) is to handle the inverse of the within-class scatter matrix. In this paper, based on the generalized singular value decomposition LDA (LDA/GSVD), we develop a new ILDA algorithm called GSVD-ILDA. Different from the existing techniques in which the new projection matrix is found in a restricted subspace, the proposed GSVD-ILDA determines the projection matrix in full space. Extensive experiments are performed to compare the proposed GSVD-ILDA with the LDA/GSVD as well as the existing ILDA methods using the face recognition technology face database and the Carneggie Mellon University Pose, Illumination, and Expression face database. Experimental results show that the proposed GSVD-ILDA algorithm gives the same performance as the LDA/GSVD with much smaller computational complexity. The experimental results also show that the proposed GSVD-ILDA gives better classification performance than the other recently proposed ILDA algorithms.     

Incremental two-dimensional linear discriminant analysis with applications to face recognition

Two dimensional linear discriminant analysis (2DLDA) has been verified as an effective method to solve the small sample size (SSS) problem in linear discriminant analysis (LDA). However, most of the existing 2DLDA techniques do not support incremental subspace analysis for updating the discriminant eigenspace. Incremental learning has proven to enable efficient training if large amounts of training data have to be processed or if not all data are available in advance as, for example, in on-line situations. Instead of having to re-training across the entire training data whenever a new sample is added, this paper proposed an incremental two-dimensional linear discriminant analysis (I2DLDA) algorithm with closed-form solution to extract facial features of the appearance image on-line. The proposed I2DLDA inherits the advantages of the 2DLDA and the Incremental LDA (ILDA) and overcomes the number of the classes or chunk size limitation in the ILDA because the size of the between-class scatter matrix and the size of the within-class scatter matrix in the I2DLDA are much smaller than the ones in the ILDA. The results on experiments using the ORL and XM2VTS databases show that the I2DLDA is computationally more efficient than the batch 2DLDA and can achieve better recognition results than the ILDA.     

一种两阶段判别分析方法

为了解决LDA 对复杂分布数据的表达问题,本文提出了一种新的非参数形式的散度矩阵构造方法。该方法 能更好的刻画分类边界信息,并保留更多对分类有用的信息。同时针对小样本问题中非参数结构形式的类内散度矩阵可能奇 异,提出了一种两阶段鉴别分析方法对准则函数进行了最优化求解。该方法通过奇异值分解把人脸图像投影到混合散度矩阵 的主元空间,使类内散度矩阵在投影空间中是非奇异的,通过CS 分解,从理论上分析了同时对角化散度矩阵的求解,并证明了 得到的投影矩阵满足正交约束条件。在ORL,Yale 和YaleB 人脸库上测试的结果显示,改进的算法在性能上优于PCA+LDA, ULDA 和OLDA 等子空间方法。     

一种对称散布矩阵空间的人脸识别方法

为有效解决小样本问题 ,从线性子空间的角度出发 ,构造了一种矩阵变换 ,得到了类内散布矩阵的另一个对称线性子空间 ;通过对两个子空间的分别求解 ,从而得到样本有效的鉴别信息。该方法有效地解决了传统 Fisher鉴别分析方法中的最终特征维数受类别数限制的问题。在 NUST603和 ORL人脸数据库上的实验结果验证了算法的有效性。     

Linear dimensionality reduction using relevance weighted LDA

The linear discriminant analysis (LDA) is one of the most traditional linear dimensionality reduction methods. This paper incorporates the inter-class relationships as relevance weights into the estimation of the overall within-class scatter matrix in order to improve the performance of the basic LDA method and some of its improved variants. We demonstrate that in some specific situations the standard multi-class LDA almost totally fails to find a discriminative subspace if the proposed relevance weights are not incorporated. In order to estimate the relevance weights of individual within-class scatter matrices, we propose several methods of which one employs the evolution strategies.     

An optimal symmetrical null space criterion of Fisher discriminant for feature extraction and recognition

Linear discriminant analysis (LDA) is one of the most effective feature extraction methods in statistical pattern recognition, which extracts the discriminant features by maximizing the so-called Fisher’s criterion that is defined as the ratio of between-class scatter matrix to within-class scatter matrix. However, classification of high-dimensional statistical data is usually not amenable to standard pattern recognition techniques because of an underlying small sample size (SSS) problem. A popular approach to the SSS problem is the removal of non-informative features via subspace-based decomposition techniques. Motivated by this viewpoint, many elaborate subspace decomposition methods including Fisherface, direct LDA (D-LDA), complete PCA plus LDA (C-LDA), random discriminant analysis (RDA) and multilinear discriminant analysis (MDA), etc., have been developed, especially in the context of face recognition. Nevertheless, how to search a set of complete optimal subspaces for discriminant analysis is still a hot topic of research in area of LDA. In this paper, we propose a novel discriminant criterion, called optimal symmetrical null space (OSNS) criterion that can be used to compute the Fisher’s maximal discriminant criterion combined with the minimal one. Meanwhile, by the reformed criterion, the complete symmetrical subspaces based on the within-class and between-class scatter matrices are constructed, respectively. Different from the traditional subspace learning criterion that derives only one principal subspace, in our approach two null subspaces and their orthogonal complements were all obtained through the optimization of OSNS criterion. Therefore, the algorithm based on OSNS has the potential to outperform the traditional LDA algorithms, especially in the cases of small sample size. Experimental results conducted on the ORL, FERET, XM2VTS and NUST603 face image databases demonstrate the effectiveness of the proposed method.