改进的LDA算法及秩限制问题研究

针对经典线性判别分析中存在的秩限制和小样本问题,通过改进原有的Fisher准则,提出了一种改进的线性判别分析算法ILDA,以克服秩限制问题并同时解决了小样本问题。重点研究了ILDA在解决样本类间离散度矩阵秩限制方面的有效性。在多个国际标准数据集和人工数据集上实验的结果表明ILDA算法不仅有效地突破了秩限制,达到提取更多判别特征的目的,而且具有良好的识别效果。     

改进的线性判别分析算法

线性判别分析是一种有效的特征提取方法,但其存在两个缺陷：小样本问题和秩限制问题。为了解决上述问题,提出一种改进的线性判别分析算法ILDA。该方法引进类间离散度标量和类内离散度标量,通过求解样本各维的权值达到特征提取的目的。若干标准人脸数据集和人工数据集上的实验表明ILDA在特征提取方面的有效性。     

基于集成学习的多重集典型相关分析方法

特征提取是模式识别中的关键问题之一,对提高系统分类性能具有重要意义。常用的特征提取方法包括主成分分析、线性鉴别分析、典型相关分析等等,多重集典型相关分析是基于传统的典型相关分析基础上发展而来,利用多组（大于2）特征数据集进行特征提取。基于集成学习的多重集典型相关分析的方法（EMCCA）,是通过将样本化分成若干小的样本,形成若干个特征数据集,利用多重集典型相关分析对这组数据集做特征提取,并结合集成学习对样本进行分类。在UCI上的多特征手写体数据集上的实验结果表明：相比于传统的PCA,CCA特征提取方法,多重集典型相关分析具有更优的特征提取效果,结合集成学习后具有更好的分类效果。     

一种改进的线性判别分析算法在人脸识别中的应用

线性判别分析算法是一种经典的特征提取方法,但其仅在大样本情况下适用。本文针对传统线性判别分析算法面临的小样本问题和秩限制问题,提出了一种改进的线性判别分析算法ILDA。该方法在矩阵指数的基础上,重新定义了类内离散度矩阵和类间离散度矩阵,有效地同时提取类内离散度矩阵零空间和非零空间中的信息。若干人脸数据库上的比较实验表明了ILDA在人脸识别方面的有效性。     

Incremental linear discriminant analysis for face recognition.

Dimensionality reduction methods have been successfully employed for face recognition. Among the various dimensionality reduction algorithms, linear (Fisher) discriminant analysis (LDA) is one of the popular supervised dimensionality reduction methods, and many LDA-based face recognition algorithms/systems have been reported in the last decade. However, the LDA-based face recognition systems suffer from the scalability problem. To overcome this limitation, an incremental approach is a natural solution. The main difficulty in developing the incremental LDA (ILDA) is to handle the inverse of the within-class scatter matrix. In this paper, based on the generalized singular value decomposition LDA (LDA/GSVD), we develop a new ILDA algorithm called GSVD-ILDA. Different from the existing techniques in which the new projection matrix is found in a restricted subspace, the proposed GSVD-ILDA determines the projection matrix in full space. Extensive experiments are performed to compare the proposed GSVD-ILDA with the LDA/GSVD as well as the existing ILDA methods using the face recognition technology face database and the Carneggie Mellon University Pose, Illumination, and Expression face database. Experimental results show that the proposed GSVD-ILDA algorithm gives the same performance as the LDA/GSVD with much smaller computational complexity. The experimental results also show that the proposed GSVD-ILDA gives better classification performance than the other recently proposed ILDA algorithms.     

Discriminative features for text document classification

Abstract The bag-of-words approach to text document representation typically results in vectors of the order of 5000–20,000 components as the representation of documents. To make effective use of various statistical classifiers, it may be necessary to reduce the dimensionality of this representation. We point out deficiencies in class discrimination of two popular such methods, Latent Semantic Indexing (LSI), and sequential feature selection according to some relevant criterion. As a remedy, we suggest feature transforms based on Linear Discriminant Analysis (LDA). Since LDA requires operating both with large and dense matrices, we propose an efficient intermediate dimension reduction step using either a random transform or LSI. We report good classification results with the combined feature transform on a subset of the Reuters-21578 database. Drastic reduction of the feature vector dimensionality from 5000 to 12 actually improves the classification performance.An erratum to this article can be found at     

Facial Feature Extraction Method Based on Coefficients of Variances

Principal Component Analysis （PCA） and Linear Discriminant Analysis （LDA） are two popular feature extraction techniques in statistical pattern recognition field. Due to small sample size problem LDA cannot be directly applied to appearance-based face recognition tasks. As a consequence, a lot of LDA-based facial feature extraction techniques are proposed to deal with the problem one after the other. Nullspace Method is one of the most effective methods among them. The Nullspace Method tries to find a set of discriminant vectors which maximize the between-class scatter in the null space of the within-class scatter matrix. The calculation of its discriminant vectors will involve performing singular value decomposition on a high-dimensional matrix. It is generally memory- and time-consuming. Borrowing the key idea in Nullspace method and the concept of coefficient of variance in statistical analysis we present a novel facial feature extraction method, i.e., Discriminant based on Coefficient of Variance （DCV） in this paper. Experimental results performed on the FERET and AR face image databases demonstrate that DCV is a promising technique in comparison with Eigenfaces, Nullspace Method, and other state-of-the-art facial feature extraction methods.     

Automatic hepatitis diagnosis system based on Linear Discriminant Analysis and Adaptive Network based on Fuzzy Inference System

In this paper, an automatic diagnosis system based on Linear Discriminant Analysis (LDA) and Adaptive Network based on Fuzzy Inference System (ANFIS) for hepatitis diseases is introduced. This automatic diagnosis system deals with the combination of feature extraction and classification. This automatic hepatitis diagnosis system has two stages, which feature extraction – reduction and classification stages. In the feature extraction – reduction stage, the hepatitis features were obtained from UCI Repository of Machine Learning Databases. Then, the number of these features was reduced to 8 from 19 by using Linear Discriminant Analysis (LDA). In the classification stage, these reduced features are given to inputs ANFIS classifier. The correct diagnosis performance of the LDA-ANFIS automatic diagnosis system for hepatitis disease is estimated by using classification accuracy, sensitivity and specificity analysis, respectively. The classification accuracy of this LDA-ANFIS automatic diagnosis system for the diagnosis of hepatitis disease was obtained in about 94.16%.     

基于Gabor变换和Zernike矩的人脸特征提取方法

针对目前常用的三种人脸特征提取方法中存在的识别率低、抗噪性较弱的问题,提出一种基于Gabor变换和Zernike矩的人脸特征提取方法.该方法首先对人脸进行多分辨的Gabor变换,然后利用Zernike矩获得具有平移、尺度、旋转不变性的特征,并用线性判别分析(LDA)方法进一步进行特征选择,最后采用K最近邻分类方法进行人脸的识别.实验结果表明,在与常用的三种人脸特征提取方法的比较中,该方法具有更高的识别率和更强的抗噪性能.     

An automatic diabetes diagnosis system based on LDA-Wavelet Support Vector Machine Classifier

In this paper, an automatic diagnosis system for diabetes on Linear Discriminant Analysis (LDA) and Morlet Wavelet Support Vector Machine Classifier: LDA–MWSVM is introduced. The structure of this automatic system based on LDA-MWSVM for the diagnosis of diabetes is composed of three stages: The feature extraction and feature reduction stage by using the Linear Discriminant Analysis (LDA) method and the classification stage by using Morlet Wavelet Support Vector Machine (MWSVM) classifier stage. The Linear Discriminant Analysis (LDA) is used to separate features variables between healthy and patient (diabetes) data in the first stage. The healthy and patient (diabetes) features obtained in the first stage are given to inputs of the MWSVM classifier in the second stage. Finally, in the third stage, the correct diagnosis performance of this automatic system based on LDA–MWSVM for the diagnosis of diabetes is calculated by using sensitivity and specificity analysis, classification accuracy, and confusion matrix, respectively. The classification accuracy of this system was obtained at about 89.74%.     

A scalable supervised algorithm for dimensionality reduction on streaming data

Algorithms on streaming data have attracted increasing attention in the past decade. Among them, dimensionality reduction algorithms are greatly interesting due to the desirability of real tasks. Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are two of the most widely used dimensionality reduction approaches. However, PCA is not optimal for general classification problems because it is unsupervised and ignores valuable label information for classification. On the other hand, the performance of LDA is degraded when encountering limited available low-dimensional spaces and singularity problem. Recently, Maximum Margin Criterion (MMC) was proposed to overcome the shortcomings of PCA and LDA. Nevertheless, the original MMC algorithm could not satisfy the streaming data model to handle large-scale high-dimensional data set. Thus an effective, efficient and scalable approach is needed. In this paper, we propose a supervised incremental dimensionality reduction algorithm and its extension to infer adaptive low-dimensional spaces by optimizing the maximum margin criterion. Experimental results on a synthetic dataset and real datasets demonstrate the superior performance of our proposed algorithm on streaming data.     

Kernel Self-optimized Locality Preserving Discriminant Analysis for feature extraction and recognition

We propose Kernel Self-optimized Locality Preserving Discriminant Analysis (KSLPDA) for feature extraction and recognition. The procedure of KSLPDA is divided into two stages, i.e., one is to solve the optimal expansion of the data-dependent kernel with the proposed kernel self-optimization method, and the second is to seek the optimal projection matrix for dimensionality reduction. Since the optimal parameters of data-dependent kernel are achieved automatically through solving the constraint optimization equation, based on maximum margin criterion and Fisher criterion in the empirical feature space, KSLPDA works well on feature extraction for classification. The comparative experiments show that KSLPDA outperforms PCA, LDA, LPP, supervised LPP and kernel supervised LPP.     

一种适用于小样本问题的基于边界的特征提取算法

特征提取技术是模式识别领域进行数据降维和强化判别信息的有效方法.线性判别分析是监督特征提取方法的典型代表,获得广泛应用,但受到小样本问题的制约.对此提出一种适用于小样本问题的基于边界的特征提取算法.算法利用高维数据小样本情况下线性可分概率增加以及其低维投影趋于正态分布的特点,定义了新的类别边界,不但考虑了由线性判别分析提出的类内、类间离散度,也兼顾各类别的方差差异性.通过极大化该边界获得最优投影向量,同时避免因类内离散度矩阵奇异导致的小样本问题.进一步将算法推广到多类问题.高光谱数据特征提取与分类实验表明,算法在小样本情况下对于两类和多类问题均具有良好的推广性能,优于多种线性判别分析的改进算法,并且在样本较多时也取得了满意结果.     

Optimized discriminative transformations for speech features based on minimum classification error

Feature extraction is an important component of pattern classification and speech recognition. Extracted features should discriminate classes from each other while being robust to environmental conditions such as noise. For this purpose, several feature transformations are proposed which can be divided into two main categories: data-dependent transformation and classifier-dependent transformation. The drawback of data-dependent transformation is that its optimization criteria are different from the measure of classification error which can potentially degrade the classifier’s performance. In this paper, we propose a framework to optimize data-dependent feature transformations such as PCA (Principal Component Analysis), LDA (Linear Discriminant Analysis) and HLDA (Heteroscedastic LDA) using minimum classification error (MCE) as the main objective. The classifier itself is based on Hidden Markov Model (HMM). In our proposed HMM minimum classification error technique, the transformation matrices are modified to minimize the classification error for the mapped features, and the dimension of the feature vector is not changed. To evaluate the proposed methods, we conducted several experiments on the TIMIT phone recognition and the Aurora2 isolated word recognition tasks. The experimental results show that the proposed methods improve performance of PCA, LDA and HLDA transformation for mapping Mel-frequency cepstral coefficients (MFCC).     

一种基于ICA和LDA组合的人脸识别新方法

特征提取是模式识别研究领域的一个热点。本文提出了一种基于独立成分分析和线性鉴别分析的特征提取方法。该方法中引入了零空间的概念，指出了前人算法中的不足之处，并且给出了一个完整的独立成分分析和线性鉴别分析的组合算法。在ORL和Yale人脸数据库上的实验表明了该方法的有效性。     

Wrapper–Filter Feature Selection Algorithm Using a Memetic Framework

This correspondence presents a novel hybrid wrapper and filter feature selection algorithm for a classification problem using a memetic framework. It incorporates a filter ranking method in the traditional genetic algorithm to improve classification performance and accelerate the search in identifying the core feature subsets. Particularly, the method adds or deletes a feature from a candidate feature subset based on the univariate feature ranking information. This empirical study on commonly used data sets from the University of California, Irvine repository and microarray data sets shows that the proposed method outperforms existing methods in terms of classification accuracy, number of selected features, and computational efficiency. Furthermore, we investigate several major issues of memetic algorithm (MA) to identify a good balance between local search and genetic search so as to maximize search quality and efficiency in the hybrid filter and wrapper MA     

基于KDDA的人脸识别研究

由于PCA 和LDA算法存在小样本问题(Smell Sample Size),结合D-LDA 和Kernel,将线性不可分的低维空间映射到高维空间,并借助于"kernel 技巧"克服了维度灾难问题,并且充分的利用曾经被抛弃的有用信息Null-Space.经过才ORL人脸库的实验表明,此方法比PCA,LDA提高了人脸识别的可分性,并有效地解决了小样本问题.     

Wrapper-filter feature selection algorithm using a memetic framework.

This correspondence presents a novel hybrid wrapper and filter feature selection algorithm for a classification problem using a memetic framework. It incorporates a filter ranking method in the traditional genetic algorithm to improve classification performance and accelerate the search in identifying the core feature subsets. Particularly, the method adds or deletes a feature from a candidate feature subset based on the univariate feature ranking information. This empirical study on commonly used data sets from the University of California, Irvine repository and microarray data sets shows that the proposed method outperforms existing methods in terms of classification accuracy, number of selected features, and computational efficiency. Furthermore, we investigate several major issues of memetic algorithm (MA) to identify a good balance between local search and genetic search so as to maximize search quality and efficiency in the hybrid filter and wrapper MA.     

ECG biometric authentication based on non-fiducial approach using kernel methods

Identity recognition faces several challenges especially in extracting an individual's unique features from biometric modalities and pattern classifications. Electrocardiogram (ECG) waveforms, for instance, have unique identity properties for human recognition, and their signals are not periodic. At present, in order to generate a significant ECG feature set, non-fiducial methodologies based on an autocorrelation (AC) in conjunction with linear dimension reduction methods are used. This paper proposes a new non-fiducial framework for ECG biometric verification using kernel methods to reduce both high autocorrelation vectors' dimensionality and recognition system after denoising signals of 52 subjects with Discrete Wavelet Transform (DWT). The effects of different dimensionality reduction techniques for use in feature extraction were investigated to evaluate verification performance rates of a multi-class Support Vector Machine (SVM) with the One-Against-All (OAA) approach. The experimental results demonstrated higher test recognition rates of Gaussian OAA SVMs on random unknown ECG data sets with the use of the Kernel Principal Component Analysis (KPCA) as compared to the use of the Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA).     

综合颜色和Contourlet直方图的图像检索方法

为提高图像检索性能,使用Harris彩色点提取器提取颜色特征点,设计一种基于颜色特征点的环形颜色直方图,在对图像进行Contourlet变换的基础上,提出Contourlet直方图的概念,改进其旋转不变性,并提取图像的纹理信息。仿真实验结果表明,该方法能够快速准确地检索彩色图像。