Optimum and near optimum feature selection for multivariate data

In designing pattern classification systems for a small number of training samples, investigators have quoted that there exists an optimal number of features one can use. This number is a function of the number of training samples and the probability structure of the data. A recursive algorithm is given for selecting a near optimal feature set for classifying a data set consisting of two classes described by two equiprobable multivariate normal densities with a common covariance matrix. We compare this near optimal feature selection algorithm with the optimal exhaustive search algorithm and discuss examples for both algorithms.