一种新的有监督流形学习方法

提出了一种新的有监督流形学习方法,目的是提供将流形学习降维方法高效应用于有监督学习问题的全新策略.算法的核心思想是集成流形学习方法对高维流形结构数据的降维有效性与支撑向量机(SVM)在中小规模分类数据集上的优良特性实现高效有监督流形学习.算法具体实现步骤为:首先利用SVM在流形学习降维数据中选出对分类决策最重要的数据集,即支撑向量集;按标号返回可得到原空间的支撑向量集;在这个集合上再次使用SVM即可得到原空间的分类决策,从而完成有监督流形学习.在一系列人工与实际数据集上的实验验证了方法的有效性.     

基于局部线性空间划分的流形泛化算法

为了将流形学习算法获取的映射关系扩展到新的样本数据,提出一种基于局部线性空间划分的流形泛化算法.提出局部线性空间划分的局部性、曲率自适应性原则.在此基础上,构建定维投影距离测度,采用定维投影向量量化算法将整个流形划分为若干个局部线性空间.在局部空间上构建流形映射的线性近似映射.在流形映射重构的基础上,针对新样本数据,判断其局部线性空间的归属,进而采用线性近似映射获取低维空间上的映射估计值.在人工合成数据集以及手写数字图像库上的实验证明本文算法的有效性.     

基于黎曼流形稀疏编码的图像检索算法

针对视觉词袋（Bag-of-visual-words,BOVW）模型直方图量化误差大的缺点,提出基于稀疏编码的图像检索算法.由于大多数图像特征属于非线性流形结构,传统稀疏编码使用向量空间对其度量必然导致不准确的稀疏表示.考虑到图像特征空间的流形结构,选择对称正定矩阵作为特征描述子,构建黎曼流形空间.利用核技术将黎曼流形结构映射到再生核希尔伯特空间,非线性流形转换为线性稀疏编码,获得图像更准确的稀疏表示.实验在Corel1000和Caltech101两个数据集上进行,与已有的图像检索算法对比,提出的图像检索算法不仅提高了检索准确率,而且获得了更好的检索性能.     

面向弱匹配的跨媒异构迁移学习

针对面向弱匹配的跨媒异构迁移学习中存在的迁移学习性能不高的问题,提出了一种基于平衡异构距离的混合拉普拉斯特征映射的跨媒异构迁移学习方法.利用大量非成对数据和相对少量的成对数据蕴含的语义信息,获取不同媒体域原始特征空间到潜在公共特征空间的映射矩阵;并在跨媒异构迁移学习中,构建混合图拉普拉斯矩阵,不仅保持了同一域下样本间的流形结构,而且保持不同域下样本间的流形结构;提升训练获得的模型在跨媒异构目标域的分类预测性能.在2个公共数据集NUS-WIDE和LabelMe上进行实验,表明了在成对数据的基础上,利用大量非成对数据可以增加模型的准确率和鲁棒性.     

基于子空间流形的图像集识别方法

近年来,基于黎曼流形将图像集在线性子空间中进行表征的图像集识别方法已经被证实有良好的效果,针对该领域存在的图像线性子空间大多高维所导致现有黎曼流形的图像集识别方法存在计算成本高、适用性有限的问题,提出一种基于子空间流形的图像集识别方法。首先,从线性子空间的几何结构出发,利用Grassmann流形对线性子空间进行建模,得到基于Grassmann流形的联合黎曼度量。然后,通过该联合黎曼度量,从高维的Grassmann流形中学习到一个低维的Grassmann流形。最后,对通过学习得到的低维流形上的图像集数据进行图像集识别。实验结果表明,在ETH-80数据集上该方法的识别准确率比投影度量学习（PML）和图嵌入Grassmann判别分析（GGDA）都分别提升了2.5个百分点。证明了在通过提出的度量与方法学习到的低维流形上,图像集数据具有更好的分类结构,从而降低图像集识别计算成本,扩大适用范围,提升识别准确率。     

流形学习中的算法研究

详细介绍了一种新的机器学习的方法--流形学习.流形学习是一种新的非监督学习方法,可以有效地发现高维非线性数据集的内在维数并进行维数约简,近年来越来越受到机器学习和认知科学领域的研究者的重视.目前已经出现了很多有效的流形学习算法,如等度规映射(ISOMAP)、局部线性嵌套(Locally Linear Embedding ,LLE)等.详细讲述了当前常用的几种流形学习算法以及在流形方面已经取得的研究成果,并对流形学习目前在各方面的应用作了较为细致的阐述.最后展望了流形学习的研究发展趋势,且提出了流形学习中仍需解决的关键问题.     

一种结构自适应径向基函数神经网络

提出了一种新的结构自适应的径向基函数(RBF)神经网络模型。在该网络中,自组织映射(SOM)神经网络作为聚类网络,采用无监督学习算法对输入样本进行自组织分类,并将分类中心及其对应的权值向量传递给RBF神经网络,作为径向基函数的中心和相应的权值向量;RBF神经网络作为基础网络,采用高斯函数实现输入层到隐层的非线性映射,输出层则采用有监督学习算法训练网络的权值,从而实现输入层到输出层的非线性映射。通过对字母数据集进行仿真,表明该网络具有较好的性能。     

流形学习与非线性回归结合的头部姿态估计

流形学习的目的是发现非线性数据的内在结构,可用于非线性降维。广义回归网络是人工神经网络的一种,可用于非线性回归。基于流形学习和非线性回归,提出了用于解决头部姿态估计的ManiNLR方法。该方法首先用流形学习对图像数据进行降维,然后用非线性回归的方法将数据映射到线性可分空间,利用非线性回归的结果对人脸的头部姿态进行估计。实验结果表明,ManiNLR算法能够较好地估计图像中的头部姿态,并具有较快的速度和较高的鲁棒性。     

一种基于LS-SVM的特征提取新方法及其在智能质量控制中的应用

提出一种基于最小二乘支持向量机(LS-SVM)的特征提取新方法，并将其成功应用到智能质量控制领域。首先，将线性特征提取公式表达成与LS-SVM回归算法中相同的形式；再遵循SVM方法将数据集由原输入空间映射到高维特征空间，进而使用该技巧通过线性形式实现非线性特征提取。然后，用常规控制图提取出一个含有6种模式、50维特征的仿真数据集用于测试，通过LS-SVM特征提取后，原数据集的特征被降到了3维并保留了原80%的分类信息。最后，用BP分类器对特征提取后的样本进行识别，其结果优于新型RSFM网络直接对原始样本进行识别的效果。仿真实验结果表明了LS-SVM特征提取方法的可行性和有效性。     

基于CNN和SVM的报文入侵检测方法

为了进一步提高网络异常检测的准确率, 本文在对现有入侵检测模型分析的基础上, 提出了一种基于卷积神经网络和支持向量机的网络报文入侵检测方法. 该方法首先将数据预处理成二维矩阵, 为了防止算法模型过拟合, 利用permutation函数将数据随机打乱, 然后利用卷积神经网络CNN从预处理后的数据中学习有效特征, 最后通过支持向量机SVM分类器将得到的向量进行分类处理. 在数据集选择上, 采用网络入侵检测常用的权威数据集—京都大学蜜罐系统数据集, 通过与GRU-Softmax、GRU-SVM等现有检测率较高的模型进行实验对比, 该模型在准确率上最高分别提高了19.39% 和12.83%, 进一步提升了网络异常检测的准确度. 同时, 本研究所提出方法在训练速度和测试速度上有较大提高.     

The self-organizing field [Kohonen maps]

Many of the properties of the well-known Kohonen map algorithm are not easily derivable from its discrete formulation. For instance, the "projection" implemented by the map from a high dimensional input space to a lower dimensional map space must be properly regarded as a projection from a smooth manifold to a lattice and, in this framework, some of its properties are not easily identified. This paper describes the self-organizing field: a continuous embedding of a smooth manifold (the map) into another (the input manifold) that implements a topological map by self-organization. The adaptation of the self-organizing field is governed by a set of differential equations analogous to the difference equations that determine weights updates in the Kohonen map. This paper derives several properties of the self-organizing field, and shows that the emergence of certain structures on the brain-like the columnar organization in the primary visual cortex-arise naturally in the new model.     

Face Matching in Large Database by Self-Organizing Maps

A novel self-organizing map (SOM) based retrieval system is proposed for performing face matching in large database. The proposed system provides a small subset of faces that are most similar to a given query face, from which user can easily verify the matched images. The architecture of the proposed system consists of two major parts. First, the system provides a generalized integration of multiple feature-sets using multiple self-organizing maps. Multiple feature-sets are obtained from different feature extraction methods like Gabor filter, Local Autocorrelation Coefficients, etc. In this platform, multiple facial features are integrated to form a compressed feature vector without concerning scaling and length of individual feature set. Second, an SOM is trained to organize all the face images in a database through using the compressed feature vector. Using the organized map, similar faces to a query can be efficiently identified. Furthermore, the system includes a relevance feedback to enhance the face retrieval performance. The proposed method is computationally efficient. Comparative results show that the proposed approach is promising for identifying face in a given large image database.     

Nonlinear discriminant clustering based on spectral regularization

Owing to sparseness, directly clustering high-dimensional data is still a challenge problem. Therefore, obtaining their low-dimensional compact representation by dimensional reduction is an effective method for clustering high-dimensional data. Most of existing dimensionality reduction methods, however, are developed originally for classification (such as Linear Discriminant Analysis) or recovering the geometric structure (known as manifold) of high-dimensional data (such as Locally Linear Embedding) rather than clustering purpose. Hence, a novel nonlinear discriminant clustering by dimensional reduction based on spectral regularization is proposed. The contributions of the proposed method are two folds: (1) it can obtain nonlinear low-dimensional representation that can recover the intrinsic manifold structure as well as enhance the cluster structure of the original high-dimensional data; (2) the clustering results can also be obtained in the dimensionality reduction procedure. Firstly, the desired low-dimensional coordinates are represented as linear combinations of predefined smooth vectors with respect to the data manifold, which are characterized by a weighted graph. Then, the optimal combination coefficients and the optimal cluster assignment matrix are computed by maximizing the ratio between the between-cluster scatter and the total scatter simultaneously as well as preserving the smoothness of the cluster assignment matrix with respect to the data manifold. Finally, the optimization problem is solved in an iterative procedure, which is proved to be convergent. Experiments on UCI data sets and real world data sets demonstrated the effectiveness of the proposed method for both clustering and visualization high-dimensional data set.     

基于重叠片排列的流形学习算法

针对线性数据降维算法对处理非线性结构数据的降维效果不是很好,提出一种基于重叠片排列的流形学习算法,该算法根据局部的线性贴片处在非线性流形中的特性,将流形划分为线性互相重叠的局部区域贴片,且利用主成分分析方法得到局部区域贴片的低维表示,然后排列且对齐其低维坐标,以获得整体数据的低维坐标.通过仿真结果证明,基于重叠片排列的流形学习算法在应用于人脸识别和分类问题时以及在识别准确率方面要优于其他经典的流形学习算法.     

Adaptive nonlinear manifolds and their applications to pattern recognition

Dimensionality reduction has long been associated with retinotopic mapping for understanding cortical maps. Multisensory information is processed, fused and mapped to an essentially 2-D cortex in an information preserving manner. Data processing and projection techniques inspired by this biological mechanism are playing an increasingly important role in pattern recognition, computational intelligence, data mining, information retrieval and image recognition. Dimensionality reduction involves reduction of features or volume of data and has become an essential step of information processing in many fields. The topic of manifold learning has recently attracted a great deal of attention, and a number of advanced techniques for extracting nonlinear manifolds and reducing data dimensions have been proposed from statistics, geometry theory and adaptive neural networks. This paper provides an overview of this challenging and emerging topic and discusses various recent methods such as self-organizing map (SOM), kernel PCA, principal manifold, isomap, local linear embedding, and Laplacian eigenmap. Many of them can be considered in a learning manifold framework. The paper further elaborates on the biologically inspired SOM model and its metric preserving variant ViSOM under the framework of adaptive manifold; and their applications in dimensionality reduction with face recognition are investigated. The experiments demonstrate that adaptive ViSOM-based methods produce markedly improved performance over the others due to their metric scaling and preserving properties along the nonlinear manifold.     

黎曼流形上的保局投影在图像集匹配中的应用

目的提出了黎曼流形上局部结构特征保持的图像集匹配方法。方法该方法使用协方差矩阵建模图像集合,利用对称正定的非奇异协方差矩阵构成黎曼流形上的子空间,将图像集的匹配转化为流形上的点的匹配问题。通过基于协方差矩阵度量学习的核函数将黎曼流形上的协方差矩阵映射到欧几里德空间。不同于其他方法黎曼流形上的鉴别分析方法,考虑到样本分布的局部几何结构,引入了黎曼流形上局部保持的图像集鉴别分析方法,保持样本分布的局部邻域结构的同时提升样本的可分性。结果在基于图像集合的对象识别任务上测试了本文算法,在ETH80和YouTube Celebrities数据库分别进行了对象识别和人脸识别实验,分别达到91.5%和65.31%的识别率。结论实验结果表明,该方法取得了优于其他图像集匹配算法的效果。     

Supervised learning based on the self-organizing maps for forward kinematic modeling of Stewart platform

In this study, we propose an alternative technique for solving the forward kinematic problem of parallel manipulator which is designed based on generalized Stewart platform. The focus of this work is to predict a pose vector of a moving plate from a given set of six leg lengths. Since the data of parallel kinematics are usually available in the form of nonlinear dynamic system, several methods of system identification have been proposed in order to construct the forward kinematic model and approximate the pose vectors. Although these methods based on a multilayer perceptron (MLP) neural network provide acceptable results, MLP training suffers from convergence to local optima. Thus, we propose to use an alternative supervised learning algorithm called vector-quantized temporal associative memory (VQTAM) instead of MLP-based methods. VQTAM relying on self-organizing map architecture is used to build the mapping from the input space to the output space such that the training/testing datasets are generated from inverse kinematic model. The solutions from standard VQTAM are improved by an autoregressive (AR) model and locally linear embedding (LLE). The experimental results indicate that VQTAM with AR/LLE gives the outputs with nearly 100% prediction accuracy in the case of smooth data, while VQTAM + LLE provides the most accurate prediction on noisy data. Therefore, VQTAM + LLE is a very robust estimation method and can practically be used for solving the forward kinematic problem.

     

图正则化迁移稀疏概念编码的跨域图像分类

为克服不同图像域之间的特征“差异”，跨越分布“鸿沟”，提出了一种基于正则化迁移稀疏概念编码的跨域图像分类方法。将图像域间的分布差异性和标签相关性信息融入稀疏编码模型中，以学习跨域图像的鲁棒性稀疏表示，从高维的图像特征空间中挖掘图像低维流形结构，形成基向量集，构造跨域图像的迁移稀疏概念编码。该方法挖掘不同图像域之间的共同特征表达，实现了图像标签的跨域迁移。通过在多个图像数据库中的比较实验表明，该方法获得更为鲁棒的图像特征表达，其分类性能显著优于其他相关比较方法。     

Locally linear discriminant embedding: An efficient method for face recognition

In this paper an efficient feature extraction method named as locally linear discriminant embedding (LLDE) is proposed for face recognition. It is well known that a point can be linearly reconstructed by its neighbors and the reconstruction weights are under the sum-to-one constraint in the classical locally linear embedding (LLE). So the constrained weights obey an important symmetry: for any particular data point, they are invariant to rotations, rescalings and translations. The latter two are introduced to the proposed method to strengthen the classification ability of the original LLE. The data with different class labels are translated by the corresponding vectors and those belonging to the same class are translated by the same vector. In order to cluster the data with the same label closer, they are also rescaled to some extent. So after translation and rescaling, the discriminability of the data will be improved significantly. The proposed method is compared with some related feature extraction methods such as maximum margin criterion (MMC), as well as other supervised manifold learning-based approaches, for example ensemble unified LLE and linear discriminant analysis (En-ULLELDA), locally linear discriminant analysis (LLDA). Experimental results on Yale and CMU PIE face databases convince us that the proposed method provides a better representation of the class information and obtains much higher recognition accuracies.