基于流形学习的多示例回归算法

多示例学习是一种新型机器学习框架，以往的研究主要集中在多示例分类上，最近多示例回归受到了国际机器学习界的关注．流形学习旨在获得非线性分布数据的内在结构，可以用于非线性降维．文中基于流形学习技术，提出了用于解决多示例同归问题的Mani MIL算法．该算法首先对训练包中的示例降维，利用降维结果出现坍缩的特性对多示例包进行预测．实验表明，Mani MIL算法比现有的多示例算法例如Citation-kNN等有更好的性能．     

基于核融合的多信息流形学习算法*

流形学习算法可分为全局流形学习与局部流形学习,它们分别保持了流形上的全局特征信息与局部特征信息。但是实验证明仅基于单一特征信息的流形学习算法不能很好的保持真实的流形结构,影响了学习效果。因此,基于流形学习的核的视角,将全局流形学习算法ISOMAP与局部流形学习算法LTSA的核进行融合,提出了可以同时保持流形结构的全局特征信息与局部特征信息的流形学习算法,在人工数据集和人脸图像集上的仿真实验证明了本文算法的有效性。     

基于放大因子和延伸方向研究流形学习算法

流形学习是一种新的非监督学习方法，可以有效地发现高维非线性数据集的内在维数和进行维数约简，近年来越来越受到机器学习和认知科学领域研究者的重视．虽然目前已经出现了很多有效的流形学习算法，如等度规映射（ISOMAP）、局部线性嵌套（Locally Linear Embedding，LLE）等，然而，对观测空间的高维数据与降维后的低维数据之间的定量关系，尚难以直观地进行分析．这一方面不利于对数据内在规律的深入探察，一方面也不利于对不同流形学习算法的降维效果进行直观比较．文中提出了一种方法，可以从放大因子和延伸方向这两个方面显示出观测空间的高维数据与降维后的低维数据之间的联系；比较了两种著名的流形学习算法（ISOMAP和LLE）的性能，得出了一些有意义的结论；提出了相应的算法从而实现了以上理论．对几组数据的实验表明了研究的有效性和意义．     

机车控制电源故障特征向量维数约简方法研究

通过对电力机车控制系统中控制电源的电路结构特点分析,进行故障特征值提取,对所造成的高维数据无法进行模式识别这一特点。通过对传统的流形学习算法中LE理论进行改进,提出了基于马氏距离的LE算法理论,对LE算法中的邻域选择问题进行了深入的研究,使K具有自适应性,而且利用关联维数理论克服了非线性电路的故障特征提取中所造成的维数灾难,对其高维数据进行本征维数的估计,去除了不相关的信息维数,解决了流形学习理论中d的选取难点。最后通过验证得出该方法的有效性与准确性。 关键词：移相全桥变换器;流形学习;数据降维;马氏距离;关联维数;     

一种在源数据稀疏情况下的数据降维算法

流形学习方法是根据流形的定义提出的一种非线性数据降维方法,主要思想是发现嵌入在高维数据空间的低维光滑流形。从分析基于流形学习理论的局部线性嵌入算法入手,针对传统的局部线性嵌入算法在源数据稀疏时会失效的缺点,提出了基于局部线性逼近思想的流形学习算法,并在S-曲线上采样测试取得良好降维效果。     

流形学习与基于线性耦合映射的流形对齐

从一些具有代表性的经典流形学习方法的回顾来看,传统的流形学习主要处理来自单一流形的数据的降维问题.随着流形学习研究的不断深入,以多流形作为研究对象的流形学习问题逐步引起了研究者的注意.提出了一种基于线性耦合映射的流形对齐算法.算法克服非线性流形对齐算法不能够直接处理Out-of-sample数据的问题.同时,与已有的线性流形对齐算法相比,该算法不需要假设流形间满足仿射变换关系,因而能够更加灵活地处理一些比较实际的流形对齐问题.     

基于推广流形学习的高分辨遥感影像目标分类

针对传统的流形学习算法不能对位于黎曼流形上的协方差描述子进行有效降维这一问题，本文提出一种推广的流形学习算法，即基于Log-Euclidean黎曼核的自适应半监督正交局部保持投影（Log-Euclidean Riemannian kernel-based adaptive semi-supervised orthogonal locality preserving projection，LRK-ASOLPP），并将其成功用于高分辨率遥感影像目标分类问题.首先，提取图像每个像素点处的几何结构特征，计算图像特征的协方差描述子；其次，通过采用Log-Euclidean黎曼核将协方差描述子投影到再生核Hilbert空间；然后，基于流形学习理论，建立黎曼流形上半监督正交局部保持投影算法模型，利用交替迭代更新算法对目标函数进行优化求解，同时获得相似性权矩阵和低维投影矩阵；最后，利用求得的低维投影矩阵计算测试样本的低维投影，并用K—近邻、支持向量机（Support victor machine，SVM）等分类器对其进行分类.三个高分辨率遥感影像数据集上的实验结果说明了该算法的有效性与可行性.     

一种基于LBP特征与流形知识的人脸识别方法

人脸识别是计算机视觉领域的研究热点,应用背景广泛。近年来,流形被认为是视觉感知的基础,流形学习算法被用来发现图像的内在特征。如何利用流形学习后的低维内蕴变量成为相关研究的核心问题。但是利用传统的流形学习算法降维得到的人脸低维特征在可分性上存在一定的不足。此外,流形学习算法对光照和姿态变化敏感。针对这两个问题,提出了一种基于局部二值模式（LBP）和流形知识的人脸识别方法。该方法首先利用LBP算子对人脸图像进行局部特征描述,然后使用流形学习算法获得高维特征数据的低维内蕴变量,并用泰勒展开式近似该流形,获取流形知识,最后利用流形知识估计流形距离来实现人脸识别。实验证明,该方法增强了人脸识别对光照变化的鲁棒性,从而提高了识别性能。     

基于参数化流形学习的压缩传感重构方法

压缩传感是一种新的信息获取理论,它突破了传统的采样理论,将数据采集和压缩合二为一,再利用重构算法将原始数据恢复。为了能够得到更好的压缩传感重构效果,把流形学习的思想和方法与压缩传感相结合,提出了一种基于参数化流形学习的压缩传感重构方法。实验结果表明,提出的方法对自然图像进行重构取得了很好的效果,充分验证了基于参数化流形学习的压缩传感重构方法的有效性。     

流形学习概述

流形学习是一种新的非监督学习方法，近年来引起越来越多机器学习和认知科学工作者的重视．为了加深对流形学习的认识和理解，该文由流形学习的拓扑学概念入手，追溯它的发展过程．在明确流形学习的不同表示方法后，针对几种主要的流形算法，分析它们各自的优势和不足，然后分别引用Isomap和LLE的应用示例．结果表明，流形学习较之于传统的线性降维方法，能够有效地发现非线性高维数据的本质维数。利于进行维数约简和数据分析．最后对流形学习未来的研究方向做出展望，以期进一步拓展流形学习的应用领域．     

基于拉普拉斯特征映射的启发式Q学习

在基于目标的强化学习任务中, 欧氏距离常作为启发式函数用于策略选择, 其用于状态空间在欧氏空间内不连续的任务效果不理想. 针对此问题, 引入流形学习中计算复杂度较低的拉普拉斯特征映射法, 提出一种基于谱图理论的启发式策略选择方法. 所提出的方法适用于状态空间在某个内在维数易于估计的流形上连续, 且相邻状态间的连接关系为无向图的任务. 格子世界的仿真结果验证了所提出方法的有效性.

     

Towards a theoretical foundation for Laplacian-based manifold methods

In recent years manifold methods have attracted a considerable amount of attention in machine learning. However most algorithms in that class may be termed “manifold-motivated” as they lack any explicit theoretical guarantees. In this paper we take a step towards closing the gap between theory and practice for a class of Laplacian-based manifold methods. These methods utilize the graph Laplacian associated to a data set for a variety of applications in semi-supervised learning, clustering, data representation.We show that under certain conditions the graph Laplacian of a point cloud of data samples converges to the Laplace-Beltrami operator on the underlying manifold. Theorem 3.1 contains the first result showing convergence of a random graph Laplacian to the manifold Laplacian in the context of machine learning.     

基于随机游走的流形学习与可视化

现有的全局流形学习算法都敏感于邻域大小这一难以高效选取的参数,它们都采用了基于欧氏距离的邻域图创建方法,从而使邻域图容易产生“短路”边。本文提出了一种基于随机游走模型的全局 流形学习算法(Random walk-based isometric mapping,RW-ISOMAP)。和欧氏距离相比,由随机游走模型得到的通勤时间距离是由给定两点间的所有通路以概率为权组合而成的,不但鲁棒性更高,而且还能在一定程度上度量具有非线性几何结构的数据之间的相似性。因此采用通勤时间距离来创建邻域图的RW-ISOMAP算法将不再敏感于邻域大小参数,从而可以更容易地选取邻域大小参数,同时还具有更高的鲁棒性。最后的实验结果证实了该算法的有效性。     

基于判别正则化的局部保留投影

局部保留投影（Locality preserving projections,LPP）是一种常用的线性化流形学习方法,其通过线性嵌入来保留基于图所描述的流形数据本质结构特征,因此LPP对图的依赖性强,且在嵌入过程中缺少对图描述的进一步分析和挖掘。当图对数据本质结构特征描述不恰当时,LPP在嵌入过程中不易实现流形数据本质结构的有效提取。为了解决这个问题,本文在给定流形数据图描述的条件下,通过引入局部相似度阈值进行局部判别分析,并据此建立判别正则化局部保留投影（简称DRLPP）。该方法能够在现有图描述的条件下,有效突出不同流形结构在线性嵌入空间中的可分性。在人造合成数据集和实际标准数据集上对DRLPP以及相关算法进行对比实验,实验结果证明了DRLPP的有效性。     

Image clustering based on sparse patch alignment framework

Image clustering methods are efficient tools for applications such as content-based image retrieval and image annotation. Recently, graph based manifold learning methods have shown promising performance in extracting features for image clustering. Typical manifold learning methods adopt appropriate neighborhood size to construct the neighborhood graph, which captures local geometry of data distribution. Because the density of data points’ distribution may be different in different regions of the manifold, a fixed neighborhood size may be inappropriate in building the manifold. In this paper, we propose a novel algorithm, named sparse patch alignment framework, for the embedding of data lying in multiple manifolds. Specifically, we assume that for each data point there exists a small neighborhood in which only the points that come from the same manifold lie approximately in a low-dimensional affine subspace. Based on the patch alignment framework, we propose an optimization strategy for constructing local patches, which adopt sparse representation to select a few neighbors of each data point that span a low-dimensional affine subspace passing near that point. After that, the whole alignment strategy is utilized to build the manifold. Experiments are conducted on four real-world datasets, and the results demonstrate the effectiveness of the proposed method.     

A sparse neighborhood preserving non-negative tensor factorization algorithm for facial expression recognition

In this paper, a novel sparse neighborhood preserving non-negative tensor factorization (SNPNTF) algorithm is proposed for facial expression recognition. It is derived from non-negative tensor factorization (NTF), and it works in the rank-one tensor space. A sparse constraint is adopted into the objective function, which takes the optimization step in the direction of the negative gradient, and then projects onto the sparse constrained space. To consider the spatial neighborhood structure and the class-based discriminant information, a neighborhood preserving constraint is adopted based on the manifold learning and graph preserving theory. The Laplacian graph which encodes the spatial information in the face samples and the penalty graph which considers the pre-defined class information are considered in this constraint. By using it, the obtained parts-based representations of SNPNTF vary smoothly along the geodesics of the data manifold and they are more discriminant for recognition. SNPNTF is a quadratic convex function in the tensor space, and it could converge to the optimal solution. The gradient descent method is used for the optimization of SNPNTF to ensure the convergence property. Experiments are conducted on the JAFFE database, the Cohn–Kanade database and the AR database. The results demonstrate that SNPNTF provides effective facial representations and achieves better recognition performance, compared with non-negative matrix factorization, NTF and some variant algorithms. Also, the convergence property of SNPNTF is well guaranteed.     

一种基于局部流形结构的无监督特征学习方法(英文)

Unsupervised feature selection is fundamental in statistical pattern recognition, and has drawn persistent attention in the past several decades. Recently, much work has shown that feature selection can be formulated as nonlinear dimensionality reduction with discrete constraints. This line of research emphasizes utilizing the manifold learning techniques, where feature selection and learning can be studied based on the manifold assumption in data distribution. Many existing feature selection methods such as Laplacian score, SPEC(spectrum decomposition of graph Laplacian), TR(trace ratio) criterion, MSFS(multi-cluster feature selection) and EVSC(eigenvalue sensitive criterion) apply the basic properties of graph Laplacian, and select the optimal feature subsets which best preserve the manifold structure defined on the graph Laplacian. In this paper, we propose a new feature selection perspective from locally linear embedding(LLE), which is another popular manifold learning method. The main difficulty of using LLE for feature selection is that its optimization involves quadratic programming and eigenvalue decomposition, both of which are continuous procedures and different from discrete feature selection. We prove that the LLE objective can be decomposed with respect to data dimensionalities in the subset selection problem, which also facilitates constructing better coordinates from data using the principal component analysis(PCA) technique. Based on these results, we propose a novel unsupervised feature selection algorithm,called locally linear selection(LLS), to select a feature subset representing the underlying data manifold. The local relationship among samples is computed from the LLE formulation, which is then used to estimate the contribution of each individual feature to the underlying manifold structure. These contributions, represented as LLS scores, are ranked and selected as the candidate solution to feature selection. We further develop a locally linear rotation-selection(LLRS) algorithm which extends LLS to identify the optimal coordinate subset from a new space. Experimental results on real-world datasets show that our method can be more effective than Laplacian eigenmap based feature selection methods.     

基于流行排序的多示例图像检索方法

为了提高图像检索的性能,提出了一种基于流行排序的多示例图像检索方法,将分割后的图像表示为多示例的形式,通过给出适合图像在包空间的度量方式,有效结合流行排序和多示例学习的方法来进行图像检索.实验结果表明,采用所提出的方法的检索结果与传统的检索方法相比,检索率得到了明显的提高,检索结果更符合人的视觉习惯.     

Editorial of the Special Issue on Manifold Learning

In many information analysis tasks, one is often confronted with thousands to millions dimensional data, such as images, documents, videos, web data, bioinformatics data, etc. Conventional statistical and computational tools are severely inadequate for processing and analysing high-dimensional data due to the curse of dimensionality, where we often need to conduct inference with a limited number of samples. On the other hand, naturally occurring data may be generated by structured systems with possibly much fewer degrees of freedom than the ambient dimension would suggest. Recently, various works have considered the case when the data is sampled from a submanifold embedded in the much higher dimensional Euclidean space. Learning with full consideration of the low dimensional manifold structure, or specifically the intrinsic topological and geometrical properties of the data manifold is referred to as manifold learning, which has been receiving growing attention in our community in recent years. This special issue is to attract articles that (a) address the frontier problems in the scientific principles of manifold learning, and (b) report empirical studies and applications of manifold learning algorithms, including but not limited to pattern recognition, computer vision, web mining, image processing and so on. A total of 13 submissions were received. The papers included in this special issue are selected based on the reviews by experts in the subject area according to the journal''s procedure and quality standard. Each paper is reviewed by at least two reviewers and some of the papers were revised for two rounds according to the reviewers'' comments. The special issue includes 6 papers in total: 3 papers on the foundational theories of manifold learning, 2 papers on graph-based methods, and 1 paper on the application of manifold learning to video compression. The papers on the foundational theories of manifold learning cover the topics about the generalization ability of manifold learning, manifold ranking, and multi-manifold factorization. In the paper entitled ``Manifold Learning: Generalizing Ability and Tangential Proximity'', Bernstein and Kuleshov propose a tangential proximity based technique to address the generalized manifold learning problem. The proposed method ensures not only proximity between the points and their reconstructed values but also proximity between the corresponding tangent spaces. The traditional manifold ranking methods are based on the Laplacian regularization, which suffers from the issue that the solution is biased towards constant functions. To overcome this issue, in the paper entitled ``Manifold Ranking using Hessian Energy'', Guan et al. propose to use the second-order Hessian energy as regularization for manifold ranking. In the paper entitled ``Multi-Manifold Concept Factorization for Data Clustering'', Li et al. incorporate the multi-manifold ensemble learning into concept factorization to better preserve the local structure of the data, thus yielding more satisfactory clustering results. The papers on graph-based methods cover the topics about label propagation and graph-based dimensionality reduction. In the paper entitled ``Bidirectional Label Propagation over Graphs'', Liu et al. propose a novel label propagation algorithm to propagate labels along positive and negative edges in the graph. The construction of the graph is novel against the conventional approach by incorporating the dissimilarity among data points into the affinity matrix. In the paper entitled ``Locally Regressive Projections'', Lijun Zhang proposes a novel graph-based dimensionality reduction method that captures the local discriminative structure of the data space. The key idea is to fit a linear model locally around each data point, and then use the fitting error to measure the performance of dimensionality reduction. In the last paper entitled ``Combining Active and Semi-Supervised Learning for Video Compression'', motivated from manifold regularization, Zhang and Ji propose a machine learning approach for video compression. Active learning is used to select the most representative pixels in the encoding process, and semi-supervised learning is used to recover the color video in the decoding process. One remarking property of this approach is that the active learning algorithm shares the same loss function as the semi-supervised learning algorithm, providing a unified framework for video compression. Many people have been involved in making this special issue possible. The guest editor would like to express his gratitude to all the contributing authors for their insightful work on manifold learning. The guest editor would like to thank the reviewers for their comments and useful suggestions in order to improve the quality of the papers. The guest editor would also like to thank Prof. Ruqian Lu, the editor-in-chief of the International Journal of Software and Informatics, for providing the precious opportunity to publish this special issue. Finally, we hope the reader will enjoy this special issue and find it useful.