基于流形正则化的分类与回归算法及应用

利用流形正则化的思想,围绕半监督学习,提出了一种针对流形正则化的模式分类和回归分析的新算法.该算法基于流形上的正则化项和传统的正则化项相结合的方法,利用支持向量机分类与回归已有的结果,解决半监督学习的分类与回归问题,提高了泛化能力.该算法实现简单,无需调用其他程序.通过数值试验,验证了该算法具有较好的泛化能力,对噪音具有较强的鲁棒性.且在分类问题上,该算法在输入极少数有标签样本时,也能保持较好的分类效果;在回归问题上,也具有较好的学习精度,尤其在输入带有噪音的流形数据上时,表现就更为突出.     

自组织增量学习神经网络综述

自组织增量学习神经网络SOINN（self-organizing incremental neural network）是一种基于竞争学习的两层神经网络,用于在没有先验知识的情况下对动态输入数据进行在线聚类和拓扑表示,同时,对噪音数据具有较强的鲁棒性.SOINN的增量性,使得它能够发现数据流中出现的新模式并进行学习,同时不影响之前学习的结果.因此,SOINN能够作为一种通用的学习算法应用于各类非监督学习问题中.对SOINN的模型和算法进行相应的调整,可以使其适用于监督学习、联想记忆、基于模式的推理、流形学习等多种学习场景中.SOINN已经在许多领域得到了应用,包括机器人智能、计算机视觉、专家系统、异常检测等.     

基于非线性流形学习和支持向量机的文本分类算法

为解决文本自动分类问题,提出一种流形学习和支持向量机相结合的文本分类算法(LLE-LSSVM)。LLE-LSSVM算法利用非线性流形学习算法LEE对高维文本特征进行非线性降维,挖掘出特征内在规律与本征信息,从而得到低维特征空间,然后将其输入到LSSVM中进行学习,同时利用混沌粒子群算法对LSSVM参数进行优化,建立文本分类模型。仿真实验结果表明,LLE-LSSVM算法提高了文本分类准确率,减少了分类运行时间,是一种有效的文本分类算法。     

基于流形正则化的支持向量回归及应用

利用流形正则化的思想,围绕半监督学习,提出了一种针对回归问题的新算法。该算法基于流形上的正则化项和传统的正则化项相结合的方法,利用支持向量机回归已有的结果,解决半监督学习的回归问题,提高了泛化能力。通过数值试验,验证了该算法具有较好的泛化能力,对噪音具有较强的鲁棒性,与支持向量回归相比,具有更高的学习精度。     

基于Isomap的流形结构重建方法

已有的流形学习方法仅能建立点对点的降维嵌入,而未建立高维数据流形空间与低维表示空间之间的相互映射.此缺陷已限制了流形学习方法在诸多数据挖掘问题中的进一步应用.针对这一问题,文中提出了两种新型高效的流形结构重建算法:快速算法与稳健算法.其均以经典的Isomap方法内在运行机理为出发点,进而推导出高维流形空间与低维表示空间之间双向的显式映射函数关系,基于此函数即可实现流形映射的有效重建.理论分析与实验结果证明,所提算法在计算速度、噪音敏感性、映射表现等方面相对已有方法具有明显优势.     

流形学习算法中的参数选择问题研究

流形学习(Manifold Learning)算法是近年来发展起来的非线性降维机器学习算法.等度规特征映射Isomap(Isometric feature mapping)和局部线性嵌入LLE(Locally Linear Embedding)是两种典型的流形学习算法.通过实验比较和分析两种算法中邻接参数K和采样点数N的选取对降维结果以及执行时间的影响,实验结果表明Isomap对邻接参数K和采样点数N具有较高的容忍度,而LLE算法在计算速度上优势明显.     

一种基于核的监督流形学习算法

针对流形学习算法--局部保持映射存在的参数选择及不能进行非线性特征提取的问题,提出一种基于核的监督流形学习算法.该算法作为局部保持映射算法的改进算法用样本类标识信息指导建立局部最近邻图,并在建立局部最近邻图使用无参数的相似度量.利用核方法来解决局部保持映射算法在处理线性不可分问题上的局限性问题.在两个常用数据库上验证本文算法的可行性和有效性.     

基于重构权的离群点检测方法

近几年来,流形学习在模式识别、机器学习和数据挖掘等许多领域都受到了广泛的关注.但是,通常的流形学习方法对离群点缺乏鲁棒性.对此,提出了一种基于重构权的流形离群点检测方法.该方法在每个样本点构造局部"强"邻域,再利用局部重构权来计算每个样本点的可靠值,最后利用可靠值检测出离群点.该算法具有计算快、参数少、参数敏感性小等优点.基于此离群点检测方法,提出了鲁棒的Isomap算法.实验结果表明,该方法能够有效检测离群点,从而提高流形学习方法对离群点的鲁棒性.     

等谱流形学习算法

基于谱方法的流形学习算法的目标是发现嵌入在高维数据空间中的低维表示.近年来,该算法已得到广泛的应用.等谱流形学习是谱方法中的主要内容之一.等谱流形学习源于这样的结论:只要两个流形的谱相同,其内部结构就是相同的.而谱计算难以解决的问题是近邻参数的选择以及如何构造合理邻接权.为此,提出了等谱流形学习算法(isospectral manifold learning algorithm,简称IMLA).它通过直接修正稀疏重构权矩阵,将类内的判别监督信息和类间的判别监督信息同时融入邻接图,达到既能保持数据间稀疏重建关系,又能利用监督信息的目的,与PCA等算法相比具有明显的优势.该算法在3 个常用人脸数据集(Yale,ORL,Extended Yale B)上得到了验证,这进一步说明了IMLA 算法的有效性.     

流形学习与基于线性耦合映射的流形对齐

从一些具有代表性的经典流形学习方法的回顾来看,传统的流形学习主要处理来自单一流形的数据的降维问题.随着流形学习研究的不断深入,以多流形作为研究对象的流形学习问题逐步引起了研究者的注意.提出了一种基于线性耦合映射的流形对齐算法.算法克服非线性流形对齐算法不能够直接处理Out-of-sample数据的问题.同时,与已有的线性流形对齐算法相比,该算法不需要假设流形间满足仿射变换关系,因而能够更加灵活地处理一些比较实际的流形对齐问题.     

Robust local tangent space alignment via iterative weighted PCA

Recently manifold learning has attracted extensive interest in machine learning and related communities. This paper investigates the noise manifold learning problem, which is a key issue in applying manifold learning algorithm to practical problems. We propose a robust version of LTSA algorithm called RLTSA. The proposed RLTSA algorithm makes LTSA more robust from three aspects: firstly robust PCA algorithm based on iterative weighted PCA is employed instead of the standard SVD to reduce the influence of noise on local tangent space coordinates; secondly RLTSA chooses neighborhoods that are well approximated by the local coordinates to align with the global coordinates; thirdly in the alignment step, the influence of noise on embedding result is further reduced by endowing clean data points and noise data points with different weights into the local alignment errors. Experiments on both synthetic data sets and real data sets demonstrate the effectiveness of our RLTSA when dealing with noise manifold.     

拉普拉斯阶梯网络

阶梯网络不仅是一种基于深度学习的特征提取器，而且能够应用于半监督学习中.深度学习在实现了复杂函数逼近的同时，也缓解了多层神经网络易陷入局部最小化的问题.传统的自编码、玻尔兹曼机等方法易忽略高维数据的低维流形结构信息，使用这些方法往往会获得无意义的特征表示，这些特征不能有效地嵌入到后续的预测或识别任务中.从流形学习的角度出发，提出一种基于阶梯网络的深度表示学习方法，即拉普拉斯阶梯网络LLN （Laplacian ladder network）.拉普拉斯阶梯网络在训练的过程中不仅对每一编码层嵌入噪声并进行重构，而且在各重构层引入图拉普拉斯约束，将流形结构嵌入到多层特征学习中，以提高特征提取的鲁棒性和判别性.在有限的有标签数据情况下，拉普拉斯阶梯网络将监督学习损失和非监督损失融合到了统一的框架进行半监督学习.在标准手写数据数据集MNIST和物体识别数据集CIFAR-10上进行了实验，结果表明，相对于阶梯网络和其他半监督方法，拉普拉斯阶梯网络都得到了更好的分类效果，是一种有效的半监督学习算法.     

The dynamical neighborhood selection based on the sampling density and manifold curvature for isometric data embedding

The construction of the neighborhood is a critical problem of manifold learning. Most of manifold learning algorithms use a stable neighborhood parameter (such as k-NN), but it may not work well for the entire manifold, since manifold curvature and sampling density may vary over the manifold. Although some dynamical neighborhood algorithms have been proposed, they are limited by either another global parameter or an assumption. This paper proposes a new approach to select the dynamical neighborhood for each point while constructing the tangent subspace based on the sampling density and the manifold curvature. And the parameters of the approach can be automatically determined by computing the correlation coefficient of the matrices of geodesic distances between pairs of points in input and output spaces. When we apply it to ISOMAP, the results of experiments on the synthetic data as well as the real world patterns demonstrate that the proposed approach can efficiently maintain an accurate low dimensional representation of the manifold data with less distortion, and give higher average classification rate compared to others.     

Data‐based iterative learning mechanism for unknown input‐output coupling parameters/matrices

In this paper, we explore how to get the information of input‐output coupling parameters (IOCPs) for a class of uncertain discrete‐time systems by using iterative learning technique. Firstly, by taking advantage of repetitiveness of control system and informative input and output data, we design an iterative learning scheme for unknown IOCPs. It is shown that we can get the exact values of IOCPs one by one through running the repetitive system T+1 times if the control system is with identical initial state and noise free. Secondly, we give the iterative learning scheme for unknown IOCPs in the presence of measurement noise, system noise, or initial state drift and analyze the influence factors on the performance of developed iterative learning scheme. Meanwhile, we introduce the maximum allowable control deviation into the iterative learning mechanism to minimize the negative impact of noise on the performance of learning scheme and to enhance the robust of iterative learning scheme. Thirdly, for a class of multiple‐input–multiple‐output systems, we also develop iterative learning mechanism for unknown input‐output coupling matrices. Finally, an illustrative example is given to demonstrate the effectiveness of proposed iterative learning scheme.     

流形距离与压缩感知核稀疏投影的局部线性嵌入算法

流形学习算法的目的是发现嵌入在高维数据空间中的低维表示,现有的流形学习算法对邻域参数k和噪声比较敏感。针对此问题,文中提出一种流形距离与压缩感知核稀疏投影的局部线性嵌入算法,其核心思想是集成局部线性嵌入算法对高维流形结构数据的降维有效性与压缩感知核稀疏投影的强鉴别性,以实现高效有降噪流形学习。首先,在选择各样本点的近邻域时,采用流形距离代替欧氏距离度量数据间相似度的方法,创建能够正确反映流形内部结构的邻域图,解决以欧氏距离作为相似性度量时对邻域参数的敏感。其次,利用压缩感知核稀疏投影作为从高维观测空间到低维嵌入空间的映射,增强算法的鉴别性。最后,利用Matlab工具对实验数据集进行仿真,进一步验证所提算法的有效性。     

输入不确定BELM在发酵过程软测量中的应用

为复杂的发酵过程建立软测量模型要求模型最好能够给出预测值的置信区间,以便技术人员对发酵过程的真实状况和模型的可靠性进行评估。贝叶斯极限学习机能够在实现预测的同时一并给出预测值的置信区间,因此将其用于发酵过程的软测量建模。然而,实际发酵过程中的输入数据往往带有噪声,贝叶斯极限学习机仅能处理输出含噪声的情况。针对这个问题,提出了输入不确定贝叶斯极限学习机。在原有的贝叶斯推理过程中引入输入不确定性,得到了综合考虑输入输出噪声的模型参数和预测置信区间。最后利用青霉素发酵过程进行仿真验证,建立了产物质量浓度的软测量模型,结果表明该方法预测精度高,得到的预测置信区间包含了所有真实值。     

A robust learning algorithm based on support vector regression and robust fuzzy cerebellar model articulation controller

For real-world applications, the obtained data are always subject to noise or outliers. The learning mechanism of cerebellar model articulation controller (CMAC), a neurological model, is to imitate the cerebellum of human being. CMAC has an attractive property of learning speed in which a small subset addressed by the input space determines output instantaneously. For fuzzy cerebellar model articulation controller (FCMAC), the concept of fuzzy is incorporated into CMAC to improve the accuracy problem. However, the distributions of errors into the addressed hypercubes may cause unacceptable learning performance for input data with noise or outliers. For robust fuzzy cerebellar model articulation controller (RFCMAC), the robust learning of M-estimator can be embedded into FCMAC to degrade noise or outliers. Meanwhile, support vector machine (SVR) is a machine learning theory based algorithm which has been applied successfully to a number of regression problems when noise or outliers exist. Unfortunately, the practical application of SVR is limited to defining a set of parameters for obtaining admirable performance by the user. In this paper, a robust learning algorithm based on support SVR and RFCMAC is proposed. The proposed algorithm has both the advantage of SVR, the ability to avoid corruption effects, and the advantage of RFCMAC, the ability to obtain attractive properties of learning performance and to increase accurate approximation. Additionally, particle swarm optimization (PSO) is applied to obtain the best parameters setting for SVR. From simulation results, it shows that the proposed algorithm outperforms other algorithms.     

Iterative learning of model reference adaptive controller for uncertain nonlinear systems with only output measurement

In this paper, a model reference adaptive control strategy is used to design an iterative learning controller for a class of repeatable nonlinear systems with uncertain parameters, high relative degree, initial output resetting error, input disturbance and output noise. The class of nonlinear systems should satisfy some differential geometric conditions such that the plant can be transformed via a state transformation into an output feedback canonical form. A suitable error model is derived based on signals filtered from plant input and output. The learning controller compensates for the unknown parameters, uncertainties and nonlinearity via projection type adaptation laws which update control parameters along the iteration domain. It is shown that the internal signals remain bounded for all iterations. The output tracking error will converge to a profile which can be tuned by design parameters and the learning speed is improved if the learning gain is large.