一种针对多关系数据的半监督协同训练算法

半监督学习是机器学习领域的研究热点。协同训练研究数据有多个特征集时的半监督学习问题。将图表示法引入协同训练,使用多个图结构表示多关系数据。在每个图上进行半监督学习,在多个图之间进行协同学习,使多个图上的学习器对数据的预测一致。创新性地提出一种针对多关系数据的半监督协同训练算法,并从概率角度分析学习过程。在真实数据集上的实验表明,提出的算法处理多关系数据时具有较好的性能。     

异构信息网络上基于图正则化的半监督学习

现实世界中存在着大量包含多种类型的对象和联系的异构信息网络,从中挖掘信息获取知识已成为当前的研究热点之一．基于图正则化的半监督学习在近年来得到了广泛的研究,然而,现有的半监督学习算法大都只能应用于同构网络．基于同构节点和异构节点的一致性假设,提出了任意结构的异构信息网络上的半监督学习的正则化分类函数,并得到分类函数的闭式解,以此预测未标记节点的类别．提出了异构信息网络上的半监督学习的迭代框架,标记节点的信息可以在邻近的节点上迭代传播,直至达到稳定状态,并证明了迭代算法将收敛于正则化分类函数的闭式解．DBL P数据集上的实验表明该方法优于经典的半监督学习算法．     

基于异质信息融合的网络图像半监督学习方法

网络图像通常包含文本、颜色和纹理等异质信息. 本文提出了一种基于多类异质信息融合的网络图像半监督学习方法---局部协同训练(Local co-training, LCT). 该方法在每个视图(对应一类 信息)上对每个样本点的邻域构建线性局部模型, 利用一组局部模型来表示数据关系;基于信息传播和协同训练对模型进行增量式迭代更新. 该算法在协同训练和基于图正则化的方法这两类半监督学习算法间建立了桥梁. 局部协同训练算法能够准确地描述样本的复杂分布, 并且可以进行高效的增量学习, 有利于大规模网络图像的在线学习. 在Corel, Pascal和ImageNet数据集上的实验结果表明该方法具有良好的性能.     

基于多视图半监督集成学习的人体动作识别算法

移动设备上难以获取大量标签样本，而训练不足导致分类模型在人体动作识别上表现欠佳。针对这一问题，提出一种基于多视图半监督集成学习的人体动作识别算法。首先，利用两种内置传感器收集的数据构建两个特征视图，将两个视图和两种基分类器进行组合构建协同学习框架；然后，根据多分类任务重新定义置信度，结合主动学习思想在迭代过程中控制预测伪标签结果；使用LightGBM对扩充后的训练集进行学习。实验结果表明，算法的精确率、召回率和F1值较高，能稳定、准确地识别多种人体动作。     

基于TE-DS的半监督化工过程故障诊断方法

针对现有基于深度学习的化工过程故障诊断方法通常需要完备的标签数据才能构建故障诊断模型等局限,提出一种基于时间集成—双重学生模型(temporal ensembling-dual student, TE-DS)的半监督化工过程故障诊断方法。该方法首先以双重学生模型为基础,通过分类项约束、稳定性约束和一致性约束条件指导相互训练,有效地缓解了误差累积情况的发生;然后利用时间集成(temporal ensembling)将多个先前网络评估的预测集成作为一致性正则化对象,达到缓解预测值噪声、降低模型训练时间的目的,以提高分类性能,实现故障诊断;最后通过田纳西—伊斯曼(Tennessee-Eastman)化工过程基准数据进行故障诊断实验,验证提出方法的有效性和可行性,并与BNLSTM、DCNN和MCLSTM等有监督方法进行比较,证明了TE-DS算法对故障诊断的优越性。     

结合全局和局部正则化的半监督二分类算法

针对在半监督分类问题中单独使用全局学习容易出现的在整个输入空间中较难获得一个优良的决策函数的问题,以及单独使用局部学习可在特定的局部区域内习得较好的决策函数的特点,提出了一种结合全局和局部正则化的半监督二分类算法。该算法综合全局正则项和局部正则项的优点,基于先验知识构建的全局正则项能平滑样本的类标号以避免局部正则项学习不充分的问题,通过基于局部邻域内样本信息构建的局部正则项使得每个样本的类标号具有理想的特性,从而构造出半监督二分类问题的目标函数。通过在标准二类数据集上的实验,结果表明所提出的算法其平均分类正确率和标准误差均优于基于拉普拉斯正则项方法、基于正则化拉普拉斯正则项方法和基于局部学习正则项方法。     

利用正则化矩阵分解技术的多视图聚类方法

为了解决具有多种特征属性的多媒体数据（多视图数据）挖掘问题,在非负矩阵分解（NMF）算法的基础上,提出了一种多视图正则化矩阵分解算法（MRMF）,该算法使用了多元非负矩阵分解技术,同时使用[L2,1]范数描述矩阵分解的损失函数,并采用多视图流形正则化对矩阵分解进行正则化约束。与现有的一些数据聚类或多视图聚类算法相比,提出的MRMF算法不易受到原始数据中噪声的影响,而且能够充分考虑到不同视图在聚类中所具有不同权重的问题,能够对多视图数据进行较为准确的聚类。MRMF算法的有效性在一些经典的公开数据集上进行了验证,并取得了较好的聚类精度。     

半监督多示例核

在多示例学习中引入利用未标记示例的机制,能降低训练的成本并提高学习器的泛化能力。当前半监督多示例学习算法大部分是基于对包中的每一个示例进行标记,把多示例学习转化为一个单示例半监督学习问题。考虑到包的类标记由包中示例及包的结构决定,提出一种直接在包层次上进行半监督学习的多示例学习算法。通过定义多示例核,利用所有包(有标记和未标记)计算包层次的图拉普拉斯矩阵,作为优化目标中的光滑性惩罚项。在多示例核所张成的RKHS空间中寻找最优解被归结为确定一个经过未标记数据修改的多示例核函数,它能直接用在经典的核学习方法上。在实验数据集上对算法进行了测试,并和已有的算法进行了比较。实验结果表明,基于半监督多示例核的算法能够使用更少量的训练数据而达到与监督学习算法同样的精度,在有标记数据集相同的情况下利用未标记数据能有效地提高学习器的泛化能力。     

结合包光滑性的半监督多示例核学习方法

以往半监督多示例学习算法常把未标记包分解为示例集合,使用传统的半监督单示例学习算法确定这些示例的潜在标记以对它们进行利用。但该类方法认为多示例样本的分类与其概率密度分布紧密相关,且并未考虑包结构对包分类标记的影响。提出一种基于包层次的半监督多示例核学习方法,直接利用未标记包进行半监督学习器的训练。首先通过对示例空间聚类把包转换为概念向量表示形式,然后计算概念向量之间的海明距离,在此基础上计算描述包光滑性的图拉普拉斯矩阵,进而计算包层次的半监督核,最后在多示例学习标准数据集和图像数据集上测试本算法。测试表明本算法有明显的改进效果。     

半监督深度学习图像分类方法研究综述

作为人工智能领域近十年来最受关注的技术之一,深度学习在诸多应用中取得了优异的效果,但目前的学习策略严重依赖大量的有标记数据.在许多实际问题中,获得众多有标记的训练数据并不可行,因此加大了模型的训练难度,但容易获得大量无标记的数据.半监督学习充分利用无标记数据,提供了在有限标记数据条件下提高模型性能的解决思路和有效方法,在图像分类任务中达到了很高的识别精准度.首先对于半监督学习进行概述,然后介绍了分类算法中常用的基本思想,重点对近年来基于半监督深度学习框架的图像分类方法,包括多视图训练、一致性正则、多样混合和半监督生成对抗网络进行全面的综述,总结多种方法共有的技术,分析比较不同方法的实验效果差异,最后思考当前存在的问题并展望未来可行的研究方向.     

Sparse regularization for semi-supervised classification

Manifold regularization (MR) is a promising regularization framework for semi-supervised learning, which introduces an additional penalty term to regularize the smoothness of functions on data manifolds and has been shown very effective in exploiting the underlying geometric structure of data for classification. It has been shown that the performance of the MR algorithms depends highly on the design of the additional penalty term on manifolds. In this paper, we propose a new approach to define the penalty term on manifolds by the sparse representations instead of the adjacency graphs of data. The process to build this novel penalty term has two steps. First, the best sparse linear reconstruction coefficients for each data point are computed by the l¹-norm minimization. Secondly, the learner is subject to a cost function which aims to preserve the sparse coefficients. The cost function is utilized as the new penalty term for regularization algorithms. Compared with previous semi-supervised learning algorithms, the new penalty term needs less input parameters and has strong discriminative power for classification. The least square classifier using our novel penalty term is proposed in this paper, which is called the Sparse Regularized Least Square Classification (S-RLSC) algorithm. Experiments on real-world data sets show that our algorithm is very effective.     

A comprehensive survey on regularization strategies in machine learning

In machine learning, the model is not as complicated as possible. Good generalization ability means that the model not only performs well on the training data set, but also can make good prediction on new data. Regularization imposes a penalty on model’s complexity or smoothness, allowing for good generalization to unseen data even when training on a finite training set or with an inadequate iteration. Deep learning has developed rapidly in recent years. Then the regularization has a broader definition: regularization is a technology aimed at improving the generalization ability of a model. This paper gave a comprehensive study and a state-of-the-art review of the regularization strategies in machine learning. Then the characteristics and comparisons of regularizations were presented. In addition, it discussed how to choose a regularization for the specific task. For specific tasks, it is necessary for regularization technology to have good mathematical characteristics. Meanwhile, new regularization techniques can be constructed by extending and combining existing regularization techniques. Finally, it concluded current opportunities and challenges of regularization technologies, as well as many open concerns and research trends.     

Towards making co-training suffer less from insufficient views

Co-training is a famous semi-supervised learning algorithm which can exploit unlabeled data to improve learning performance. Generally it works under a two-view setting (the input examples have two disjoint feature sets in nature), with the assumption that each view is sufficient to predict the label. However, in real-world applications due to feature corruption or feature noise, both views may be insufficient and co-training will suffer from these insufficient views. In this paper, we propose a novel algorithm named Weighted Co-training to deal with this problem. It identifies the newly labeled examples that are probably harmful for the other view, and decreases their weights in the training set to avoid the risk. The experimental results show that Weighted Co-training performs better than the state-of-art co-training algorithms on several benchmarks.     

拉普拉斯多层极速学习机

极速学习机不仅仅是有效的分类器,还能应用到半监督学习中.但是,半监督极速学习机和拉普拉斯光滑孪生支持向量机一样是一种浅层学习算法.深度学习实现了复杂函数的逼近并缓解了以前多层神经网络算法的局部最小性问题,目前在机器学习领域中引起了广泛的关注.多层极速学习机（ML-ELM）是根据深度学习和极速学习机的思想提出的算法,通过堆叠极速学习机-自动编码器算法（ELM-AE）构建多层神经网络模型,不仅实现复杂函数的逼近,并且训练过程中无需迭代,学习效率高.我们把流形正则化框架引入ML-ELM中提出拉普拉斯多层极速学习机算法（Lap-ML-ELM）.然而,ELM-AE不能很好的解决过拟合问题,针对这一问题我们把权值不确定引入ELM-AE中提出权值不确定极速学习机-自动编码器算法（WU-ELM-AE）,它学习到更为鲁棒的特征.最后,我们在前面两种算法的基础上提出权值不确定拉普拉斯多层极速学习机算法（WUL-ML-ELM）,它堆叠WU-ELM-AE构建深度模型,并且用流形正则化框架求取输出权值,该算法在分类精度上有明显提高并且不需花费太多的时间.实验结果表明,Lap-ML-ELM与WUL-ML-ELM都是有效的半监督学习算法.     

基于样本条件价值改进的 Co-training 算法

Co-training是一种主流的半监督学习算法. 该算法中两视图下的分类器通过迭代的方式, 互为对方从无标记样本集中挑选新增样本, 以更新对方训练集. Co-training以分类器的后验概率输出作为新增样本的挑选策略, 该策略忽略了样本对于当前分类器的价值. 针对该问题, 本文提出一种改进的Co-training式算法—CVCOT (Conditional value-based co-training), 即采用基于样本条件价值的挑选策略来优化Co-training. 通过定义无标记样本的条件价值, 各视图下的分类器以样本条件价值为依据来挑选新增样本, 以此更新训练集. 该策略既可保证新增样本的标记可靠性, 又能优先将价值较高的富信息样本补充到训练集中, 可以有效地优化分类器. 在UCI数据集和网页分类应用上的实验结果表明: CVCOT具有较好的分类性能和学习效率.     

弹性网络核极限学习机的多标记学习算法

将正则化极限学习机或者核极限学习机理论应用到多标记分类中,一定程度上提高了算法的稳定性。但目前这些算法关于损失函数添加的正则项都基于L2正则,导致模型缺乏稀疏性表达。同时,弹性网络正则化既保证模型鲁棒性且兼具模型稀疏化学习,但结合弹性网络的极限学习机如何解决多标记问题鲜有研究。基于此,本文提出一种对核极限学习机添加弹性网络正则化的多标记学习算法。首先,对多标记数据特征空间使用径向基核函数映射;随后,对核极限学习机损失函数施加弹性网络正则项;最后,采用坐标下降法迭代求解输出权值以得到最终预测标记。通过对比试验和统计分析表明,提出的算法具有更好的性能表现。     

基于正则化的半监督弱标签分类方法

针对多标签学习中实例标签的缺失补全和预测问题,本文提出一种基于正则化的半监督弱标签分类方法(简称SWCMR),方法同时兼顾实例相似性和标签相关性.SWCMR首先根据标签相关性对弱标签实例的缺失标签进行初步预估,然后利用弱标签实例和无标签实例构造邻域图,从实例相似性和标签相关性角度构建基于平滑性假设的正则化项,接下来利用预估后的弱标签实例结合无标签实例训练半监督弱标签分类模型.在多种公共多标签数据集上的实验结果表明,SWCMR提高了分类性能,尤其是标签信息较少时,分类效果提升更显著.     

Deep Learning in Sheet Metal Bending With a Novel Theory-Guided Deep Neural Network

Sheet metal forming technologies have been intensively studied for decades to meet the increasing demand for lightweight metal components. To surmount the springback occurring in sheet metal forming processes, numerous studies have been performed to develop compensation methods. However, for most existing methods, the development cycle is still considerably time-consumptive and demands high computational or capital cost. In this paper, a novel theory-guided regularization method for training of deep neural networks (DNNs), implanted in a learning system, is introduced to learn the intrinsic relationship between the workpiece shape after springback and the required process parameter, e.g., loading stroke, in sheet metal bending processes. By directly bridging the workpiece shape to the process parameter, issues concerning springback in the process design would be circumvented. The novel regularization method utilizes the well-recognized theories in material mechanics, Swift’s law, by penalizing divergence from this law throughout the network training process. The regularization is implemented by a multi-task learning network architecture, with the learning of extra tasks regularized during training. The stress-strain curve describing the material properties and the prior knowledge used to guide learning are stored in the database and the knowledge base, respectively. One can obtain the predicted loading stroke for a new workpiece shape by importing the target geometry through the user interface. In this research, the neural models were found to outperform a traditional machine learning model, support vector regression model, in experiments with different amount of training data. Through a series of studies with varying conditions of training data structure and amount, workpiece material and applied bending processes, the theory-guided DNN has been shown to achieve superior generalization and learning consistency than the data-driven DNNs, especially when only scarce and scattered experiment data are available for training which is often the case in practice. The theory-guided DNN could also be applicable to other sheet metal forming processes. It provides an alternative method for compensating springback with significantly shorter development cycle and less capital cost and computational requirement than traditional compensation methods in sheet metal forming industry.      

半监督学习的Co-training算法研究

介绍一种基于半监督学习的协同训练（Co-training）分类算法,当可用的训练样本比较少时,使用传统的方法进行分类,如决策树分类,将无法得到用户满意的结果,而且它们需要大量的标记样本。事实上,获取有标签的样本的代价是相当昂贵的。于是,使用较少的已标记样本和大量的无标记样本进行协同训练的半监督学习,成为研究者首选。