组合凸线性感知器的极大切割构造方法

组合凸线性感知器（Multiconlitron）是用来构造分片线性分类器的一个通用理论框架,对于凸可分和叠可分情况,分别使用支持凸线性感知器算法（Support conlitron algorithm,SCA）和支持组合凸线性感知器算法（Support multiconlitron algorithm,SMA）将两类样本分开. 本文在此基础上,提出了一种基于极大切割（Maximal cutting）的组合凸线性感知器构造方法. 该方法由两阶段训练构成,第一阶段称为极大切割过程（Maximal cutting process,MCP）,通过迭代不断寻求能够切开最多样本的线性边界,并因此来构造尽可能小的决策函数集,最大程度减少决策函数集中线性函数的数量,最终简化分类模型. 第二阶段称为边界调整过程（Boundary adjusting process,BAP）,对MCP得到的初始分类边界进行一个二次训练,调整边界到适当位置,以提高感知器的泛化能力. 数值实验说明,此方法能够产生更为合理的分类模型,提高了感知器的性能. 同其他典型分片线性分类器的性能对比,也说明了这种方法的有效性和竞争力.     

一种新的语言模型判别训练方法

已有的一些判别训练（discriminative training）方法如Boosting为了提高算法的效率,要求损失函数（loss function）是可以求导的,这样的损失函数无法体现最直接的优化目标．而根据最直接优化目标定义的损失函数通常是不可导的阶梯函数的形式．为了解决上述问题,文章提出了一种新的判别训练的方法GAP（Greedy Approximation Processing）．这种方法具有很强的通用性,只要满足阶梯函数形式的损失函数都可以通过此算法进行训练．由于阶梯形式的损失函数是不可导的,无法使用梯度下降的方式计算极值并获得特征权值．因此,GAP采用“贪心”算法的方式,顺序地从特征集合中选取特征,通过穷举搜索的方式确定其权值．为了提高GAP算法的速度,作者在GAP算法中引入了特征之间独立的假设,固定特征的更新顺序,提出了GAP的改进算法FGAP（Fast Greedy Approximation Processing）．为了证明FGAP算法的有效性,该文将FGAP算法训练的模型应用到日文输入法中．实验结果表明通过FGAP算法训练的语言模型优于Boosting算法训练的模型,与基础模型相比相对错误率下降了15％～19％．     

集成学习算法的差异性及性能比较

从差异性出发,研究了基于特征集技术（通过一定的策略选取不同特征集以组成训练集）与数据技术（通过取样技术选取不同的训练集）的集成学习算法,分析了两种集成学习算法产生差异性的方法。针对决策树与神经网络模型,在标准数据集中对集成学习算法的性能进行实验研究,结果表明集成学习算法的性能依赖于数据集的特性以及产生差异性的方法等因素。从总体性能考虑,基于数据的集成学习算法在大多数数据集上优于基于特征集的集成学习算法。     

基于感知器的SVM自学习模型

提出了一种基于感知器的SVM分类模型（PSVM）。该模型在对分类器的训练中,引入感知器分类思想,其先利用SVM的核函数进行核计算,判断其分类性能,分类正确则不作任何修改,反之则转化成感知器分类问题。实验结果表明该模型不但能提高SVM的分类性能,而且还可以降低SVM分类性能对核函数及参数选择的依赖。     

基于多特征提取的图像语义描述算法

针对图像语义描述方法中存在的图像特征信息提取不完全以及循环神经网络（RNN）产生的梯度消失问题,提出了一种基于多特征提取的图像语义描述算法。所构建模型由三个部分组成：卷积神经网络（CNN）用于图像特征提取,属性提取模型（ATT）用于图像属性提取,而双向长短时记忆（Bi-LSTM）网络用于单词预测。该模型通过提取图像属性信息来增强图像表示,从而精确描述图中事物,并且使用Bi-LSTM捕捉双向语义依赖,从而进行长期的视觉语言交互学习。首先,使用CNN和ATT分别提取图像全局特征与图像属性特征;其次,将两种特征信息输入到Bi-LSTM中生成能够反映图像内容的句子;最后,在Microsoft COCO Caption、Flickr8k和Flickr30k数据集上验证了所提出算法的有效性。实验结果表明,与m-RNN方法相比,所提出的算法在描述性能方面提高了6.8~11.6个百分点。所提算法能够有效地提高模型对图像的语义描述性能。     

基于Motif聚集系数与时序划分的高阶链接预测方法

高阶链接预测是当前网络分析研究的热点和难点,一个优秀的高阶链接预测算法不仅可以挖掘出复杂网络中节点间存在的潜在联系,还有助于认识网络结构随时间演化的规律,对于探索未知的网络关系有着重要的作用.大多数传统的链接预测算法仅考虑节点间的结构相似性特征,而忽略高阶结构的特性以及网络变化的信息.本文提出了一种基于Motif聚集系数与时序划分的高阶链接预测模型（简称MTLP模型）,该模型通过提取网络中高阶结构的Motif聚集系数特征和网络结构演变等特征,将其构建成可表示性特征向量,并使用多层感知器网络模型进行训练完成链接预测任务.该模型能够同时结合网络中高阶结构的聚集特征与网络结构演变信息,从而改善预测效果.通过在不同的数据集上进行实验,其结果表明,本文所提出的MTLP模型具有更好的高阶链接预测性能.     

基于知识的模糊多层感知器

通过与标准模糊多层感知器的对比文章描述了基于知识的模糊多层感知器的具体结构,讲述了如何利用先验知识对连接权进行初始化以及如何从已经过训练的网络中抽取推理规则,文章最后指出这种网络在训练效率和识别性能上优于标准模糊多层感知器。     

基于PPI网络与机器学习的蛋白质功能预测方法

针对现有的基于蛋白质相互作用（PPI）网络的蛋白质功能预测方法预测精度不高、易受数据噪声影响的问题,提出一种基于机器学习（层次聚类、主成分分析和多层感知器）的蛋白质功能预测方法HPMM。该方法综合考虑蛋白质宏观和微观层面的信息,将蛋白质家族、结构域和重要位点信息作为顶点属性整合到PPI网络中以减轻网络中数据噪声的影响。首先,基于层次聚类和主成分分析进行特征提取,得到功能模块和属性主成分特征,然后训练多层感知器模型,建立多特征与多功能之间的映射关系以用于功能预测。在三个分别被分子功能（MF）、生物过程（BP）和细胞组件（CC）注释的人类PPI网络上进行测试,对HPMM、余弦迭代算法（CIA）和有向PPI网络基因本体术语传播（GoDIN）算法的功能预测效果进行比较分析。实验结果表明,相比CIA和GoDIN这两种完全基于PPI网络的方法,HPMM的精确度与F值更高。     

基于差分进化生物地理学优化的多层感知器训练方法

针对生物地理学优化训练多层感知器存在的早熟收敛以及初始化灵敏等问题,提出一种基于差分进化生物地理学优化的多层感知器训练方法。将生物地理学优化（Biogeography-based Optimization,BBO）与差分进化(Differential Evolution,DE)算法相结合,形成改进的混合DE_BBO算法;采用改进的DE_BBO来训练多层感知器（Multi-Layer Perceptron,MLP）,并应用于虹膜、乳腺癌、输血、钞票验证等4类数据分类。与BBO、PSO、GA、ACO、ES、PBIL等6种主流启发式算法的实验结果进行比较表明,DE_BBO_MLP算法在分类精度和收敛速度等方面优于已有方法。     

一种基于强化规则学习的高效入侵检测方法

在入侵检测研究领域中,提高检测模型的检测率并降低误报率是一个重要的研究课题．在对归纳学习理论深入研究的基础上,将规则学习算法应用到入侵检测建模中．针对审计训练数据不足时出现的检测精度下降的情况,提出了一种基于强化规则学习的高效入侵检测方法EAIDBRL（efficient approach to intrusion detection based on boosting rule learning）．在EAIDBRL方法中,首先调整传统Boosting算法的权重更新过程在各个预测目标类内部进行,以消除退化现象;然后修改传统规则学习算法中规则生长和规则剪枝过程的评价准则函数;最后使用改进后的Boosting算法来增强弱规则学习器对网络审计数据的分类性能．标准入侵检测数据集上的测试结果表明,EAIDBRL方法能够较大地提高传统规则学习检测模型在小样本条件下的入侵检测性能．     

Learning deterministic finite automata with a smart state labeling evolutionary algorithm

Learning a deterministic finite automaton (DFA) from a training set of labeled strings is a hard task that has been much studied within the machine learning community. It is equivalent to learning a regular language by example and has applications in language modeling. In this paper, we describe a novel evolutionary method for learning DFA that evolves only the transition matrix and uses a simple deterministic procedure to optimally assign state labels. We compare its performance with the evidence driven state merging (EDSM) algorithm, one of the most powerful known DFA learning algorithms. We present results on random DFA induction problems of varying target size and training set density. We also study the effects of noisy training data on the evolutionary approach and on EDSM. On noise-free data, we find that our evolutionary method outperforms EDSM on small sparse data sets. In the case of noisy training data, we find that our evolutionary method consistently outperforms EDSM, as well as other significant methods submitted to two recent competitions.     

Learning fuzzy rules for similarity assessment in case-based reasoning

Fundamental to case-based reasoning is the assumption that similar problems have similar solutions. The meaning of the concept of “similarity” can vary in different situations and remains an issue. This paper proposes a novel similarity model consisting of fuzzy rules to represent the semantics and evaluation criteria for similarity. We believe that fuzzy if-then rules present a more powerful and flexible means to capture domain knowledge for utility oriented similarity modeling than traditional similarity measures based on feature weighting. Fuzzy rule-based reasoning is utilized as a case matching mechanism to determine whether and to which extent a known case in the case library is similar to a given problem in query. Further, we explain that such fuzzy rules for similarity assessment can be learned from the case library using genetic algorithms. The key to this is pair-wise comparisons of cases with known solutions in the case library such that sufficient training samples can be derived for genetic-based fuzzy rule learning. The evaluations conducted have shown the superiority of the proposed method in similarity modeling over traditional schemes as well as the feasibility of learning fuzzy similarity rules from a rather small case base while still yielding competent system performance.     

Single imputation with multilayer perceptron and multiple imputation combining multilayer perceptron and k-nearest neighbours for monotone patterns

The knowledge discovery process is supported by data files information gathered from collected data sets, which often contain errors in the form of missing values. Data imputation is the activity aimed at estimating values for missing data items. This study focuses on the development of automated data imputation models, based on artificial neural networks for monotone patterns of missing values. The present work proposes a single imputation approach relying on a multilayer perceptron whose training is conducted with different learning rules, and a multiple imputation approach based on the combination of multilayer perceptron and k-nearest neighbours. Eighteen real and simulated databases were exposed to a perturbation experiment with random generation of monotone missing data pattern. An empirical test was accomplished on these data sets, including both approaches (single and multiple imputations), and three classical single imputation procedures – mean/mode imputation, regression and hot-deck – were also considered. Therefore, the experiments involved five imputation methods. The results, considering different performance measures, demonstrated that, in comparison with traditional tools, both proposals improve the automation level and data quality offering a satisfactory performance.     

Application of the recurrent multilayer perceptron in modelingcomplex process dynamics

A nonlinear dynamic model is developed for a process system, namely a heat exchanger, using the recurrent multilayer perceptron network as the underlying model structure. The perceptron is a dynamic neural network, which appears effective in the input-output modeling of complex process systems. Dynamic gradient descent learning is used to train the recurrent multilayer perceptron, resulting in an order of magnitude improvement in convergence speed over a static learning algorithm used to train the same network. In developing the empirical process model the effects of actuator, process, and sensor noise on the training and testing sets are investigated. Learning and prediction both appear very effective, despite the presence of training and testing set noise, respectively. The recurrent multilayer perceptron appears to learn the deterministic part of a stochastic training set, and it predicts approximately a moving average response of various testing sets. Extensive model validation studies with signals that are encountered in the operation of the process system modeled, that is steps and ramps, indicate that the empirical model can substantially generalize operational transients, including accurate prediction of instabilities not in the training set. However, the accuracy of the model beyond these operational transients has not been investigated. Furthermore, online learning is necessary during some transients and for tracking slowly varying process dynamics. Neural networks based empirical models in some cases appear to provide a serious alternative to first principles models.     

基于LSTM网络的序列标注中文分词法

当前主流的中文分词方法是基于字标注的传统机器学习的方法。但传统机器学习方法需要人为地从中文文本中配置并提取特征,存在词库维度高且仅利用CPU训练模型时间长的缺点。针对以上问题,进行了研究提出基于LSTM（Long Short-Term Memory）网络模型的改进方法,采用不同词位标注集并加入预先训练的字嵌入向量（character embedding）进行中文分词。在中文分词评测常用的语料上进行实验对比,结果表明：基于LSTM网络模型的方法能得到比当前传统机器学习方法更好的性能;采用六词位标注并加入预先训练的字嵌入向量能够取得相对最好的分词性能;而且利用GPU可以大大缩短深度神经网络模型的训练时间;LSTM网络模型的方法也更容易推广并应用到其他自然语言处理（NLP）中序列标注的任务。     

Two algorithms for neural-network design and training withapplication to channel equalization

We describe two algorithms for designing and training neural-network classifiers. The first, the linear programming slab algorithm (LPSA), is motivated by the problem of reconstructing digital signals corrupted by passage through a dispersive channel and by additive noise. It constructs a multilayer perceptron (MLP) to separate two disjoint sets by using linear programming methods to identify network parameters. The second, the perceptron learning slab algorithm (PLSA), avoids the computational costs of linear programming by using an error-correction approach to identify parameters. Both algorithms operate in highly constrained parameter spaces and are able to exploit symmetry in the classification problem. Using these algorithms, we develop a number of procedures for the adaptive equalization of a complex linear 4-quadrature amplitude modulation (QAM) channel, and compare their performance in a simulation study. Results are given for both stationary and time-varying channels, the latter based on the COST 207 GSM propagation model     

Robust TSK fuzzy modeling for function approximation with outliers

The Takagi-Sugeno-Kang (TSK) type of fuzzy models has attracted a great attention of the fuzzy modeling community due to their good performance in various applications. Most approaches for modeling TSK fuzzy rules define their fuzzy subspaces based on the idea of training data being close enough instead of having similar functions. Besides, training data sets algorithms often contain outliers, which seriously affect least-square error minimization clustering and learning algorithms. A robust TSK fuzzy modeling approach is presented. In the approach, a clustering algorithm termed as robust fuzzy regression agglomeration (RFRA) is proposed to define fuzzy subspaces in a fuzzy regression manner with robust capability against outliers. To obtain a more precision model, a robust fine-tuning algorithm is then employed. Various examples are used to verify the effectiveness of the proposed approach. From the simulation results, the proposed robust TSK fuzzy modeling indeed showed superior performance over other approaches     

Improving the interpretability of TSK fuzzy models by combiningglobal learning and local learning

The fuzzy inference system proposed by Takagi, Sugeno, and Kang, known as the TSK model in fuzzy system literature, provides a powerful tool for modeling complex nonlinear systems. Unlike conventional modeling where a single model is used to describe the global behavior of a system, TSK modeling is essentially a multimodel approach in which simple submodels (typically linear models) are combined to describe the global behavior of the system. Most existing learning algorithms for identifying the TSK model are based on minimizing the square of the residual between the overall outputs of the real system and the identified model. Although these algorithms can generate a TSK model with good global performance (i.e., the model is capable of approximating the given system with arbitrary accuracy, provided that sufficient rules are used and sufficient training data are available), they cannot guarantee the resulting model to have a good local performance. Often, the submodels in the TSK model may exhibit an erratic local behavior, which is difficult to interpret. Since one of the important motivations of using the TSK model (also other fuzzy models) is to gain insights into the model, it is important to investigate the interpretability issue of the TSK model. We propose a new learning algorithm that integrates global learning and local learning in a single algorithmic framework. This algorithm uses the idea of local weighed regression and local approximation in nonparametric statistics, but remains the component of global fitting in the existing learning algorithms. The algorithm is capable of adjusting its parameters based on the user's preference, generating models with good tradeoff in terms of global fitting and local interpretation. We illustrate the performance of the proposed algorithm using a motorcycle crash modeling example     

Stabilizing position/force control of robots interacting with environment by learning connectionist structures

This paper is mostly concerned with the application of connectionist architectures for fast on-line learning of robot dynamic uncertainties used at the executive hierarchical control level in robot contact tasks. The connectionist structures are integrated in non-learning control laws for contact tasks which enable stabilization and good tracking performance of position and force. It has been shown that the problem of tracking a specified reference trajectory and specified force profile with a present quality of their transient response can be efficiently solved by means of the application of a four-layer perceptron. A four-layer perceptron is part of a hybrid learning control algorithm through the process of synchronous training which uses fast learning rules and available sensor information in order to improve robotic performance progressively in the minimum possible number of learning epochs. Some simulation results of the deburring process with robot MANUTEC r3 are shown to verify effectiveness of the proposed control learning algorithms.     

Advanced search algorithms for information-theoretic learning with kernel-based estimators

Recent publications have proposed various information-theoretic learning (ITL) criteria based on Renyi's quadratic entropy with nonparametric kernel-based density estimation as alternative performance metrics for both supervised and unsupervised adaptive system training. These metrics, based on entropy and mutual information, take into account higher order statistics unlike the mean-square error (MSE) criterion. The drawback of these information-based metrics is the increased computational complexity, which underscores the importance of efficient training algorithms. In this paper, we examine familiar advanced-parameter search algorithms and propose modifications to allow training of systems with these ITL criteria. The well known algorithms tailored here for ITL include various improved gradient-descent methods, conjugate gradient approaches, and the Levenberg-Marquardt (LM) algorithm. Sample problems and metrics are presented to illustrate the computational efficiency attained by employing the proposed algorithms.