基于粗集理论的机器学习与推理

利用粗集理论探讨机器学习中的几个重要概念及研究方法,提出一种基于祖集理论的推理和学习方法。这种研究方法不仅开拓了一条机器学习的新途径,而且也是从数据中推理决策规则的一种新探索。     

一种新的归纳学习算法——基于特征可分性的归纳学习算法

本文提出了一种新的基于特征可分性的归纳学习算法(SBI)。与现有各种归纳学习算法相比,该方法直接从特征对不同类型的可分性出发,建立可分性判据,然后形成决策树,可对多种概念进行判别。SBI算法具有直观且计算简便等优点。本文以实例表明了SBI算法的有效性。     

Learning hard concepts through constructive induction: framework and rationale

The intrinsic accuracy of an inductive problem is the accuracy achieved by exhaustive table look-up. Intrinsic accuracy is the upper bound for any inductive method. Hard concepts are concepts that have high intrinsic accuracy, but which cannot be learned effectively with traditional inductive methods. To learn hard concepts, we must use constructive induction - methods that create new features. We use measures of concept dispersion to explore (conceptually and empirically) the inherent weaknesses of traditional inductive approaches. These structural defects are buried in the design of the algorithms and prevent the learning of hard concepts. After studying some examples of successful and unsuccessful feature construction ("success" being defined here in terms of accuracy), we introduce a single measure of inductive difficulty that we call variation. We argue for a specific approach to constructive induction that reduces variation by incorporating various kinds of domain knowledge. All of these kinds of domain knowledge boil down to utility invariants, i.e., transformations that group together non-contiguous portions of feature space having similar class-membership values. Utility invariants manifest themselves in various ways: in some cases they exist in the user's stock of domain knowledge, in other cases they may be discovered via methods we describe.     

Selective ensemble extreme learning machine modeling of effluent quality in wastewater treatment plants

Real-time and reliable measurements of the effluent quality are essential to improve operating efficiency and reduce energy consumption for the wastewater treatment process.Due to the low accuracy and unstable performance of the traditional effluent quality measurements,we propose a selective ensemble extreme learning machine modeling method to enhance the effluent quality predictions.Extreme learning machine algorithm is inserted into a selective ensemble frame as the component model since it runs much faster and provides better generalization performance than other popular learning algorithms.Ensemble extreme learning machine models overcome variations in different trials of simulations for single model.Selective ensemble based on genetic algorithm is used to further exclude some bad components from all the available ensembles in order to reduce the computation complexity and improve the generalization performance.The proposed method is verified with the data from an industrial wastewater treatment plant,located in Shenyang,China.Experimental results show that the proposed method has relatively stronger generalization and higher accuracy than partial least square,neural network partial least square,single extreme learning machine and ensemble extreme learning machine model.     

模糊神经网络算法在倒立摆控制中的应用

本文利用一种可以进行结构和参数学习的模糊神经网络成功地控制一级倒立摆，该网络是一种多层前馈网络，它将传统模糊控制器的基本要件综合到网络结构中。从而使该网络既具备神经网络的低级学习能力，从而还具备模糊逻辑系统类似人的高级推理能力。因而，给定训练数据后，该网络不仅可以学习网络参数，同时还可以学习网络结构。结构学习确定了表示了模糊规则和模糊分段数的连接类型以及隐节点数目。对一级倒立摆的实际控制效果可以证明该算法的性能和实用性。     

机器学习算法在中医诊疗中的研究综述

机器学习算法包括传统机器学习算法和深度学习算法。传统机器学习算法在中医诊疗领域中的应用研究较多,为探究中医辩证规律提供了参考,也为中医诊疗过程的客观化提供了依据。与此同时,随着其在多个领域不断取得成功,深度学习算法在中医诊疗中的价值越来越多地得到业界的重视。通过对中医诊疗领域中使用到的传统机器学习算法与深度学习算法进行述评,总结了两类算法在中医领域中的研究与应用现状,分析了两类算法的特点以及对中医的应用价值,以期为机器学习算法在中医诊疗领域的进一步研究提供参考。     

基于波谱库的特色农作物智能专家分类系统设计

农作物的长势监测和产量估算一直是遥感技术应用的重要方面,而一个好的农作物分类算法对于农作物产量和长势进行监测十分关键。目前对于一些特色农作物而言,这方面的研究比较缺乏。因此拳研究设计了符合特色农作物的长势监测和产量测算功能模块,将数据挖掘和知识发现应用到专家分类算法中,自行开发了适合农作物数据发现和挖掘的归纳学习算法,充分利用了波谱库中大量的波谱数据、相关属性和空间数据,形成了基于波谱库的特色农作物智能专家分类系统。     

规则向量投影算法──一种归纳机器学习方法

为了支持在事实不完全或不充分环境中的有效推理，作者提出了一种归纳机器学习方法，并设计了一个规则向量投影算法，使用木文介绍的算法可对原始知识实行归纳，生成含一系列全新分类概念和推理路经的网络知识库，基于该知识库的机器推理系统，在作出诊断决策时所需事实量可大为减少，因此在信息量不足的情况下仍能具有很高的推理性能．     

Nonlinear phenomena in spaces of algorithms

Nonlinear phenomena, which are so important in nature and society, are considered here in relation to the world of algorithms and computations. To have a mathematical model for this world, formal computability spaces are introduced. It is demonstrated that the traditional approach to algorithms, which is based on such popular models as Turing machines, results in linear subspaces of the computability space. Nonlinear phenomena appear when we go to the more powerful class of such super-recursive algorithms as inductive Turing machines. It is demonstrated how this nonlinearity imports much higher computing power of inductive Turing machines in comparison with conventional Turing machines. This provides a base to consider problems of chaos, emergent computations and infinity from the algorithmic point of view.     

基于变权重迁移学习的BN参数学习算法

针对小数据集条件下的贝叶斯网络(Bayesian network,BN)参数估计困难问题,提出了一种基于变权重迁移学习(DWTL)的BN参数学习算法。首先,利用MAP和MLE方法学习得到目标域初始参数和各源域参数;然后根据不同源域数据样本贡献的不同计算源权重因子;接着基于目标域样本统计量与小数据集样本阈值的关系设计了目标域初始参数和源域参数的平衡系数;最后,基于上述参数、源权重因子和平衡系数计算得到新的目标参数。在实验研究中,通过对经典BN模型的参数学习问题验证了DWTL算法的有效性;针对小数据集下的轴承故障诊断问题,相较于传统迁移学习(LP)算法,DWTL算法学习精度提高了10%。实验结果表明:所提出的算法能够较好地解决样本数据集在相对稀缺条件下的目标参数建模问题。     

Learning Concepts by Arranging Appropriate Training Order

Machine learning has been proven useful for solving the bottlenecks in building expert systems. Noise in the training instances will, however, confuse a learning mechanism. Two main steps are adopted here to solve this problem. The first step is to appropriately arrange the training order of the instances. It is well known from Psychology that different orders of presentation of the same set of training instances to a human may cause different learning results. This idea is used here for machine learning and an order arrangement scheme is proposed. The second step is to modify a conventional noise-free learning algorithm, thus making it suitable for noisy environment. The generalized version space learning algorithm is then adopted to process the training instances for deriving good concepts. Finally, experiments on the Iris Flower problem show that the new scheme can produce a good training order, allowing the generalized version space algorithm to have a satisfactory learning result.     

一种基于强化规则学习的高效入侵检测方法

在入侵检测研究领域中，提高检测模型的检测率并降低误报率是一个重要的研究课题．在对归纳学习理论深入研究的基础上，将规则学习算法应用到入侵检测建模中．针对审计训练数据不足时出现的检测精度下降的情况，提出了一种基于强化规则学习的高效入侵检测方法EAIDBRL（efficient approach to intrusion detection based on boosting rule learning）．在EAIDBRL方法中，首先调整传统Boosting算法的权重更新过程在各个预测目标类内部进行，以消除退化现象；然后修改传统规则学习算法中规则生长和规则剪枝过程的评价准则函数；最后使用改进后的Boosting算法来增强弱规则学习器对网络审计数据的分类性能．标准入侵检测数据集上的测试结果表明，EAIDBRL方法能够较大地提高传统规则学习检测模型在小样本条件下的入侵检测性能．     

Convex multi-task feature learning

We present a method for learning sparse representations shared across multiple tasks. This method is a generalization of the well-known single-task 1-norm regularization. It is based on a novel non-convex regularizer which controls the number of learned features common across the tasks. We prove that the method is equivalent to solving a convex optimization problem for which there is an iterative algorithm which converges to an optimal solution. The algorithm has a simple interpretation: it alternately performs a supervised and an unsupervised step, where in the former step it learns task-specific functions and in the latter step it learns common-across-tasks sparse representations for these functions. We also provide an extension of the algorithm which learns sparse nonlinear representations using kernels. We report experiments on simulated and real data sets which demonstrate that the proposed method can both improve the performance relative to learning each task independently and lead to a few learned features common across related tasks. Our algorithm can also be used, as a special case, to simply select—not learn—a few common variables across the tasks. Editors: Daniel Silver, Kristin Bennett, Richard Caruana. This is a longer version of the conference paper (Argyriou et al. in Advances in neural information processing systems, vol. 19, 2007a). It includes new theoretical and experimental results.     

基于支持向量机的学习评价系统

在分析学生学习行为的基础上，文章提出了支持向量机的学习评估系统的设计方案并进行了模拟，结果显示该系统能够有效地实现对学生的学习指导，同时给教师的教学提供了一个有力的教学辅助工具。     

Incremental Abductive EBL

In previous work, we described a knowledge-intensive inductive learning algorithm called abductive explanation-based learning (A-EBL) that uses background knowledge to improve the performance of a concept learner. A disadvantage of A-EBL is that it is not incremental. This article describes an alternative learning algorithm called IA-EBL that learns incrementally; IA-EBL replaces the set-cover-based learning algorithm of A-EBL with an extension of a perceptron learning algorithm. IA-EBL is in most other respects comparable to A-EBL, except that the output of the learning system can no longer be easily expressed as a logical theory. In this article, IA-EBL is described, analyzed according to Littlestone's model of mistake-bounded learnability, and finally compared experimentally to A-EBL. IA-EBL is shown to provide order-of-magnitude speedups over A-EBL in two domains when used in an incremental setting.     

Neighborhood preserving regression for image retrieval

In content-based image retrieval (CBIR), relevance feedback has been proven to be a powerful tool for bridging the gap between low level visual features and high level semantic concepts. Traditionally, relevance feedback driven CBIR is often considered as a supervised learning problem where the user provided feedbacks are used to learn a distance metric or classification function. However, CBIR is intrinsically a semi-supervised learning problem in which the testing samples (images in the database) are present during the learning process. Moreover, when there are no sufficient feedbacks, these methods may suffer from the overfitting problem. In this paper, we propose a novel neighborhood preserving regression algorithm which makes efficient use of both labeled and unlabeled images. By using the unlabeled images, the geometrical structure of the image space can be incorporated into the learning system through a regularizer. Specifically, from all the functions which minimize the empirical loss on the labeled images, we select the one which best preserves the local neighborhood structure of the image space. In this way, our method can obtain a regression function which respects both semantic and geometrical structures of the image database. We present experimental evidence suggesting that our algorithm is able to use unlabeled data effectively for image retrieval.     

SLAVE: a genetic learning system based on an iterative approach

SLAVE is an inductive learning algorithm that uses concepts based on fuzzy logic theory. This theory has been shown to be a useful representational tool for improving the understanding of the knowledge obtained from a human point of view. Furthermore, SLAVE uses an iterative approach for learning based on the use of a genetic algorithm (GA) as a search algorithm. We propose a modification of the initial iterative approach used in SLAVE. The main idea is to include more information in the process of learning one individual rule. This information is included in the iterative approach through a different proposal of calculus of the positive and negative example to a rule. Furthermore, we propose the use of a new fitness function and additional genetic operators that reduce the time needed for learning and improve the understanding of the rules obtained     

Multimedia event detection with multimodal feature fusion and temporal concept localization

We present a system for multimedia event detection. The developed system characterizes complex multimedia events based on a large array of multimodal features, and classifies unseen videos by effectively fusing diverse responses. We present three major technical innovations. First, we explore novel visual and audio features across multiple semantic granularities, including building, often in an unsupervised manner, mid-level and high-level features upon low-level features to enable semantic understanding. Second, we show a novel Latent SVM model which learns and localizes discriminative high-level concepts in cluttered video sequences. In addition to improving detection accuracy beyond existing approaches, it enables a unique summary for every retrieval by its use of high-level concepts and temporal evidence localization. The resulting summary provides some transparency into why the system classified the video as it did. Finally, we present novel fusion learning algorithms and our methodology to improve fusion learning under limited training data condition. Thorough evaluation on a large TRECVID MED 2011 dataset showcases the benefits of the presented system.