Rough sets for adapting wavelet neural networks as a new classifier system

Classification is an important theme in data mining. Rough sets and neural networks are two techniques applied to data mining problems. Wavelet neural networks have recently attracted great interest because of their advantages over conventional neural networks as they are universal approximations and achieve faster convergence. This paper presents a hybrid system to extract efficiently classification rules from decision table. The neurons of such hybrid network instantiate approximate reasoning knowledge gleaned from input data. The new model uses rough set theory to help in decreasing the computational effort needed for building the network structure by using what is called reduct algorithm and a rules set (knowledge) is generated from the decision table. By applying the wavelets, frequencies analysis, rough sets and dynamic scaling in connection with neural network, novel and reliable classifier architecture is obtained and its effectiveness is verified by the experiments comparing with traditional rough set and neural networks approaches.     

控制增益为未知函数的不确定系统预设性能反演控制

对一类控制增益为未知函数的不确定严格反馈系统的预设性能反演控制进行研究.首先,提出一种新的变参数约束方案,放宽了对初始跟踪误差已知的限制,并通过误差转化将不等 式约束的受限系统转化为非受限系统.随后,通过引入积分型Lyapunov函数,避免了因控制增益未知而引起的系统奇异问题.最后,综合应用自适应技术、径向基函数(Radial basis function,RBF)神经网络和反演控制技术完成了控制器的设计,系统中的未知函数利用RBF神经网络直接进行逼近.所设计的控制器能够满足预设性能的要求,且保证闭环系统所有的状态量有界.仿真研究证明了控制器设计方法的有效性.     

稳态在线数据校正在炼油厂气体分离装置上的应用

文章系统地研究了稳态过程在线数据校正技术，在具体实现中采用均值法进行稳态检测，修正系数法进行误差的侦破、识别，通过两层次变换进行数据分类。开发了稳态过程在线数据校正软件，并将校正后的数据作为输入值用于某炼油厂气体分离系统产品质量的在线预测，应用结果表明，采用校正后的数据作为输入值进行产品质量在线预测比直接用原始数据更稳定、更符合实际情况。     

Morphological associative memories

The theory of artificial neural networks has been successfully applied to a wide variety of pattern recognition problems. In this theory, the first step in computing the next state of a neuron or in performing the next layer neural network computation involves the linear operation of multiplying neural values by their synaptic strengths and adding the results. A nonlinear activation function usually follows the linear operation in order to provide for nonlinearity of the network and set the next state of the neuron. In this paper we introduce a novel class of artificial neural networks, called morphological neural networks, in which the operations of multiplication and addition are replaced by addition and maximum (or minimum), respectively. By taking the maximum (or minimum) of sums instead of the sum of products, morphological network computation is nonlinear before possible application of a nonlinear activation function. As a consequence, the properties of morphological neural networks are drastically different than those of traditional neural network models. The main emphasis of the research presented here is on morphological associative memories. We examine the computing and storage capabilities of morphological associative memories and discuss differences between morphological models and traditional semilinear models such as the Hopfield net.     

GenSoFNN: a generic self-organizing fuzzy neural network

Existing neural fuzzy (neuro-fuzzy) networks proposed in the literature can be broadly classified into two groups. The first group is essentially fuzzy systems with self-tuning capabilities and requires an initial rule base to be specified prior to training. The second group of neural fuzzy networks, on the other hand, is able to automatically formulate the fuzzy rules from the numerical training data. No initial rule base needs to be specified prior to training. A cluster analysis is first performed on the training data and the fuzzy rules are subsequently derived through the proper connections of these computed clusters. However, most existing neural fuzzy systems (whether they belong to the first or second group) encountered one or more of the following major problems. They are (1) inconsistent rule-base; (2) heuristically defined node operations; (3) susceptibility to noisy training data and the stability-plasticity dilemma; and (4) needs for prior knowledge such as the number of clusters to be computed. Hence, a novel neural fuzzy system that is immune to the above-mentioned deficiencies is proposed in this paper. This new neural fuzzy system is named the generic self-organizing fuzzy neural network (GenSoFNN). The GenSoFNN network has strong noise tolerance capability by employing a new clustering technique known as discrete incremental clustering (DIC). The fuzzy rule base of the GenSoFNN network is consistent and compact as GenSoFNN has built-in mechanisms to identify and prune redundant and/or obsolete rules. Extensive simulations were conducted using the proposed GenSoFNN network and its performance is encouraging when benchmarked against other neural and neural fuzzy systems.     

基于扩展BP神经网络的一类非线性系统自适应控制设计

针对单输入单输出非线性系统的不确定性问题,提出了一种新型的基于扩展反向传播（BP）神经网络的自适应控制方法。首先,采用离线数据来训练BP神经网络的权值向量;然后,通过在线调节伸缩因子和逼近精度估计值的更新律,从而来达到控制整个系统的目的。在控制器的设计过程中,利用李亚普诺夫稳定性分析原理,保证了闭环系统的所有状态一致终极有界（UUB）。相比传统的BP神经网络自适应控制,所提方法能有效地减少在线调节的参数数目、减轻计算负担。仿真结果表明,该方法能够使闭环系统的所有状态都趋于零,即系统达到稳定状态。     

粒子群优化RBF神经网络光伏电池建模研究

本文研究神经网络在光伏电池建模优化问题。由于光伏电池具有高度非线性特性,其输出功率受到外界自然因素的影响,使得传统方法不能满足光伏控制系统动态要求。针对上述问题,本文提出一种粒子群优化的神经网络光伏电池建模算法。改进的方法以日照、温度和负载电压作为提出的RBF神经网络模型的输入值,把光伏电池的输出功率作为神经网络的输出,采用RBF神经网络对光伏电池进行建模,同时利用粒子群算法对神经网络参数进行优化,最后建立光伏电池的动态响应模型。仿真实验结果证明,所提模型更好地克服传统方法的缺点,收敛速度快,具有较高的预测精度和适合能力。     

From local neural networks to granular neural networks: A study in information granulation

In this study, we are concerned with a construction of granular neural networks (GNNs)—architectures formed as a direct result reconciliation of results produced by a collection of local neural networks constructed on a basis of individual data sets. Being cognizant of the diversity of the results produced by the collection of networks, we arrive at the concept of granular neural network, producing results in the form of information granules (rather than plain numeric entities) that become reflective of the diversity of the results generated by the contributing networks. The design of a granular neural network exploits the concept of justifiable granularity. Introduced is a performance index quantifying the quality of information granules generated by the granular neural network. This study is illustrated with the aid of machine learning data sets. The experimental results provide a detailed insight into the developed granular neural networks.     

Autonomous Learning System Towards Mobile Intelligence

How to efficiently deploy machine learning models on mobile devices has drawn a lot of attention in both academia and industries, among which the model training is a critical part. However, with increasingly public attention on data privacy and the recently adopted laws and regulations, it becomes harder for developers to collect training data from users and thus they cannot train high-quality models. Researchers have been exploring approaches to training neural networks on decentralized data. Those efforts will be summarized and their limitations be pointed out. To this end, this work presents a novel neural network training paradigm on mobile devices, which distributes all training computations associated with private data on local devices and requires no data to be uploaded in any form. Such training paradigm is named autonomous learning. To deal with two main challenges of autonomous learning, i.e., limited data volume and insufficient computing power available on mobile devices, this paper designs and implements the first autonomous learning system AutLearn. It incorporates the cloud (public data and pre-training)--client (private data and transfer learning) cooperation methodology and data augmentation techniques to ensure the model convergence on mobile devices. Furthermore, by optimization techniques such as model compression, neural network compiler, and runtime cache reuse, AutLearn can significantly reduce the on-client training cost. Two classical scenarios of autonomous learning are implemented based on AutLearn,with a set of experiments carried out. The results show that AutLearn can train the neural networks with comparable or even higher accuracy compared to traditional centralized/federated training mode with privacy preserved. AutLearn can also remarkably cut the computational and energy cost of neural network training on mobile devices.     

Dynamic Ridge Polynomial Neural Network: Forecasting the univariate non-stationary and stationary trading signals

This paper considers the prediction of noisy time series data, specifically, the prediction of financial signals. A novel Dynamic Ridge Polynomial Neural Network (DRPNN) for financial time series prediction is presented which combines the properties of both higher order and recurrent neural network. In an attempt to overcome the stability and convergence problems in the proposed DRPNN, the stability convergence of DRPNN is derived to ensure that the network posses a unique equilibrium state. In order to provide a more accurate comparative evaluation in terms of profit earning, empirical testing used in this work encompass not only on the more traditional criteria of NMSE, which concerned at how good the forecasts fit their target, but also on financial metrics where the objective is to use the networks predictions to generate profit. Extensive simulations for the prediction of one and five steps ahead of stationary and non-stationary time series were performed. The resulting forecast made by DRPNN shows substantial profits on financial historical signals when compared to various neural networks; the Pi-Sigma Neural Network, the Functional Link Neural Network, the feedforward Ridge Polynomial Neural Network, and the Multilayer Perceptron. Simulation results indicate that DRPNN in most cases demonstrated advantages in capturing chaotic movement in the financial signals with an improvement in the profit return and rapid convergence over other network models.     

Data-driven soft sensor development based on deep learning technique

In industrial process control, some product qualities and key variables are always difficult to measure online due to technical or economic limitations. As an effective solution, data-driven soft sensors provide stable and reliable online estimation of these variables based on historical measurements of easy-to-measure process variables. Deep learning, as a novel training strategy for deep neural networks, has recently become a popular data-driven approach in the area of machine learning. In the present study, the deep learning technique is employed to build soft sensors and applied to an industrial case to estimate the heavy diesel 95% cut point of a crude distillation unit (CDU). The comparison of modeling results demonstrates that the deep learning technique is especially suitable for soft sensor modeling because of the following advantages over traditional methods. First, with a complex multi-layer structure, the deep neural network is able to contain richer information and yield improved representation ability compared with traditional data-driven models. Second, deep neural networks are established as latent variable models that help to describe highly correlated process variables. Third, the deep learning is semi-supervised so that all available process data can be utilized. Fourth, the deep learning technique is particularly efficient dealing with massive data in practice.     

A self-adaptive data analysis for fault diagnosis of an automotive air-conditioner blower

This paper presents a fault diagnosis system for an automotive air-conditioner blower based on a noise emission signal using a self-adaptive data analysis technique. The proposed diagnosis system consists of feature extraction using the empirical mode decomposition (EMD) method and fault classification using the artificial neural network technique. The EMD method has been developed quite recently to adaptively decompose the non-stationary and non-linear signals. It sifts the complex signal of time series without losing its original properties and then obtains some useful intrinsic mode function (IMF) components. Calculating the energy of each component can reduce the computation dimensions and enhance classification performance. These energy features of various fault conditions are used as inputs to train the artificial neural network. In the fault classification, the probabilistic neural network (PNN) is used to verify the performance of the proposed system and compare with the traditional technique, back-propagation neural network (BPNN). The experimental results indicated the proposed technique performed well for quickly and accurately estimating fault conditions.     

采用遗传算法训练对角递归神经网络预测控制器

本文提出了一种基于广义预测控制的神经网络预测控制方案．预测控制器由对角递归 神经网络预测控制器和前向神经网络静态补偿器组成．两种神经网络均采用遗传算法进行训 练．仿真实验表明,对于带纯时延的非线性被控对象,采用遗传算法设计的对角递归神经网 络预测控制器具有令人满意的控制性能．     

积分法动态数据校正技术的研究

基于积分法的动态数据校正具有简单、快速和适于在线应用的优点。本文对积分法动态数据校正技术的原理及其应用方法进行了研究。研究结果表明,该方法不要求有状态空间模型,能够充分利用整个时间轴的时间冗余信息；但积分法中的区间长度对其校正精度有影响,因此,采用该方法进行校正时应首先确定适宜的区间长度。将积分法应用于常减压炼油装置拟稳态过程的数据校正,计算结果表明该方法的计算精度高于稳态数据校正。     

Method of predicting the position of the energy center of the image of a laser beam using a parallel–hierarchical network

This paper presents a prediction method using a parallel–hierarchical (PH) network and hyperbolic smoothing of empirical data. The average prediction error is 0.55% for the developed method and 1.62% for neural networks; therefore, this method is more efficient as applied to real-time systems than traditional neural networks due to the use of the PH network and hyperbolic smoothing in implementing the operation of predicting the positions of energy centers of laser beam spot images for optical communication systems.     

The nature of unsupervised learning in deep neural networks: A new understanding and novel approach

Over the last decade, the deep neural networks are a hot topic in machine learning. It is breakthrough technology in processing images, video, speech, text and audio. Deep neural network permits us to overcome some limitations of a shallow neural network due to its deep architecture. In this paper we investigate the nature of unsupervised learning in restricted Boltzmann machine. We have proved that maximization of the log-likelihood input data distribution of restricted Boltzmann machine is equivalent to minimizing the cross-entropy and to special case of minimizing the mean squared error. Thus the nature of unsupervised learning is invariant to different training criteria. As a result we propose a new technique called “REBA” for the unsupervised training of deep neural networks. In contrast to Hinton’s conventional approach to the learning of restricted Boltzmann machine, which is based on linear nature of training rule, the proposed technique is founded on nonlinear training rule. We have shown that the classical equations for RBM learning are a special case of the proposed technique. As a result the proposed approach is more universal in contrast to the traditional energy-based model. We demonstrate the performance of the REBA technique using wellknown benchmark problem. The main contribution of this paper is a novel view and new understanding of an unsupervised learning in deep neural networks.     

神经网络集成的城市道路状态判别模型研究

针对城市道路交通状态影响因素多、判别难的特点,在分析K-均值聚类算法和概率神经网络（PNN）的基础上,利用多源检测信息的互补性,提出一种基于快速全局聚类分析的概率神经网络集成模型,通过聚类提高集成网络间的差异度,同时利用主成分分析（PCA）优化概率神经网络结构,仿真实验表明该模型与传统的集成方法Bagging相比,能够利用更简单的网络结构,快速有效地识别出城市道路交通状态,为交通预警和诱导策略的制定提供数据依据。     

Ensemble of Evolving Neural Networks in Classification

The ensemble of evolving neural networks, which employs neural networks and genetic algorithms, is developed for classification problems in data mining. This network meets data mining requirements such as smart architecture, user interaction, and performance. The evolving neural network has a smart architecture in that it is able to select inputs from the environment and controls its topology. A built-in objective function of the network offers user interaction for customized classification. The bagging technique, which uses a portion of the training set in multiple networks, is applied to the ensemble of evolving neural networks in order to improve classification performance. The ensemble of evolving neural networks is tested by various data sets and produces better performance than both classical neural networks and simple ensemble methods.     

An insect classification analysis based on shape features using quality threshold ARTMAP and moment invariant

The main objective of this paper is to investigate the use of Quality Threshold ARTMAP (QTAM) neural network in classifying the feature vectors generated by moment invariant for the insect recognition task. In this work, six different types of moment invariant technique are adopted to extract the shape features of the insect images. These moment techniques are Geometrical Moment Invariant (GMI), United Moment Invariant (UMI), Zernike Moment Invariant (ZMI), Legendre Moment Invariant (LMI), Tchebichef Moment Invariant (TMI) and Krawtchouk Moment Invariant (KMI). All the moment techniques are analyzed using the concept of intraclass and interclass analysis. In intraclass analysis, several computation methods are introduced in order to examine the invariance properties of adopted moment techniques for the same insect object. Meanwhile, the classification accuracy of neural networks is adopted to measure the interclass characteristic and the effectiveness of moment technique in extracting the shape features of insect images. Other types of neural networks are also utilized in this research work. This includes novel enhancement technique based on the Gaussian and Mahalanobis function that design to increase its prediction accuracy. All the other networks used to classify the feature vectors are based on the Fuzzy ARTMAP (FAM) neural network. The experimental results indicated that the Krawtchouk Moment Invariant technique generated the highest classification accuracy for most of the networks used and generated the smallest error for the intraclass analysis. Using different normalization technique, the Quality Threshold ARTMAP and Mahalanobis distance function (QTAM-m) network gave the highest insect recognition results when compared to other networks.     

Reliability analysis for a single-area generation system

In the long-range system transmission expansion planning, the goal is to select the most desirable transmission network, given a generation expansion pattern and projected demand. A basic problem in transmission planning is the determination of adequate capacity with forced outage of various system components.

In a single-area generation system, only single source and single demand are considered. The objective of this research is to develop a suitable technique for determining whether the state of a system is acceptable or not after a disturbance has reached steady state in transmission planning problems. An acceptable state implies that no system component is overloaded and that all demands are met. In this paper, a solution procedure for single-area reliability analysis is presented. This algorithm is written in Fortran and run on a VAX/VMS system.