Robust fuzzy clustering neural network based on ɛ-insensitive loss function

In the paper, as an improvement of fuzzy clustering neural network FCNN proposed by Zhang et al., a novel robust fuzzy clustering neural network RFCNN is presented to cope with the sensitive issue of clustering when outliers exist. This new algorithm is based on Vapnik's ɛ-insensitive loss function and quadratic programming optimization. Our experimental results demonstrate that RFCNN has much better robustness for outliers than FCNN.     

Intelligent complementary sliding-mode control with dead-zone parameter modification

This paper proposes an intelligent complementary sliding-mode control (ICSMC) system which is composed of a computed controller and a robust controller. The computed controller includes a neural dynamics estimator and the robust compensator is designed to prove a finite L₂-gain property. The neural dynamics estimator uses a recurrent neural fuzzy inference network (RNFIN) to approximate the unknown system term in the sense of the Lyapunov function. In traditional neural network learning process, an over-trained neural network would force the parameters to drift and the system may become unstable eventually. To resolve this problem, a dead-zone parameter modification is proposed for the parameter tuning process to stop when tracking performance index is smaller than performance threshold. To investigate the capabilities of the proposed ICSMC approach, the ICSMC system is applied to a one-link robotic manipulator and a DC motor driver. The simulation and experimental results show that favorable control performance can be achieved in the sense of the L₂-gain robust control approach by the proposed ICSMC scheme.     

Robust learning algorithm for multiplicative neuron model artificial neural networks

The two most commonly used types of artificial neural networks (ANNs) are the multilayer feed-forward and multiplicative neuron model ANNs. In the literature, although there is a robust learning algorithm for the former, there is no such algorithm for the latter. Because of its multiplicative structure, the performance of multiplicative neuron model ANNs is affected negatively when the dataset has outliers. On this issue, a robust learning algorithm for the multiplicative neuron model ANNs is proposed that uses Huber's loss function as fitness function. The training of the multiplicative neuron model is performed using particle swarm optimization. One principle advantage of this algorithm is that the parameter of the scale estimator, which is an important factor affecting the value of Huber's loss function, is also estimated with the proposed algorithm. To evaluate the performance of the proposed method, it is applied to two well-known real world time series datasets, and also a simulation study is performed. The algorithm has superior performance both when it is applied to real world time series datasets and the simulation study when compared with other ANNs reported in the literature. Another of its advantages is that, for datasets with outliers, the results are very close to the results obtained from the original datasets. In other words, we demonstrate that the algorithm is unaffected by outliers and has a robust structure.     

RBF neural network based on q-Gaussian function in function approximation

To enhance the generalization performance of radial basis function (RBF) neural networks, an RBF neural network based on a q-Gaussian function is proposed. A q-Gaussian function is chosen as the radial basis function of the RBF neural network, and a particle swarm optimization algorithm is employed to select the parameters of the network. The non-extensive entropic index q is encoded in the particle and adjusted adaptively in the evolutionary process of population. Simulation results of the function approximation indicate that an RBF neural network based on q-Gaussian function achieves the best generalization performance.     

Effective recognition of MCCs in mammograms using an improved neural classifier

Computer-aided diagnosis is one of the most important engineering applications of artificial intelligence. In this paper, early detection of breast cancer through classification of microcalcification clusters from mammograms is emphasized. Although artificial neural network (ANN) has been widely applied in this area, the average accuracy achieved is only around 80% in terms of the area under the receiver operating characteristic curve A_z. This performance may become much worse when the training samples are imbalanced. As a result, an improved neural classifier is proposed, in which balanced learning with optimized decision making are introduced to enable effective learning from imbalanced samples. When the proposed learning strategy is applied to individual classifiers, the results on the DDSM database have demonstrated that the performance from has been significantly improved. An average improvement of more than 10% in the measurements of F₁ score and A_z has fully validated the effectiveness of our proposed method for the successful classification of clustered microcalcifications.     

基于似然损失函数的组样本排序学习方法*

组样本用于模型训练,为排序学习方法的构造提供一种新的思路.文中改进已有的组样本排序学习方法,构造组样本损失函数,用于排序学习模型的训练.基于似然损失函数,采用样本偏序权重损失函数和最优初始序列选择方法,构造基于神经网络的组排序学习方法,实验证明文中方法能够有效提高排序准确率.     

带比例因子的卷积神经网络压缩方法

针对卷积神经网络在图像分类任务中,分类准确率高但实时性差的问题。提出了一种含比例因子的“知识提取”算法。此方法在已有的“知识提取”算法上,加入了衡量样本类间相近关系的比例因子,充实了网络压缩手段,使得神经网络可以更精确地进行“知识提取”。其原理是将比例因子误差值作为代价函数的一部分参与训练调节神经网络的神经元参数,进而使得神经网络的泛化能力更加趋近于具有更好分类表现能力的压缩参考网络。结果表明,含比例因子的神经网络压缩算法可以更细致地刻画训练集的类间相近关系,拥有比原“知识提取”算法更好的训练性能,进而训练出泛化性能更强、精度更高的神经网络。实现了在网络分类准确率下降尽量小的前提下,较大程度地减少神经网络的分类耗时,以达到网络压缩的目的。     

Robust neural network control system design for linear ultrasonic motor

Linear ultrasonic motor (LUSM) has much merit, such as high precision, fast control dynamics and large driving force, etc.; however, the dynamic characteristic of LUSM is nonlinear and the precise dynamic model of LUSM is difficult to obtain. To tackle this problem, this study presents a robust neural network control (RNNC) system for LUSM to track a reference trajectory with L ₂ robust tracking performance. The developed RNNC system is composed of a neural network controller and a robust controller. The neural network controller is the principal controller used to mimic an ideal controller and the robust controller is adopted to achieve L ₂ robust tracking performance. The developed RNNC system is then applied to control an LUSM. Experimental results show that the developed RNNC system can achieve favorable tracking performance with unknown of LUSM model.     

基于RNN的低压变压器区域日线损率基准测试

输变电线路损耗是输变电阶段的固有现象,其是评价低压变压器区域日线损率的重要指标和基准。当对大数据样本上进行训练时,区域的数量通常非常大,并且线损率数据集包含大量的异常值。为了准确的计算低压变压器区域日线损率,提出了一种具有去噪自动编码器(DAE)多径网络模型的鲁棒神经网络(RNN)方法,利用丢包层、L2正则论和Huber损失函数的优点获得多种不同的输出,并利用比较结果计算出基准值和合理区间,实现了精确评估采样数据集的质量并消除线损率的异常值,从而提高数据检测的稳定性。通过与传统的机器学习模型相比,所提出的RNN具有较好的鲁棒性和准确性。根据所提出的RNN的最终结果,在整个数据点中约有13%的异常值,一个月内线损率无缺失值和异常值的区域仅占20%左右,说明了计电设备可靠性较低。     

Robust multilayer neural network based on median neuron model

Multilayer perceptron has been widely used in time series forecasting for last two decades. However, it is a well-known fact that the forecasting performance of multilayer perceptron is negatively affected when data have outliers and this is an important problem. In recent years, some alternative neuron models such as generalized-mean neuron, geometric mean neuron, and single multiplicative neuron have been also proposed in the literature. However, it is expected that forecasting performance of artificial neural network approaches based on these neuron models can be also negatively affected by outliers since the aggregation function employed in these models is based on mean value. In this study, a new multilayer feed forward neural network, which is called median neuron model multilayer feed forward (MNM-MFF) model, is proposed in order to deal with this problem caused by outliers and to reach high accuracy level. In the proposed model, unlike other models suggested in the literature, MNM which has median-based aggregation function is employed. MNM is also firstly defined in this study. MNM-MFF is a robust neural network method since aggregation functions in MNM-MFF are based on median, which is not affected much by outliers. In addition, to train MNM-MFF model, particle swarm optimization method was utilized. MNM-MFF was applied to two well-known time series in order to evaluate the performance of the proposed approach. As a result of the implementation, it was observed that the proposed MNM-MFF model has high forecasting accuracy and it is not affected by outlier as much as multilayer perceptron model. Proposed method brings improvement in 7 % for data without outlier, in 90 % for data with outlier, in 95 % for data with bigger outlier.     

Kernelized LARS–LASSO for constructing radial basis function neural networks

Model structure selection is of crucial importance in radial basis function (RBF) neural networks. Existing model structure selection algorithms are essentially forward selection or backward elimination methods that may lead to sub-optimal models. This paper proposes an alternative selection procedure based on the kernelized least angle regression (LARS)–least absolute shrinkage and selection operator (LASSO) method. By formulating the RBF neural network as a linear-in-the-parameters model, we derive a l ₁-constrained objective function for training the network. The proposed algorithm makes it possible to dynamically drop a previously selected regressor term that is insignificant. Furthermore, inspired by the idea of LARS, the computing of output weights in our algorithm is greatly simplified. Since our proposed algorithm can simultaneously conduct model structure selection and parameter optimization, a network with better generalization performance is built. Computational experiments with artificial and real world data confirm the efficacy of the proposed algorithm.     

面向图像场景转换的改进型生成对抗网络

设计了新的生成器网络、判决器网络以及新的损失函数,用于图像场景转换.首先,生成器网络采用了带跨层连接结构的深度卷积神经网络,其中,多个跨层连接以实现图像结构信息的共享;而判决器网络采用了多尺度全域卷积网络,多尺度判决器可以区分不同尺寸下的真实和生成图像.同时,对于损失函数,该算法借鉴其他算法提出了4种损失函数的组合,并通过实验对比证明了新损失函数的有效性,包括GAN损失、L₁损失、VGG损失、FM损失.从实验结果显示,该算法能够实现多种转换,且转换后图像的细节保留较为完整,生成图像较为真实,明显消除了块效应.     

一种新颖的神经网络稳健估计方法

当神经网络应用于实际工程问题时,网络的训练数据集成或多或少都有噪声或异常值掺入其中,为了使网络具有更好的稳健性,文中根据稳健统计学原理,针对前馈神经网络（ＦＮＮ）提出了一种稳健估计（ＲＥ）函数作为新的网络目标函数。     

Semisupervised Encrypted Traffic Identification Based on Auxiliary Classification Generative Adversarial Network

The rapidly increasing popularity of mobile devices has changed the methods with which people access various network services and increased network traffic markedly. Over the past few decades, network traffic identification has been a research hotspot in the field of network management and security monitoring. However, as more network services use encryption technology, network traffic identification faces many challenges. Although classic machine learning methods can solve many problems that cannot be solved by port- and payload-based methods, manually extract features that are frequently updated is time-consuming and labor-intensive. Deep learning has good automatic feature learning capabilities and is an ideal method for network traffic identification, particularly encrypted traffic identification; Existing recognition methods based on deep learning primarily use supervised learning methods and rely on many labeled samples. However, in real scenarios, labeled samples are often difficult to obtain. This paper adjusts the structure of the auxiliary classification generation adversarial network (ACGAN) so that it can use unlabeled samples for training, and use the wasserstein distance instead of the original cross entropy as the loss function to achieve semisupervised learning. Experimental results show that the identification accuracy of ISCX and USTC data sets using the proposed method yields markedly better performance when the number of labeled samples is small compared to that of convolutional neural network (CNN) based classifier.     

Enhanced neural gas network for prototype-based clustering

In practical cluster analysis tasks, an efficient clustering algorithm should be less sensitive to parameter configurations and tolerate the existence of outliers. Based on the neural gas (NG) network framework, we propose an efficient prototype-based clustering (PBC) algorithm called enhanced neural gas (ENG) network. Several problems associated with the traditional PBC algorithms and original NG algorithm such as sensitivity to initialization, sensitivity to input sequence ordering and the adverse influence from outliers can be effectively tackled in our new scheme. In addition, our new algorithm can establish the topology relationships among the prototypes and all topology-wise badly located prototypes can be relocated to represent more meaningful regions. Experimental results¹on synthetic and UCI datasets show that our algorithm possesses superior performance in comparison to several PBC algorithms and their improved variants, such as hard c-means, fuzzy c-means, NG, fuzzy possibilistic c-means, credibilistic fuzzy c-means, hard/fuzzy robust clustering and alternative hard/fuzzy c-means, in static data clustering tasks with a fixed number of prototypes.     

基于指数移动平均知识蒸馏的神经网络低比特量化方法

目前存储和计算成本严重阻碍深度神经网络应用和推广,而神经网络量化是一种有效的压缩方法.神经网络低比特量化存在的显著困难是量化比特数越低,网络分类精度也越低.为了解决这一问题,文中提出基于指数移动平均知识蒸馏的神经网络低比特量化方法.首先利用少量图像进行自适应初始化,训练激活和权重的量化步长,加快量化网络收敛.再引入指数移动平均(EMA)知识蒸馏的思想,利用EMA对蒸馏损失和任务损失进行归一化,指导量化网络训练.在ImageNet、CIFAR-10数据集上的分类任务表明,文中方法可获得接近或超过全精度网络的性能.     

New robust forecasting models for exchange rates prediction

This paper introduces two robust forecasting models for efficient prediction of different exchange rates for future months ahead. These models employ Wilcoxon artificial neural network (WANN) and Wilcoxon functional link artificial neural network (WFLANN). The learning algorithms required to train the weights of these models are derived by minimizing a robust norm called Wilcoxon norm. These models offer robust exchange rate predictions in the sense that the training of weight parameters of these models are not influenced by outliers present in the training samples. The Wilcoxon norm considers the rank or position of an error value rather than its amplitude. Simulation based experiments have been conducted using real life data and the results indicate that both models, unlike conventional models, demonstrate consistently superior prediction performance under different densities of outliers present in the training samples. Further, comparison of performance between the two proposed models reveals that both provide almost identical performance but the later involved low computational complexity and hence is preferable over the WANN model.     

基于组合模型的网络安全态势估计

为了获得更加理想的网络安全态势估计效果,提出一种基于组合方法的网络安全态势估计模型。首先收集网络安全态势样本,进行预处理得到学习样本,然后将训练样本集输入到BP神经网络进行学习,并采用布谷鸟搜索算法选择最合理的BP神经网络参数,最后通过仿真实验对模型性能进行分析。结果表明,本文模型大幅度降低了网络安全态势的拟合误差和预测误差,是一种科学、合理的网络安全态势估计模型,估计结果具有一定的实际应用价值。      

A hybrid approach to outlier detection in the offset lithographic printing process

Artificial neural networks are used to model the offset printing process aiming to develop tools for on-line ink feed control. Inherent in the modelling data are outliers owing to sensor faults, measurement errors and impurity of materials used. It is fundamental to identify outliers in process data in order to avoid using these data points for updating the model. We present a hybrid, the process-model-network-based technique for outlier detection. The outliers can then be removed to improve the process model. Several diagnostic measures are aggregated via a neural network to categorize data points into the outlier and inlier classes. We demonstrate experimentally that a soft fuzzy expert can be configured to label data for training the categorization of neural network.     

Intelligent position tracking control for LCM drive using stable online self-constructing recurrent neural network controller with bound architecture

In this paper, an intelligent position tracking control (IPTC) is developed for a linear ceramic motor (LCM) drive system. The IPTC system is comprised of a neural controller and a robust controller. The neural controller utilizes a self-constructing recurrent neural network (SCRNN) to mimic an ideal computation controller, and the robust controller is designed to achieve L₂ tracking performance with a desired attenuation level. If the approximation performance of SCRNN is insufficient, SCRNN can create new hidden neurons to increase the learning ability. If the hidden neuron of SCRNN is insignificant, it should be removed to reduce the computation load; otherwise, if the hidden neuron of SCRNN is significant, it should be retained. Moreover, the adaptive laws of controller parameters are derived in the sense of Lyapunov, so system stability can be guaranteed. Finally, the experimental results of the LCM drive system show a perfect tracking response can be achieved using the self-constructing mechanism and the on-line learning algorithm.