On line detection of mean and variance shift using neural networks and support vector machine in multivariate processes

The effective recognition of unnatural control chart patterns (CCPs) is one of the most important tools to identify process problems. In multivariate process control, the main problem of multivariate quality control charts is that they can detect an out of control event but do not directly determine which variable or group of variables has caused the out of control signal and how much is the magnitude of out of control. Recently machine learning techniques, such as artificial neural networks (ANNs), have been widely used in the research field of CCP recognition. This study presents a modular model for on-line analysis of out of control signals in multivariate processes. This model consists of two modules. In the first module using a support vector machine (SVM)-classifier, mean shift and variance shift can be recognized. Then in the second module, using two special neural networks for mean and variance, it can be recognized magnitude of shift for each variable simultaneously. Through evaluation and comparison, our research results show that the proposed modular performs substantially better than the traditional corresponding control charts. The main contributions of this work are recognizing the type of unnatural pattern and classifying the magnitude of shift for mean and variance in each variable simultaneously.     

Pattern recognition of control charts using artificial neural networks—analyzing the effect of the training parameters

In this paper, we have utilized artificial neural networks (ANN) for pattern recognition of the most common patterns which occur in quality control charts. After detecting such patterns, it is possible to relate these patterns to their causes. This could find extreme importance for on-line quality monitoring and on-line trouble shooting. It could be possible to detect problems before they become serious and the operator has to shut the line down or the process may produce defective parts. In this work, we have attempted to explore the effect of the training parameters on the performance of the neural network. The training parameters are important because they emphasis the required performance and the accuracy required from the neural network. A resolution IV fractional factorial experiment is utilized to explore a portion of the range of selected parameters to obtain better performance of the neural network. The results showed that many parameters usually assigned by experience such as minimum shift, shift range, population size and shift percentage, have significant effect on the performance of the ANN, while others such as network size and window size do not have major significance on the performance of the net.     

A neural network-based procedure for the monitoring of exponential mean

Control charts are widely used for both manufacturing and service industries. Cumulative sum (CUSUM) charts are known to be very sensitive in detecting small shifts in the mean. In this paper, we propose a neural network as an alternative approach to CUSUM charts when monitoring exponential mean. The performance of neural network was evaluated by estimating the average run lengths (ARLs) using simulation. The results obtained with simulated data suggest that control scheme based on neural network is significantly more sensitive to process shifts than CUSUM charts. This research also examines the feasibility of using CUSUM chart and neural network together in detecting process mean shifts. The results indicate that using the two methods in combination is more effective than using the methods separately.     

热定型机的神经网络优化模糊-PID控制研究

热定型机的温度是复合管材生产过程中的重要参数，往往需要精确控制。由于该温度对象具有滞后特性，采用传统的PID控制、模糊控制或神经网络控制等单一的控制策略效果并不十分理想。本文提出了神经网络优化模糊一PID控制，并进行了相应研究，研究结果表明，该控制方法在超调量、调节时间、稳态偏差等控制性能指标上比传统的PID和单一的智能控制具有较好的效果。     

改进型Elman神经网络发酵过程建模研究

依据发酵过程的机理和改进的Elman神经网络动态建模原理,提出了一个新的发酵过程建模分批训练算法。通过发酵过程仿真实验,与传统的BP建模算法比较,改进的Elman神经网络建模算法具有收敛速度快、泛化能力强等特点。此外,利用该算法编制的软件可以内嵌到发酵过程监控系统中,实现发酵过程在线建模与状态参量的在线预估。     

A neural network model and algorithm for the hybrid flow shop scheduling problem in a dynamic environment

A hybrid flow shop (HFS) is a generalized flow shop with multiple machines in some stages. HFS is fairly common in flexible manufacturing and in process industry. Because manufacturing systems often operate in a stochastic and dynamic environment, dynamic hybrid flow shop scheduling is frequently encountered in practice. This paper proposes a neural network model and algorithm to solve the dynamic hybrid flow shop scheduling problem. In order to obtain training examples for the neural network, we first study, through simulation, the performance of some dispatching rules that have demonstrated effectiveness in the previous related research. The results are then transformed into training examples. The training process is optimized by the delta-bar-delta (DBD) method that can speed up training convergence. The most commonly used dispatching rules are used as benchmarks. Simulation results show that the performance of the neural network approach is much better than that of the traditional dispatching rules.This revised version was published in June 2005 with corrected page numbers.     

Simultaneous process mean and variance monitoring using artificial neural networks

Control chart patterns (CCPs) can be employed to determine the behavior of a process. Hence, CCP recognition is an important issue for an effective process-monitoring system. Artificial neural networks (ANNs) have been applied to CCP recognition tasks and promising results have been obtained. It is well known that mean and variance control charts are usually implemented together and that these two charts are not independent of each other, especially for the individual measurements and moving range (X–R_m) charts. CCPs on the mean and variance charts can be associated independently with different assignable causes when corresponding process knowledge is available. However, ANN-based CCP recognition models for process mean and variance have mostly been developed separately in the literature with the other parameter assumed to be under control. Little attention has been given to the use of ANNs for monitoring the process mean and variance simultaneously. This study presents a real-time ANN-based model for the simultaneous recognition of both mean and variance CCPs. Three most common CCP types, namely shift, trend, and cycle, for both mean and variance are addressed in this work. Both direct data and selected statistical features extracted from the process are employed as the inputs of ANNs. The numerical results obtained using extensive simulation indicate that the proposed model can effectively recognize not only single mean or variance CCPs but also mixed CCPs in which mean and variance CCPs exist concurrently. Empirical comparisons show that the proposed model performs better than existing approaches in detecting mean and variance shifts, while also providing the capability of CCP recognition that is very useful for bringing the process back to the in-control condition. A demonstrative example is provided.     

一种基于自适应遗传BP算法的神经网络目标分类器设计方法

目标分类器是水下目标识别系统的重要组成部分.本文针对线谱特征提取,提出了一种基于自适应遗传BP算法训练神经网络目标分类器的新方法.经对海上实录三类目标噪声分类识别实验结果表明采用新方法的神经网络分类器具有更优的分类效果.     

基于PID和深度卷积神经网络的辐射源识别方法

利用神经网络进行辐射源个体识别时,训练样本的单一性会导致深度网络出现过拟合的现象,继而影响辐射源个体识别的精确性。针对该问题,本文提出一种基于PID算法的深度卷积网络结构,该结构通过在传统卷积神经网络的输出层与输入层间构建一条反馈回路,采用PID算法将网络输出错误率转化为划分训练集数据构成的概率,通过优化训练集数据构成,达到抑制过拟合的目的。将该方法应用于超短波电台识别,平均识别率达到92.59%,识别率方差约为传统算法的1/3,训练用时减少约35 min,上述指标均优于传统神经网络。实验结果表明,该算法增强了深度网络的鲁棒性,有效地抑制了过拟合现象。     

改进PSO-BP神经网络对储层参数的动态预测研究

为提高BP神经网络的收敛速度和泛化能力,防止其陷入局部最优值,在前人工作基础上对传统粒子群算法进行了改进,具体包括：设定最大限制速度、改变惯性权重因子和改进适应度函数,并把改进粒子群算法应用于BP神经网络权值和阈值的优化。之后利用改进粒子群算法优化的BP神经网络实现对储层参数的动态预测,具体步骤为：确定神经网络的输入、输出神经元,定量化时间参数[T],利用训练样本构建神经网络模型并进行检验。最后通过平均训练误差对仿真过程进行分析,结果表明改进PSO-BP算法的收敛性与泛化能力均优于BP算法和PSO-BP算法。     

模糊神经网络pH控制器的DSP实现

针对pH值控制过程具有较强非线性、纯滞后性的特点，传统PID控制往往达不到满意控制效果。介绍一种将模糊控制技术与神经网络技术相结合构成的模糊神经网络pH控制器，通过数字仿真显示了该控制算法的控制效果优于传统的PID控制和一般的模糊控制算法。并将提出的模糊神经网络控制算法在DSP上进行了实现．通过模拟实验验证了该控制器的可行性。     

深度学习批归一化及其相关算法研究进展

深度学习已经广泛应用到各个领域, 如计算机视觉和自然语言处理等, 并都取得了明显优于早期机器学习算法的效果. 在信息技术飞速发展的今天, 训练数据逐渐趋于大数据集, 深度神经网络不断趋于大型化, 导致训练越来越困难, 速度和精度都有待提升. 2013年, Ioffe等指出训练深度神经网络过程中存在一个严重问题: 中间协变量迁移(Internal covariate shift), 使网络训练过程对参数初值敏感、收敛速度变慢, 并提出了批归一化(Batch normalization, BN)方法, 以减少中间协变量迁移问题, 加快神经网络训练过程收敛速度. 目前很多网络都将BN作为一种加速网络训练的重要手段, 鉴于BN的应用价值, 本文系统综述了BN及其相关算法的研究进展. 首先对BN的原理进行了详细分析. BN虽然简单实用, 但也存在一些问题, 如依赖于小批量数据集的大小、训练和推理过程对数据处理方式不同等, 于是很多学者相继提出了BN的各种相关结构与算法, 本文对这些结构和算法的原理、优势和可以解决的主要问题进行了分析与归纳. 然后对BN在各个神经网络领域的应用方法进行了概括总结, 并且对其他常用于提升神经网络训练性能的手段进行了归纳. 最后进行了总结, 并对BN的未来研究方向进行了展望.     

Data mining model-based control charts for multivariate and autocorrelated processes

Process monitoring and diagnosis have been widely recognized as important and critical tools in system monitoring for detection of abnormal behavior and quality improvement. Although traditional statistical process control (SPC) tools are effective in simple manufacturing processes that generate a small volume of independent data, these tools are not capable of handling the large streams of multivariate and autocorrelated data found in modern systems. As the limitations of SPC methodology become increasingly obvious in the face of ever more complex processes, data mining algorithms, because of their proven capabilities to effectively analyze and manage large amounts of data, have the potential to resolve the challenging problems that are stretching SPC to its limits. In the present study we attempted to integrate state-of-the-art data mining algorithms with SPC techniques to achieve efficient monitoring in multivariate and autocorrelated processes. The data mining algorithms include artificial neural networks, support vector regression, and multivariate adaptive regression splines. The residuals of data mining models were utilized to construct multivariate cumulative sum control charts to monitor the process mean. Simulation results from various scenarios indicated that data mining model-based control charts performs better than traditional time-series model-based control charts.     

Effective recognition of control chart patterns in autocorrelated data using a support vector machine based approach

The effective recognition of unnatural control chart patterns (CCPs) is a critical issue in statistical process control, as unnatural CCPs can be associated with specific assignable causes adversely affecting the process. Machine learning techniques, such as artificial neural networks (ANNs), have been widely used in the research field of CCP recognition. However, ANN approaches can easily overfit the training data, producing models that can suffer from the difficulty of generalization. This causes a pattern misclassification problem when the training examples contain a high level of background noise (common cause variation). Support vector machines (SVMs) embody the structural risk minimization, which has been shown to be superior to the traditional empirical risk minimization principle employed by ANNs. This research presents a SVM-based CCP recognition model for the on-line real-time recognition of seven typical types of unnatural CCP, assuming that the process observations are AR(1) correlated over time. Empirical comparisons indicate that the proposed SVM-based model achieves better performance in both recognition accuracy and recognition speed than the model based on a learning vector quantization network. Furthermore, the proposed model is more robust toward background noise in the process data than the model based on a back propagation network. These results show the great potential of SVM methods for on-line CCP recognition.     

Performance study of Byzantine Agreement Protocol with artificial neural network

Since 1982, numerous Byzantine Agreement Protocols (BAPs) have been developed to solve arbitrary faults in the Byzantine Generals Problem (BGP). A novel BAP, using an artificial neural network (ANN), was proposed by Wang and Kao. It requires message exchange rounds similar to the traditional BAP and its suitability, in the context of network size, has not been investigated. In the present study, we propose to adopt Nguyen-Widrow initialization in ANN training, which modifies message communication and limits the message exchange rounds to three rounds. This modified approach is referred to as BAP-ANN. The BAP-ANN performs better than the traditional BAP, when the network size n is greater than nine. We also evaluate the message exchange matrix (MEM) constructed during the message exchange stage. For a fixed number of faulty nodes and remainder cases of (n mod 3), the study shows that the mean epoch for ANN training decreases as the network size increases, which indicates better fault tolerance.     

基于SPC和神经网络的卷烟制丝生产质量监控方法研究

为解决卷烟制丝生产过程中现有SPC监控方法存在的问题,提出了基于SPC和BP神经网络的质量监控方法.首先在传统控制图的基础上,提出了适合在线监控的移动窗口式控制图,然后分别建立了用于控制图模式识别和质量缺陷原因诊断的两个神经网络模型,最后通过松散回潮工序中出口物料含水率的质量监控实例,证明了该质量监控方法的有效性.     

基于调和模型神经网络的彩色图像复原研究

针对彩色图像复原提出了基于网络能量递减收敛的调和模型神经网络图像复原方法,研究了该方法在运动模糊图像复原上的应用。利用待复原图像重构出多幅模糊图像用于算法的实现,并首次提出基于图像局部方差的自适应正则化算子的实现方法。实验结果表明,该方法是有效的,复原效果优于有约束的最小二乘复原法和已有的传统神经网络图像复原法,对复原图像的信噪比有一定的提高。     

Mean shifts detection and classification in multivariate process: a neural-fuzzy approach

For monitoring multivariate quality control process, traditional multivariate control charts have been proposed to detect mean shifts. However, a persistent problem is that such charts are unable to provide any shift-related information when mean shifts occur in the process. In fact, the immediate classification of the magnitude of mean shifts can greatly narrow down the set of possible assignable causes, hence facilitating quick analysis and corrective action by the technician before many nonconforming units are manufactured. In this paper, we propose a neural-fuzzy model for detecting mean shifts and classifying their magnitude in multivariate process. This model is divided into training and classifying modules. In the training module, a neural network (NN) model is trained to detect various mean shifts for multivariate process. Then, in the classifying module, the outputs of NN are classified into various decision intervals by using a fuzzy classifier and an additional two-point-in-an-interval decision rule to determine shift status. An example is presented to illustrate the application of the proposed model. Simulation results show that it outperforms the multivariate T²control chart in terms of out-of-control average run length under fixed type I error. In addition, the correct classification percentages are also studied and the general guidelines are given for the proper use of the proposed model.     

多层前向人工神经网络图像分类算法

传统人工神经网络的输入均为向量形式,而图像由矩阵形式表示,因此,在用人工神经网络进行图像处理时,图像将以向量形式输入至神经网络,这破坏了图像的结构信息,从而影响了图像处理的效果。为了提高网络对图像的处理能力,文中借鉴了深度学习的思想与方法,引进了具有矩阵输入的多层前向神经网络。同时,采用传统的反向传播训练算法(BP)训练该网络,给出了训练过程与训练算法,并在USPS手写数字数据集上进行了数值实验。实验结果表明,相对于单隐层矩阵输入前向神经网络(2D-BP),所提多层网络具有较好的分类效果。此外,对于彩色图片分类问题,利用所提出的2D-BP网络,给出了一个有效的可行方法。     

Fault detection and diagnosis using statistical control charts and artificial neural networks

In order to operate a successful plant or process, continuous improvement must be made in the areas of safety, quality and reliability. Central to this continuous improvement is the early or proactive detection and correct diagnosis of process faults. This research examines the feasibility of using cumulative summation (CUSUM) control charts and artificial neural networks together for fault detection and diagnosis (FDD). The proposed FDD strategy was tested on a model of the heat transport system of a CANDU nuclear reactor.The results of the investigation indicate that a FDD system using CUSUM control charts and a radial basis function (RBF) neural network is not only feasible but also of promising potential. The control charts and neural network are linked by using a characteristic fault signature pattern for each fault which is to be detected and diagnosed. When tested, the system was able to eliminate all false alarms at steady state, promptly detect six fault conditions, and correctly diagnose five out of the six faults. The diagnosis for the sixth fault was inconclusive.