Adaptive soft sensor model using online support vector regression with time variable and discussion of appropriate hyperparameter settings and window size

Soft sensors have been widely used in chemical plants to estimate process variables that are difficult to measure online. One crucial difficulty of soft sensors is that predictive accuracy drops due to changes in state of chemical plants. The predictive accuracy of traditional soft sensor models decreases when sudden process changes occur. However, an online support vector regression (OSVR) model with the time variable can adapt to rapid changes among process variables. One crucial problem is finding appropriate hyperparameters and window size, which means the numbers of data for the model construction, and thus, we discussed three methods to select hyperparameters based on predictive accuracy and computation time. The window size of the proposed method was discussed through simulation data and real industrial data analyses and the proposed method achieved high predictive accuracy when time-varying changes in process characteristics occurred.     

Ensemble locally weighted partial least squares as a just‐in‐time modeling method

The predictive ability of soft sensors, which estimate values of an objective variable y online, decreases due to process changes in chemical plants. To reduce the decrease of predictive ability, adaptive soft sensors have been developed. We focused on just‐in‐time soft sensors, especially locally weighted partial least squares (LWPLS) regression. Since a set of hyperparameters in an LWPLS model has to be set beforehand and there is only onedataset, a traditional LWPLS model is difficult to accurately predict y‐values in multiple process states. In this study, we propose to combine LWPLS and ensemble learning, and predict y‐values with multiple LWPLS models, whose datasets and sets of hyperparameters are different. The weights of LWPLS models are determined based on Bayes’ theorem, considering their predictive ability. We confirmed that the proposed model has higher predictive accuracy than traditional models through numerical simulation data and two industrial data analyses. © 2015 American Institute of Chemical Engineers AIChE J, 62: 717–725, 2016     

Classification of the Degradation of Soft Sensor Models and Discussion on Adaptive Models

Soft sensors are used widely to estimate a process variable which is difficult to measure online. One of the crucial difficulties of soft sensors is that predictive accuracy drops due to changes of state of chemical plants. It is called as the degradation of soft sensor models. In this study, we attempted to classify this degradation of models in terms of changes in an explanatory variable and an objective variable, and the rapidity of the changes. Moreover, we discussed characteristics of adaptive soft sensor models, based on the classification results. By analyzing simulated data sets and a real industrial data set, we could obtain knowledge and information on appropriate adaptive models for each type of the degradation. © 2013 American Institute of Chemical Engineers AIChE J, 59: 2339–2347, 2013     

Applicability domains and accuracy of prediction of soft sensor models

Soft sensors are used to estimate process variables that are difficult to measure online. However, the predictive accuracy gradually decreases with changes in the state of chemical plants. Regression models can be updated, but if the model is updated with abnormal data, the predictive ability deteriorates. In practice, when the prediction error of an objective variable exceeds a threshold, an abnormal situation is detected. However, no effective method exists to decide this threshold. We have proposed a method to estimate the relationships between applicability domains and the accuracy of prediction of soft sensor models quantitatively. The larger the distances to models (DMs), the lower the estimated accuracy of prediction. Hence, the model between DMs and accuracy can separate variations in process variables and y‐analyzer fault. This method was applied to real industrial data. The fault detection ability of the proposed method was better than that of the traditional one. © 2010 American Institute of Chemical Engineers AIChE J, 2011     

Database monitoring index for adaptive soft sensors and the application to industrial process

In chemical plants, soft sensors are used to predict difficult‐to‐measure process variables. Soft sensor models must adapt to process changes by using new measured data. However, when a model is reconstructed with data that have low variation, the model cannot predict abrupt changes of process characteristics. The predictive performance of adaptive models depends on databases. We therefore propose an index to monitor database, that is, database monitoring index (DMI), and a database monitoring method using the DMI. The DMI is based on similarity between two data. The more similar two data are the smaller value the DMI has. New data are stored when the minimum DMI‐value of the data exceeds a threshold. Through the analysis of simulation data and real industrial data, we confirmed that databases can be appropriately managed and the predictive accuracy of adaptive soft sensor models increased by using the proposed method. © 2013 American Institute of Chemical Engineers AIChE J, 60: 160–169, 2014     

Development of a new soft sensor method using independent component analysis and partial least squares

Soft sensors are used widely to estimate a process variable which is difficult to measure online. One of the crucial difficulties of soft sensors is that predictive accuracy drops due to changes of state of chemical plants. To cope with this problem, a regression model can be updated. However, if the model is updated with an abnormal sample, the predictive ability can deteriorate. We have applied the independent component analysis (ICA) method to the soft sensor to increase fault detection ability. Then, we have tried to increase the predictive accuracy. By using the ICA‐based fault detection and classification model, the objective variable can be predicted, updating the PLS model appropriately. We analyzed real industrial data as the application of the proposed method. The proposed method achieved higher predictive accuracy than the traditional one. Furthermore, the nonsteady state could be detected as abnormal correctly by the ICA model. © 2008 American Institute of Chemical Engineers AIChE J, 2009     

A new process variable and dynamics selection method based on a genetic algorithm‐based wavelength selection method

Soft sensors have been used in industrial plants to estimate process variables that are difficult to measure online. Soft sensor models predicting an objective variable should be constructed with only important explanatory variables in terms of predictive ability, better interpretation of models and lower measurement costs. Besides, some process variables can affect an objective variable with time‐delays. Therefore, we have proposed the methods for selecting important process variables and optimal time‐delays of each variable simultaneously, by modifying the genetic algorithm‐based wavelength selection method that is one of the wavelength selection methods in spectrum analysis. The proposed methods can select time‐regions of process variables as a unit by using process data that includes process variables that are delayed in the range from zero to a set/given maximum value. The case study with simulation data and real industrial data confirmed that predictive, easy‐to‐interpret, and appropriate models were constructed using the proposed methods. © 2012 American Institute of Chemical Engineers AIChE J, 58: 1829–1840, 2012     

Applicability domain of soft sensor models based on one‐class support vector machine

Soft sensors are widely used to estimate process variables that are difficult to measure online. By using soft sensors, analyzer faults can be detected by estimation errors. However, it is difficult to detect abnormal data and determine the reasons because estimation errors increase not only due to analyzer faults but also due to variations caused by changes in the state of chemical plants. To separate those factors, we previously proposed to construct the relationships between distances to soft sensor models (DMs) and the accuracy of prediction of the models quantitatively and estimate the prediction accuracy of new data online. In this article, we used a one‐class support vector machine (OCSVM) to estimate data density and the output of an OCSVM as a DM. The proposed method was applied to real industrial data and the superiority of the proposed DM to the traditional ones was demonstrated by comparing their results. © 2013 American Institute of Chemical Engineers AIChE J, 59: 2046–2050, 2013     

Application of support vector regression for developing soft sensors for nonlinear processes

The field of soft sensor development has gained significant importance in the recent past with the development of efficient and easily employable computational tools for this purpose. The basic idea is to convert the information contained in the input–output data collected from the process into a mathematical model. Such a mathematical model can be used as a cost efficient substitute for hardware sensors. The Support Vector Regression (SVR) tool is one such computational tool that has recently received much attention in the system identification literature, especially because of its successes in building nonlinear blackbox models. The main feature of the algorithm is the use of a nonlinear kernel transformation to map the input variables into a feature space so that their relationship with the output variable becomes linear in the transformed space. This method has excellent generalisation capabilities to high‐dimensional nonlinear problems due to the use of functions such as the radial basis functions which have good approximation capabilities as kernels. Another attractive feature of the method is its convex optimization formulation which eradicates the problem of local minima while identifying the nonlinear models. In this work, we demonstrate the application of SVR as an efficient and easy‐to‐use tool for developing soft sensors for nonlinear processes. In an industrial case study, we illustrate the development of a steady‐state Melt Index soft sensor for an industrial scale ethylene vinyl acetate (EVA) polymer extrusion process using SVR. The SVR‐based soft sensor, valid over a wide range of melt indices, outperformed the existing nonlinear least‐square‐based soft sensor in terms of lower prediction errors. In the remaining two other case studies, we demonstrate the application of SVR for developing soft sensors in the form of dynamic models for two nonlinear processes: a simulated pH neutralisation process and a laboratory scale twin screw polymer extrusion process. A heuristic procedure is proposed for developing a dynamic nonlinear‐ARX model‐based soft sensor using SVR, in which the optimal delay and orders are automatically arrived at using the input–output data.     

Novel soft sensor method for detecting completion of transition in industrial polymer processes

Soft sensors are widely used to estimate process variables that are difficult to measure online. In polymer plants that produce various grades of polymers, the quality of products must be estimated using soft sensors in order to reduce the amount of off-grade material. However, during grade transition, the predictive accuracy deteriorates because the state in polymer reactors is unsteady, causing the values of process variables to differ from the steady-state values used to construct regression models. Therefore, we have proposed to construct models that detect the completion of transition to ensure that the polymer quality evaluated after transition conforms to the predicted one. By using these models and regression models constructed for each product grade, the polymer quality can be predicted with high accuracy, selecting a regression model appropriately. The proposed method was applied to industrial plant data and was found to exhibit higher predictive performance than traditional methods.     

选择性集成LTDGPR模型的自适应软测量建模方法

随着时间的增加,传统时间差（TD）模型会出现性能显著下降的问题。为了提高TD模型的可靠性和预测精度,同时考虑过程的时滞特征,基于一种选择性集成策略,提出一种局部时间差高斯过程回归（LTDGPR）模型的自适应软测量建模方法。首先,提取出数据库中的时滞动态信息,对建模数据进行重构;然后,采取局部化策略对差分后的重构样本进行统计划分,得到LTDGPR模型集。对于新来的输入样本,选择部分泛化能力强的LTDGPR模型进行集成,估计出含一定时间差的主导变量动态偏移值;最后,基于TD模型思想对当前时刻主导变量值进行在线预测。通过脱丁烷塔过程的数据建模仿真研究,验证了所提方法的有效性和精度。     

基于MIC的支持向量回归及其在化工过程中的应用

在化工生产中,软测量方法可以有效解决某些关键变量由于仪表故障而无法实时获取数据的问题。在建立软测量模型时,变量及回归方法的选取会直接影响模型的准确率。特别是在现代化工中,过程变量众多且变量间存在着冗余且复杂的非线性关系。对此,本文提出了一种基于最大信息系数的支持向量回归算法,利用最大信息系数在非线性相关性度量的优势,选择合适的辅助变量,避免了全部变量作为输入所造成的数据冗余。在此基础上,利用支持向量回归方法建立软测量模型,实现对软测量目标的预测。该方法被应用于存在仪表故障的某催化重整装置进料换热器热端压降的软测量中,结果表明该方法可以有效地实现对压降的软测量,实现了对仪表故障时的数据校正。     

Multi-layer features ensemble soft sensor regression model based on stacked autoencoder and vine copula

It is crucial in industrial processes to consider key variables to ensure safe operation and high product quality. Moreover, these variables are difficult to obtain using traditional measurement methods; hence, it makes sense to develop soft sensor regression models to process the variable prediction. However, there are numerous variables integrating noisy and redundant information in complex industrial processes. Using such variables in traditional regression models may result in reducing the model's efficiency and performance. Thus, this paper proposes a multi-layer feature ensemble soft sensor regression method using a stacked auto-encoder (SAE) and vine copula (ESAE–VCR) to address these problems. To do so, the number of neurons in the hidden layer of the SAE is determined by the principal component analysis (PCA). The multi-layer features of the process variables are extracted using a stacked AE, and the regression models are established for each feature layer. A linear regression ensemble method is used to combine the regression models with the multi-layer features to obtain the final predictive model that will estimate the values of the key variables. The effectiveness and practicality of the ESAE–VCR are validated by comparing them with several common soft measurement methods in two examples. In the numerical example, the ESAE–VCR yields an accuracy of prediction (R²) of 0.9898 and a root mean square error (RMSE) of 0.1804. In the industrial example, the ESAE–VCR yields an R² of 0.9908 and an RMSE of 0.1205.     

Online simulation‐based process control for injection molding

An online numerical simulation is presented that is capable of predicting state variables such as flow rate, melt temperature, shear rate, and melt viscosity by using real time data from a nozzle pressure sensor. The simulation solves the non‐Newtonian nonisothermal polymer flow into multicavity tools while executing rapidly enough for real time process control. Numerical accuracy and stability were first validated offline by comparing the online simulation to results obtained from a commercial mold filling simulation. Simulation‐based process control was then demonstrated by transferring a molding machine from fill to pack‐based on the predicted flow front position. The simulation‐based controller dynamically determined the appropriate transfer position for each part and transferred the machine at the correct time, thereby eliminating flash. The simulation, however, did increase process variability slightly due to delay times associated with the controller‐machine interface. A full factorial design of experiments (DOE) was performed varying injection velocity, mold temperature, and melt temperature. Results show that while the simulation dynamically adjusted the process on a part‐by‐part basis, it did not fully account for the process changes. Accuracy could potentially be improved by incorporating data from additional process sensors, by developing adaptive viscosity models, and by accounting for the melt compressibility. POLYM. ENG. SCI., 2009. © 2009 Society of Plastics Engineers     

A novel Bayesian inference soft sensor for real‐time statistic learning modeling for industrial polypropylene melt index prediction

A novel real‐time soft sensor based on a sparse Bayesian probabilistic inference framework is proposed for the prediction of melt index in industrial polypropylene process. The Bayesian framework consists of a relevance vector machine for predicting melt index and a particle filtering algorithm for soft sensor optimization. An online correcting strategy is also developed for improving the performance of real‐time melt index prediction. The method takes advantages of the probabilistic inference and using prior statistical knowledge of polymerization process. Developed soft sensors are validated with ten public databases from UCI machine learning repository and real data from industrial polypropylene process. Experimental results indicate the effectiveness of proposed method and show the improvement in both prediction precision and generalization capability compared with the reported models in literatures. © 2017 Wiley Periodicals, Inc. J. Appl. Polym. Sci. 2017 , 134, 45384.     

Adaptive soft sensor for online prediction and process monitoring based on a mixture of Gaussian process models

Linear models can be inappropriate when dealing with nonlinear and multimode processes, leading to a soft sensor with poor performance. Due to time-varying process behaviour it is necessary to derive and implement some kind of adaptation mechanism in order to keep the soft sensor performance at a desired level. Therefore, an adaptation mechanism for a soft sensor based on a mixture of Gaussian process regression models is proposed in this paper. A procedure for input variable selection based on mutual information is also presented. This procedure selects the most important input variables for output variable prediction, thus simplifying model development and adaptation. Apart from online prediction of the difficult-to-measure variable, this soft sensor can be used for adaptive process monitoring. The efficiency of the proposed method is benchmarked with the commonly applied recursive PLS and recursive PCA method on the Tennessee Eastman process and two real industrial examples.     

External analysis‐based regression model for robust soft sensing of multimode chemical processes

To remove the influence of operation mode changes in the chemical process, the whole variable set is partitioned into external, main, and quality variables. External variables are related to the operation mode. Two regression models are initially developed between external variables and main variables/quality variables, based on which the influence of the operation mode is removed from both input and output of the soft sensor. Then, an additional regression model is constructed for soft sensing, which is robust to the change of the operation mode. Compared to existing methods, the new method has advantages to handle two critical issues: (1) capable of quality estimation in new process modes; (2) able to distinguish changes in operation modes from process faults. Besides, a monitoring and analysis strategy is proposed for performance evaluation of the new soft sensor. Two case studies are provided to illustrate the efficiency of the proposed method. © 2013 American Institute of Chemical Engineers AIChE J, 60: 136–147, 2014     

Local Partial Least Squares Based Online Soft Sensing Method for Multi-output Processes with Adaptive Process States Division

Local learning based soft sensing methods succeed in coping with time-varying characteristics of processes as well as nonlinearities in industrial plants. In this paper, a local partial least squares based soft sensing method for multi-output processes is proposed to accomplish process states division and local model adaptation, which are two key steps in development of local learning based soft sensors. An adaptive way of partitioning process states without redundancy is proposed based on F-test, where unique local time regions are extracted. Subsequently, a novel anti-over-fitting criterion is proposed for online local model adaptation which simultaneously considers the relationship between process variables and the information in labeled and unlabeled samples. Case study is carried out on two chemical processes and simulation results illustrate the superiorities of the proposed method from several aspects.     

Real‐time property prediction for an industrial rubber‐mixing process with probabilistic ensemble Gaussian process regression models

In internal rubber‐mixing processes, data‐driven soft sensors have become increasingly important for providing online measurements for the Mooney viscosity information. Nevertheless, the prediction uncertainty of the model has rarely been explored. Additionally, traditional viscosity prediction models are based on single models and, thus, may not be appropriate for complex processes with multiple recipes and shifting operating conditions. To address both problems simultaneously, we propose a new ensemble Gaussian process regression (EGPR)‐based modeling method. First, several local Gaussian process regression (GPR) models were built with the training samples in each subclass. Then, the prediction uncertainty was adopted to evaluate the probabilistic relationship between the new test sample and several local GPR models. Moreover, the prediction value and the prediction variance was generated automatically with Bayesian inference. The prediction results in an industrial rubber‐mixing process show the superiority of EGPR in terms of prediction accuracy and reliability. © 2014 Wiley Periodicals, Inc. J. Appl. Polym. Sci. 2015 , 132, 41432.