Selection of significant input variables for time series forecasting

Appropriate selection of inputs for time series forecasting models is important because it not only has the potential to improve performance of forecasting models, but also helps reducing cost in data collection. This paper presents an investigation of selection performance of three input selection techniques, which include two model-free techniques, partial linear correlation (PLC) and partial mutual information (PMI) and a model-based technique based on genetic programming (GP). Four hypothetical datasets and two real datasets were used to demonstrate the performance of the three techniques. The results suggested that the model-free PLC technique due to its computational simplicity and the model-based GP technique due to its ability to detect non-linear relationships (demonstrated by its relatively good performance on a hypothetical complex non-linear dataset) are recommended for the input selection task. Candidate inputs which are selected by both these recommended techniques should be considered as significant inputs.     

Detecting nonlinear interrelation patterns among process variables using genetic programming

Detecting non-linear interaction patterns among process variables is an important task for fault detection and propagation analysis. There are many statistical and evolutionary techniques being developed in the literature for prediction of interaction strengths but their accuracy is generally unsatisfactory. This study demonstrates an evolutionary programming approach to uncover non-linear relations among process variables. In this study, we make an attempt to use genetic programming (GP) based approach for this purpose. GP overcomes many shortcomings faced by other statistical or evolutionary techniques in this context. The effectiveness, feasibility, and robustness of the proposed method are demonstrated on simulated data emanating from a well-known Tennessee Eastman process. The proposed method has successfully achieved reasonable detection and prediction of non-linear interaction patterns among process variables.     

Feature Selection Using Probabilistic Neural Networks

Selection of input variables (features) is a key stage in building predictive models. As exhaustive evaluation of potential feature sets using full non-linear models is impractical, it is common practice to use simple fast-evaluating models and heuristic selection strategies. This paper discusses a fast, efficient, and powerful non-linear input selection procedure using a combination of probabilistic neural networks and repeated bitwise gradient descent with resampling. The algorithm is compared with forward selection, backward selection and genetic algorithms using a selection of real-world data sets. The algorithm has comparative performance and greatly reduced execution time with respect to these alternative approaches.     

Optimal selection of soft sensor inputs for batch distillation columns using principal component analysis

In this paper, a novel methodology based on principal component analysis (PCA) is proposed to select the most suitable secondary process variables to be used as soft sensor inputs. In the proposed approach, a matrix is defined that measures the instantaneous sensitivity of each secondary variable to the primary variables to be estimated. The most sensitive secondary variables are then extracted from this matrix by exploiting the properties of PCA, and they are used as input variables for the development of a regression model suitable for on-line implementation.This method has been evaluated by developing a soft sensor that uses temperature measurements and a process regression model to estimate on-line the product compositions for a simulated batch distillation process. The identification of the optimal soft sensor inputs for this case study has been discussed with respect to the definition of the sensitivity matrix, the data sampling interval, the presence of measurement noise, and the size of the input set. The simulation results demonstrate that the proposed approach can effectively identify the size and configuration of the input set that leads to the optimal estimation performance of the soft sensor.     

Evolutionary Optimization of Machining Processes

Optimization of machining processes plays a key role in meeting the demands for high precision and productivity. The primary challenge for machining process optimization often stems from the fact that the procedure is typically highly constrained and highly non-linear, involving mixed-integer-discrete-continuous design variables. Additionally, machining process models are likely discontinuous, non-explicit, or not analytically differentiable with the design variables. Traditional non-linear optimization techniques are mostly gradient-based, posing many limitations upon application to today’s complex machining models. Genetic Algorithms (GAs) has distinguished itself as a method with the potential for solving highly non-linear, ill-behaved complex machining optimization problems. Unlike traditional optimization techniques, GAs start with a population of different designs and use direct search methods stochastically and deterministically toward optimal and feasible direction. However, GAs still has its own drawbacks when it is applied to machining process optimization, including the lack of efficiency due to its binary representation scheme for continuous design variables, a lack of local fine-tuning capabilities, a lack of a self-adaptation mechanism, and a lack of an effective constraint handling method. A novel and systematic evolutionary algorithm based on GAs is presented in this paper in the areas of problem representation; selection scheme; genetic operators for integer, discrete, and continuous variables; constraint handling method; and population initialization to overcome the underlying drawbacks. The proposed scheme has been applied to two machining problems to demonstrate its superior performance.     

基于特征约简与选择性集成算法的城市固废焚烧过程二噁英排放浓度软测量

城市固废焚烧(MSWI)排放的污染物二噁英(DXN)对生态环境与人类健康具有极大危害,其浓度的实时检测对MSWI过程运行优化和城市污染控制至关重要.具有痕量特性的DXN排放浓度不能实时检测,机理模型难以构建,并且其与过程变量间的映射关系复杂.针对上述问题,本文提出了一种基于特征约简和选择性集成算法的DXN排放浓度软测量...     

Multiobjective optimization using variable complexity modelling for control system design

A multi-stage design approach that uses a multiobjective genetic algorithm as the framework for optimization and multiobjective preference articulation, and an H_infty loop-shaping technique are used to design controllers for a gas turbine engine. A non-linear model is used to assess performance of the controller. Because the computational load of applying multiobjective genetic algorithm to this control strategy is very high, a neural network and response surface models are used in order to speed up the design process within the framework of a multiobjective genetic algorithm. The final designs are checked using the original non-linear model.     

Shape optimisation of continuum structures via evolution strategies and fixed grid finite element analysis

Evolution strategies (ES) are very robust and general techniques for finding global optima in optimisation problems. As with all evolutionary algorithms, ES apply evolutionary operators and select the most fit from a set of possible solutions. Unlike genetic algorithms, ES do not use binary coding of individuals, working instead with real variables. Many recent studies have applied evolutionary algorithms to structural problems, particularly the optimisation of trusses. This paper focuses on shape optimisation of continuum structures via ES. Stress analysis is accomplished by using the fixed grid finite element method, which reduces the computing time while keeping track of the boundary representation of the structure. This boundary is represented by b-spline functions, circles, and polylines, whose control points constitute the parameters that govern the shape of the structure. Evolutionary operations are applied to each set of variables until a global optimum is reached. Several numerical examples are presented to illustrate the performance of the method. Finally, structures with multiple load cases are considered along with examples illustrating the results obtained.     

Non-linear systems identification using radial basis functions

This paper investigates the identification of discrete-time non-linear systems using radial basis functions. A forward regression algorithm based on an orthogonal decomposition of the regression matrix is employed to select a suitable set of radial basis function centers from a large number of possible candidates and this provides, for the first time, fully automatic selection procedure for identifying parsimonious radial basis function models of structure-unknown non-linear systems. The relationship between neural networks and radial basis functions is discussed and the application of the algorithms to real data is included to demonstrate the effectiveness of this approach.     

A 'Model-on-Demand' identification methodology for non-linear process systems

An identification methodology based on multi-level pseudo-random sequence (multi-level PRS) input signals and 'Model-on-Demand' (MoD) estimation is presented for single-input, single-output non-linear process applications. 'Model-on-Demand' estimation allows for accurate prediction of non-linear systems while requiring few user choices and without solving a non-convex optimization problem, as is usually the case with global modelling techniques. By allowing the user to incorporate a priori information into the specification of design variables for multi-level PRS input signals, a sufficiently informative input-output dataset for MoD estimation is generated in a 'plant-friendly' manner. The usefulness of the methodology is demonstrated in case studies involving the identification of a simulated rapid thermal processing (RTP) reactor and a pilot-scale brine-water mixing tank. On the resulting datasets, MoD estimation displays performance comparable to that achieved via semi-physical modelling and semi-physical modelling combined with neural networks. The MoD estimator, however, achieves this level of performance with substantially lower engineering effort.     

Local identification of prototypes for genetic learning of accurate TSK fuzzy rule‐based systems

This work presents the use of local fuzzy prototypes as a new idea to obtain accurate local semantics‐based Takagi–Sugeno–Kang (TSK) rules. This allow us to start from prototypes considering the interaction between input and output variables and taking into account the fuzzy nature of the TSK rules. To do so, a two‐stage evolutionary algorithm based on MOGUL (a methodology to obtain Genetic Fuzzy Rule‐Based Systems under the Iterative Rule Learning approach) has been developed to consider the interaction between input and output variables. The first stage performs a local identification of prototypes to obtain a set of initial local semantics‐based TSK rules, following the Iterative Rule Learning approach and based on an evolutionary generation process within MOGUL (taking as a base some initial linguistic fuzzy partitions). Because this generation method induces competition among the fuzzy rules, a postprocessing stage to improve the global system performance is needed. Two different processes are considered at this stage, a genetic niching‐based selection process to remove redundant rules and a genetic tuning process to refine the fuzzy model parameters. The proposal has been tested with two real‐world problems, achieving good results. © 2007 Wiley Periodicals, Inc. Int J Int Syst 22: 909–941, 2007.     

基于随机森林的短期电量负荷精准预测方法

可再生能源并入电网后,电能供给量增加,短期电量负荷情况难以预测,无法制定准确的电能分配策略,由此,提出基于随机森林的短期电量负荷精准预测方法研究。深入分析短期电量负荷预测影响因素（气象、时间、电价与随机干扰因素）,选取适当的模型输入变量（历史电量负荷数据、温度数据与日类型）,结合随机森林算法构建短期电量负荷预测模型,并重复确定相似日的选取规则,采用粒子群优化算法寻找预测模型参数最佳值,将样本集输入至模型中,获得精准的短期电量负荷预测结果。实验数据显示:当输入变量数量达到一定值后,应用提出方法获得的短期电量负荷预测时延稳定在0.55s左右,短期电量负荷预测误差几乎为0,充分证实了提出方法应用性能较佳。     

Hybrid evolutionary learning for synthesizing multi-class pattern recognition systems

This paper describes one aspect of a machine-learning system called HELPR that blends the best aspects of different evolutionary techniques to bootstrap-up a complete recognition system from primitive input data. HELPR uses a multi-faceted representation consisting of a growing sequence of non-linear mathematical expressions. Individual features are represented as tree structures and manipulated using the techniques of genetic programming. Sets of features are represented as list structures that are manipulated using genetic algorithms and evolutionary programming. Complete recognition systems are formed in this version of HELPR by attaching the evolved features to multiple perceptron discriminators. Experiments on datasets from the University of California at Irvine (UCI) machine-learning repository show that HELPR’s performance meets or exceeds accuracies previously published.     

Parallel multiobjective memetic RBFNNs design and feature selection for function approximation problems

The design of radial basis function neural networks (RBFNNs) still remains as a difficult task when they are applied to classification or to regression problems. The difficulty arises when the parameters that define an RBFNN have to be set, these are: the number of RBFs, the position of their centers and the length of their radii. Another issue that has to be faced when applying these models to real world applications is to select the variables that the RBFNN will use as inputs. The literature presents several methodologies to perform these two tasks separately, however, due to the intrinsic parallelism of the genetic algorithms, a parallel implementation will allow the algorithm proposed in this paper to evolve solutions for both problems at the same time. The parallelization of the algorithm not only consists in the evolution of the two problems but in the specialization of the crossover and mutation operators in order to evolve the different elements to be optimized when designing RBFNNs. The subjacent genetic algorithm is the non-sorting dominated genetic algorithm II (NSGA-II) that helps to keep a balance between the size of the network and its approximation accuracy in order to avoid overfitted networks. Another of the novelties of the proposed algorithm is the incorporation of local search algorithms in three stages of the algorithm: initialization of the population, evolution of the individuals and final optimization of the Pareto front. The initialization of the individuals is performed hybridizing clustering techniques with the mutual information (MI) theory to select the input variables. As the experiments will show, the synergy of the different paradigms and techniques combined by the presented algorithm allow to obtain very accurate models using the most significant input variables.     

An efficient identification scheme for a nonlinear polynomial NARX model

Nonlinear polynomial NARX (nonlinear autoregressive with exogenous inputs) model identification often faces the problem of the huge size of the candidate pool, which makes the “wrapper” structure selection algorithm worked at low efficiency. In this article, a correlation-based orthogonal forward selection (COFS) algorithm is proposed to select the necessary input variables so that the candidate pool thus formed becomes tractable. What is more, it is trunked by an importance index-based term selection method, where a multiobjective evolutionary algorithm (MOEA)-based structure selection algorithm could be used to identify the polynomial model efficiently. Two numerical simulations are carried out to show the effectiveness of the proposed method.     

Non-linear variable selection for artificial neural networks using partial mutual information

Artificial neural networks (ANNs) have been widely used to model environmental processes. The ability of ANN models to accurately represent the complex, non-linear behaviour of relatively poorly understood processes makes them highly suited to this task. However, the selection of an appropriate set of input variables during ANN development is important for obtaining high-quality models. This can be a difficult task when considering that many input variable selection (IVS) techniques fail to perform adequately due to an underlying assumption of linearity, or due to redundancy within the available data.This paper focuses on a recently proposed IVS algorithm, based on estimation of partial mutual information (PMI), which can overcome both of these issues and is considered highly suited to the development of ANN models. In particular, this paper addresses the computational efficiency and accuracy of the algorithm via the formulation and evaluation of alternative techniques for determining the significance of PMI values estimated during selection. Furthermore, this paper presents a rigorous assessment of the PMI-based algorithm and clearly demonstrates the superior performance of this non-linear IVS technique in comparison to linear correlation-based techniques.     

Stepwise approach for the evolution of generalized genetic programming model in prediction of surface finish of the turning process

Due to the complexity and uncertainty in the process, the soft computing methods such as regression analysis, neural networks (ANN), support vector regression (SVR), fuzzy logic and multi-gene genetic programming (MGGP) are preferred over physics-based models for predicting the process performance. The model participating in the evolutionary stage of the MGGP method is a linear weighted sum of several genes (model trees) regressed using the least squares method. In this combination mechanism, the occurrence of gene of lower performance in the MGGP model can degrade its performance. Therefore, this paper proposes a modified-MGGP (M-MGGP) method using a stepwise regression approach such that the genes of lower performance are eliminated and only the high performing genes are combined. In this work, the M-MGGP method is applied in modelling the surface roughness in the turning of hardened AISI H11 steel. The results show that the M-MGGP model produces better performance than those of MGGP, SVR and ANN. In addition, when compared to that of MGGP method, the models formed from the M-MGGP method are of smaller size. Further, the parametric and sensitivity analysis conducted validates the robustness of our proposed model and is proved to capture the dynamics of the turning phenomenon of AISI H11 steel by unveiling dominant input process parameters and the hidden non-linear relationships.     

The Construction of Smooth Models using Irregular Embeddings Determined by a Gamma Test Analysis

One of the key problems in forming a smooth model from input-output data is the determination of which input variables are relevant in predicting a given output. In this paper, we show how the Gamma test can be used to select that combination of input variables which can best be employed to form a smooth model of an output. For time series prediction this amounts to the selection of an appropriate irregular embedding. We give some simple zero noise examples of time series analysis, and illustrate how using these techniques a binary message encoded into a chaotic carrier can be retrieved without knowledge of the dynamics used to generate the carrier. Provided the underlying dynamics are such as to produce a smooth embedding model with bounded partial derivatives, the sampling distribution is dense in input space, and any associated distribution of measurement error has the first few moments bounded, so that the typical prerequisite conditions of the Gamma test are satisfied, we conclude that the Gamma test is an effective tool in the determination of irregular time series embeddings. These techniques can also be useful in practical applications which involve filtering seismic data to detect anomalous events.     

On convexity and quasiconvexity of an on-line identification formulation

A method based on non-linear regression has been proposed by Toh and Devanathan (1996) to perform closed-loop process identification under naturally occurring load disturbances. The significance of this method includes relaxation of an important assumption on the knowledge of the input excitation source, given a stable controller which is to be improved. While not aiming to identify a precise model as in classical design, this method uses an approximate process model for on-line identification. This serves as a basis for controller refinement, without having to interrupt the process operation for identification test. In this paper, we present a refined formulation and then perform convexity analysis and quasiconvexity analysis on this refined formulation. We show that these results can be utilized to obtain convergence independent of the initial estimates. Applicability of the identification for controller tuning is also demonstrated using a process with an unknown model.     

Machine-learning paradigms for selecting ecologically significant input variables

Harmful algal blooms, which are considered a serious environmental problem nowadays, occur in coastal waters in many parts of the world. They cause acute ecological damage and ensuing economic losses, due to fish kills and shellfish poisoning as well as public health threats posed by toxic blooms. Recently, data-driven models including machine-learning (ML) techniques have been employed to mimic dynamics of algal blooms. One of the most important steps in the application of a ML technique is the selection of significant model input variables. In the present paper, we use two extensively used ML techniques, artificial neural networks (ANN) and genetic programming (GP) for selecting the significant input variables. The efficacy of these techniques is first demonstrated on a test problem with known dependence and then they are applied to a real-world case study of water quality data from Tolo Harbour, Hong Kong. These ML techniques overcome some of the limitations of the currently used techniques for input variable selection, a review of which is also presented. The interpretation of the weights of the trained ANN and the GP evolved equations demonstrate their ability to identify the ecologically significant variables precisely. The significant variables suggested by the ML techniques also indicate chlorophyll-a (Chl-a) itself to be the most significant input in predicting the algal blooms, suggesting an auto-regressive nature or persistence in the algal bloom dynamics, which may be related to the long flushing time in the semi-enclosed coastal waters. The study also confirms the previous understanding that the algal blooms in coastal waters of Hong Kong often occur with a life cycle of the order of 1–2 weeks.