A framework for nonparametric profile monitoring

Control charts have been widely used for monitoring the functional relationship between a response variable and some explanatory variable(s) (called profile) in various industrial applications. In this article, we propose an easy-to-implement framework for monitoring nonparametric profiles in both Phase I and Phase II of a control chart scheme. The proposed framework includes the following steps: (i) data cleaning; (ii) fitting B-spline models; (iii) resampling for dependent data using block bootstrap method; (iv) constructing the confidence band based on bootstrap curve depths; and (v) monitoring profiles online based on curve matching. It should be noted that, the proposed method does not require any structural assumptions on the data and, it can appropriately accommodate the dependence structure of the within-profile observations. We illustrate and evaluate our proposed framework by using a real data set.     

Application of integrated recurrent neural network with multivariate adaptive regression splines on SPC-EPC process

The integration of statistical process control and engineering process control has been reported as an effective way to monitor and control the autocorrelated process. However, because engineering process control compensates for the effects of underlying disturbances, the disturbance patterns become very hard to recognize, especially when various abnormal control chart patterns are mixed and co-existed in the engineering process. In this study, a new control chart pattern recognition model which integrates multivariate adaptive regression splines and recurrent neural network is proposed to not only address the problem of feature selection (i.e., lagged process measurements) but also improve the pattern recognition accuracy. The performance of the proposed method is evaluated by comparing the recognition results of multivariate adaptive regression splines and recurrent neural network with the results of four competing approaches (multivariate adaptive regression splines-extreme learning machine, multivariate adaptive regression splines-random forest, single recurrent neural network, and single random forest) on the simulated individual process data. The experimental study shows that the proposed multivariate adaptive regression splines and recurrent neural network approach can not only solve the problem of variable selection but also outperform other competing models. Moreover, according to the lagged process measurements selected by the proposed approach, lagged observations that exerted significant impact on the construction of the control chart pattern recognition model can be identified successfully. This study has significant implications for research and practice in production management and provides a valuable reference for manufacturing process managers to better understand and develop strategies for control chart pattern recognition.     

Phase II monitoring of multivariate simple linear profiles

In some quality control applications, quality of a product or process can be characterized by a relationship between two or more variables that is typically referred to as profile. Moreover, in some situations, there are several correlated quality characteristics, which can be modeled as a set of linear functions of one explanatory variable. We refer to this as multivariate simple linear profiles structure. In this paper, we propose the use of three control chart schemes for Phase II monitoring of multivariate simple linear profiles. The statistical performance of the proposed methods is evaluated in term of average run length criterion and reveals that the control chart schemes are effective in detecting shifts in the process parameters. In addition, the applicability of the proposed methods is illustrated using a real case of calibration application.     

Application of artificial neural networks in linear profile monitoring

In many quality control applications the quality of process or product is characterized and summarized by a relation (profile) between a response variable and one or more explanatory variables. Such profiles can be modeled using linear or nonlinear regression models. In this paper we use artificial neural networks to detect and classify the shifts in linear profiles. Three monitoring methods based on artificial neural networks are developed to monitor linear profiles. Their efficacies are assessed using average run length criterion.     

Comparing statistical and machine learning classifiers: alternatives for predictive modeling in human factors research

Multivariate classification models play an increasingly important role in human factors research. In the past, these models have been based primarily on discriminant analysis and logistic regression. Models developed from machine learning research offer the human factors professional a viable alternative to these traditional statistical classification methods. To illustrate this point, two machine learning approaches--genetic programming and decision tree induction--were used to construct classification models designed to predict whether or not a student truck driver would pass his or her commercial driver license (CDL) examination. The models were developed and validated using the curriculum scores and CDL exam performances of 37 student truck drivers who had completed a 320-hr driver training course. Results indicated that the machine learning classification models were superior to discriminant analysis and logistic regression in terms of predictive accuracy. Actual or potential applications of this research include the creation of models that more accurately predict human performance outcomes.     

Improved principal component monitoring using the local approach

This paper shows that current multivariate statistical monitoring technology may not detect incipient changes in the variable covariance structure nor changes in the geometry of the underlying variable decomposition. To overcome these deficiencies, the local approach is incorporated into the multivariate statistical monitoring framework to define two new univariate statistics for fault detection. Fault isolation is achieved by constructing a fault diagnosis chart which reveals changes in the covariance structure resulting from the presence of a fault. A theoretical analysis is presented and the proposed monitoring approach is exemplified using application studies involving recorded data from two complex industrial processes.     

Statistical monitoring of nonlinear profiles by using piecewise linear approximation

In many practical situations, the quality of a process, or product, is better characterized and summarized by the relationship between a response variable and one or more explanatory variables. Such a relationship between the response variable and explanatory variables is called a profile. Recently, profile monitoring has become a fertile research field in statistical process control (SPC). To handle the nonlinear profile data, the proposal considered in this paper is that the entire curve is broken into several segments of data points that exhibit a statistical fit to the linear model, and therefore each of them can be monitored separately by using existing linear profile SPC methods. A new method that determines the locations of change points based on the slop change is proposed. Two goodness-of-fit criteria are utilized for determining the best number of change points to avoid over-fitting. Two nonlinear profile examples taken from the literature are used to illustrate the proposed change-point model. Monitoring performances using the existing T² and EWMA-based approaches are presented when the nonlinear profile data is fitted by using the proposed change-point model.     

可变抽样区间二元双抽样广义方差控制图

将可变抽样区间特性融入二元双抽样广义方差控制图,构建可变抽样区间二元双抽样广义方差控制图及其监控方法.采用马尔科夫链方法完成控制图调整的平均报警时间和平均报警时间性能指标的计算,进而建立用控制图参数设计的优化模型,并通过遗传算法完成模型求解.通过与可变抽样区间的二元广义方差合成控制图、二元双抽样广义方差控制图以及基于Cornish-Fisher修正的二元双抽样广义方差控制图的判异性能对比,验证该控制图的优势,并通过加工实例对性能优势进行进一步说明.     

Modelling speech emotion recognition using logistic regression and decision trees

Speech emotion recognition has been one of the interesting issues in speech processing over the last few decades. Modelling of the emotion recognition process serves to understand as well as assess the performance of the system. This paper compares two different models for speech emotion recognition using vocal tract features namely, the first four formants and their respective bandwidths. The first model is based on a decision tree and the second one employs logistic regression. Whereas the decision tree models are based on machine learning, regression models have a strong statistical basis. The logistic regression models and the decision tree models developed in this work for several cases of binary classifications were validated by speech emotion recognition experiments conducted on a Malayalam emotional speech database of 2800 speech files, collected from ten speakers. The models are not only simple, but also meaningful since they indicate the contribution of each predictor. The experimental results indicate that speech emotion recognition using formants and bandwidths was better modelled using decision trees, which gave higher emotion recognition accuracies compared to logistic regression. The highest accuracy obtained using decision tree was 93.63%, for the classification of positive valence emotional speech as surprised or happy, using seven features. When using logistic regression for the same binary classification, the highest accuracy obtained was 73%, with eight features.     

Recognition of concurrent control chart patterns using wavelet transform decomposition and multiclass support vector machines

Statistical process control charts have been widely utilized for monitoring process variation in many applications. Nonrandom patterns exhibited by control charts imply certain potential assignable causes that may deteriorate the process performance. Though some effective approaches to recognition of control chart patterns (CCPs) have been developed, most of them only focus on recognition and analysis of single patterns. A hybrid approach by integrating wavelet transform and improved particle swarm optimization-based support vector machine (P-SVM) for on-line recognition of concurrent CCPs is developed in this paper. A statistical correlation coefficient is used to determine whether the input pattern is a single or concurrent CCP. Based on wavelet transform, a raw concurrent pattern signal is decomposed into two basic pattern signals, which can be recognized by multiclass SVMs. The performance of the hybrid approach is evaluated by simulation experiments, and numerical and graphical results are provided to demonstrate that the proposed approach can perform effectively and efficiently in on-line CCP recognition task.     

A general framework for monitoring complex processes with both in-control and out-of-control information

Processes monitoring using multivariate quality variables remains an important and challenging problem in statistical process control (SPC). Although multivariate SPC has been extensively studied in the literature, the challenges associated with designing robust and flexible control schemes have yet to be adequately addressed. This paper develops a general monitoring framework for detecting location shifts in complex processes by employing data mining methods. The historical in-control (IC) and out-of-control (OC) data are combined to set up a support vector machine (SVM) model. The working status of the process is indicated by the probabilistic outputs of the SVM classifier and the multivariate exponentially weighted moving average strategy is applied to construct the control chart. A fast diagnostic procedure can be implemented as soon as the control chart gives an alarm. Our numerical studies show that the proposed control chart is able to deliver satisfactory IC and OC run-length performance regardless of the underlying distributions and data types. An example using real data from an industrial application demonstrates the effectiveness of the proposed method.     

A holistic multi-objective optimization design procedure for ensemble member generation and selection

In the last few years, machine learning techniques have been successfully applied to solve engineering problems. However, owing to certain complexities found in real-world problems, such as class imbalance, classical learning algorithms may not reach a prescribed performance. There can be situations where a good result on different conflicting objectives is desirable, such as true positive and true negative ratios, or it is important to balance model’s complexity and prediction score. To solve such issues, the application of multi-objective optimization design procedures can be used to analyze various trade-offs and build more robust machine learning models. Thus, the creation of ensembles of predictive models using such procedures is addressed in this work. First, a set of diverse predictive models is built by employing a multi-objective evolutionary algorithm. Next, a second multi-objective optimization step selects the previous models as ensemble members, resulting on several non-dominated solutions. A final multi-criteria decision making stage is applied to rank and visualize the resulting ensembles. To analyze the proposed methodology, two different experiments are conducted for binary classification. The first case study is a famous classification problem through which the proposed procedure is illustrated. The second one is a challenging real-world problem related to water quality monitoring, where the proposed procedure is compared to four classical ensemble learning algorithms. Results on this second experiment show that the proposed technique is able to create robust ensembles that can outperform other ensemble methods. Overall, the authors conclude that the proposed methodology for ensemble generation creates competitive models for real-world engineering problems.     

Control chart based on likelihood ratio for monitoring linear profiles

A control chart based on the likelihood ratio is proposed for monitoring the linear profiles. The new chart which integrates the EWMA procedure can detect shifts in either the intercept or the slope or the standard deviation, or simultaneously by a single chart which is different from other control charts in literature for linear profiles. The results by Monte Carlo simulation show that our approach has good performance across a wide range of possible shifts. We show that the new method has competitive performance relative to other methods in literature in terms of ARL, and another feature of the new chart is that it can be easily designed. The application of our proposed method is illustrated by a real data example from an optical imaging system.     

Fault detection using robust multivariate control chart

We introduce a new multivariate statistical process control chart for fault detection using robust statistics and principal component analysis. The proposed approach consists of two main steps. In the first step, a robust covariance matrix is determined using the minimum covariance determinant algorithm. In the second step, an eigen-analysis of the robust correlation matrix is performed to derive the robust control limits of the proposed multivariate chart. Our experimental results illustrate the much better fault detection performance of the proposed method in comparison with existing statistical monitoring and process controlling charts.     

Circular and cylindrical profile monitoring considering spatial correlations

Geometric specifications are important control objects of mechanical components in modern manufacturing. For instance, circularity and cylindricity are essential indicators of high-precision rotary parts. With an increase in the number of measurement points, traditional statistical process control (SPC) methods cannot be applied in many processes because the measurements are highly correlated. During the past two decades, several studies have focused on profile monitoring. A profile, which describes the relationship between independent and response variables, is suitable for large-scale, complex and high-dimensional data monitoring. However, the issue of spatial correlations in measurement points remains unsolved. Considering spatial correlations, this study focuses on circular and cylindrical profiles and proposes a new method combining a spatial correlation model with control charting. SPC methods are utilized to establish control charts and analyze the control processes. The results of simulation and case study indicate that the proposed method is feasible and effective in monitoring circular and cylindrical profiles and can be extended to other geometric specifications.     

Using recurrent neural networks to detect changes in autocorrelated processes for quality monitoring

With the growing of automation in manufacturing, process quality characteristics are being measured at higher rates and data are more likely to be autocorrelated. A widely used approach for statistical process monitoring in the case of autocorrelated data is the residual chart. This chart requires that a suitable model has been identified for the time series of process observations before residuals can be obtained. In this work, a new neural-based procedure, which is alleviated from the need for building a time series model, is introduced for quality control in the case of serially correlated data. In particular, the Elman’s recurrent neural network is proposed for manufacturing process quality control. Performance comparisons between the neural-based algorithm and several control charts are also presented in the paper in order to validate the approach. Different magnitudes of the process mean shift, under the presence of various levels of autocorrelation, are considered. The simulation results indicate that the neural-based procedure may perform better than other control charting schemes in several instances for both small and large shifts. Given the simplicity of the proposed neural network and its adaptability, this approach is proved from simulation experiments to be a feasible alternative for quality monitoring in the case of autocorrelated process data.     

Robust learning from bites for data mining

Some methods from statistical machine learning and from robust statistics have two drawbacks. Firstly, they are computer-intensive such that they can hardly be used for massive data sets, say with millions of data points. Secondly, robust and non-parametric confidence intervals for the predictions according to the fitted models are often unknown. A simple but general method is proposed to overcome these problems in the context of huge data sets. An implementation of the method is scalable to the memory of the computer and can be distributed on several processors to reduce the computation time. The method offers distribution-free confidence intervals for the median of the predictions. The main focus is on general support vector machines (SVM) based on minimizing regularized risks. As an example, a combination of two methods from modern statistical machine learning, i.e. kernel logistic regression and ε-support vector regression, is used to model a data set from several insurance companies. The approach can also be helpful to fit robust estimators in parametric models for huge data sets.     

The random subspace binary logit (RSBL) model for bankruptcy prediction

This paper proposes the random subspace binary logit (RSBL) model (or random subspace binary logistic regression analysis) by taking the random subspace approach and using the classical logit model to generate a group of diverse logit decision agents from various perspectives for predictive problem. These diverse logit models are then combined for a more accurate analysis. The proposed RSBL model takes advantage of both logit (or logistic regression) and random subspace approaches. The random subspace approach generates diverse sets of variables to represent the current problem as different masks. Different logit decision agents from these masks, instead of a single logit model, are constructed. To verify its performance, we used the proposed RSBL model to forecast corporate failure in China. The results indicate that this model significantly improves the predictive ability of classical statistical models such as multivariate discriminant analysis, logit model, and probit model. Thus, the proposed model should make logit model more suitable for predictive problems in academic and industrial uses.     

Favorability functions based on kernel density estimation for logistic models: A case study

Susceptibility or hazard models are often established by means of logistic regression techniques in order to describe the effect of a group of explanatory variables on the probability of a dichotomous or binary response. Since the available variables do not always meet the assumptions of logit-linearity of the logistic regression, a modified approach is proposed. Firstly a favorability function associated with each explanatory variable based on the conditional probability measures is introduced. Next, a simple transformation based on the empirical probability function for non-continuous variables is suggested, while nonparametric kernel estimation is considered for continuous ones. The favorability-based transformations lead to new explanatory variables for the logistic regression model. The performance of the method is evaluated using simulated data. In addition, a real case-study is presented, in which a GIS-based landslides susceptibility model is carried out.