Forecasting nonlinear time series with neural network sieve bootstrap

A new method to construct nonparametric prediction intervals for nonlinear time series data is proposed. Within the framework of the recently developed sieve bootstrap, the new approach employs neural network models to approximate the original nonlinear process. The method is flexible and easy to implement as a standard residual bootstrap scheme while retaining the advantage of being a nonparametric technique. It is model-free within a general class of nonlinear processes and avoids the specification of a finite dimensional model for the data generating process. The results of a Monte Carlo study are reported in order to investigate the finite sample performances of the proposed procedure.     

A bootstrap technique for nearest neighbor classifier design

A bootstrap technique for nearest neighbor classifier design is proposed. Our primary interest in designing a classifier is in small training sample size situations. Conventional bootstrapping techniques sample the training samples with replacement. On the other hand, our technique generates bootstrap samples by locally combining original training samples. The nearest neighbor classifier is designed on the bootstrap samples and is tested on the test samples independent of training samples. The performance of the proposed classifier is demonstrated on three artificial data sets and one real data set. Experimental results show that the nearest neighbor classifier designed on the bootstrap samples outperforms the conventional k-NN classifiers as well as the edited 1-NN classifiers, particularly in high dimensions     

基于Bootstrap多神经网络的软测量方法

针对原油蒸馏过程常规软测量模型难以适应原油进料性质变化的问题,提出Bootstrap多神经网络的非线性软测量处理策略.通过Bootstrap算法复制出训练集样本空间上的多个样本子空间,训练出多神经网络模型,避免了单个神经网络易于陷入局部最优及过度训练的弱点,具有较高的准确率和泛化能力.本处理策略用于建立常压塔一线干点的软测量模型,仿真结果表明模型预测准确率和鲁棒性较好,对原油性质变化具有较好的适应性.该方法将会改进实际蒸馏过程在进料性质变化情况下的产品质量指标的软测量精度.     

Nonparametric estimation of conditional ROC curves: Application to discrimination tasks in computerized detection of early breast cancer

A local linear method for estimating the conditional ROC curve under the presence of continuous and categorical covariates is introduced. A data driven smoothing parameter selector based on the bootstrap is proposed. The methods are illustrated with real data from a discrimination problem emerging in the context of computer-aided diagnosis. The bootstrap approach is also used to construct pointwise confidence intervals for the area under the ROC curve.     

A Sieve Bootstrap Method for Correlation Analysis

This note presents a nonparametric sieve bootstrap method for estimating the variance of impulse response coefficients and the process steady-state gain determined via correlation analysis. The bootstrap estimates are demonstrated to be better for small samples than the analytical finite sample variance expression for the simplified form (assuming white noise input) of the Wiener-Hopf equations. Monte Carlo simulations demonstrate that solving the linear equations resulting from the Wiener-Hopf equations can result in a variance reduction.     

A smoothed bootstrap test for independence based on mutual information

A test for independence of multivariate time series based on the mutual information measure is proposed. First of all, a test for independence between two variables based on i.i.d. (time-independent) data is constructed and is then extended to incorporate higher dimensions and strictly stationary time series data. The smoothed bootstrap method is used to estimate the null distribution of mutual information. The experimental results reveal that the proposed smoothed bootstrap test performs better than the existing tests and can achieve high powers even for moderate dependence structures. Finally, the proposed test is applied to assess the actual independence of components obtained from independent component analysis (ICA).     

Model based bootstrap methods for interval censored data

The performance of model based bootstrap methods for constructing point-wise confidence intervals around the survival function with interval censored data is investigated. It is shown that bootstrapping from the nonparametric maximum likelihood estimator of the survival function is inconsistent for the current status model. A model based smoothed bootstrap procedure is proposed and proved to be consistent. In fact, a general framework for proving the consistency of any model based bootstrap scheme in the current status model is established. In addition, simulation studies are conducted to illustrate the (in)-consistency of different bootstrap methods in mixed case interval censoring. The conclusions in the interval censoring model would extend more generally to estimators in regression models that exhibit non-standard rates of convergence.     

On properties of predictors derived with a two-step bootstrap model averaging approach—A simulation study in the linear regression model

In many applications of model selection there is a large number of explanatory variables and thus a large set of candidate models. Selecting one single model for further inference ignores model selection uncertainty. Often several models fit the data equally well. However, these models may differ in terms of the variables included and might lead to different predictions. To account for model selection uncertainty, model averaging procedures have been proposed. Recently, an extended two-step bootstrap model averaging approach has been proposed. The first step of this approach is a screening step. It aims to eliminate variables with negligible effect on the outcome. In the second step the remaining variables are considered in bootstrap model averaging. A large simulation study is performed to compare the MSE and coverage rate of models derived with bootstrap model averaging, the full model, backward elimination using Akaike and Bayes information criterion and the model with the highest selection probability in bootstrap samples. In a data example, these approaches are also compared with Bayesian model averaging. Finally, some recommendations for the development of predictive models are given.     

Inferential estimation of kerosene dry point in refineries with varying crudes

A bootstrap aggregated model approach to the estimation of product quality in refineries with varying crudes is proposed in this paper. The varying crudes cause the relationship between process variables and product quality variables to change, which makes product quality estimation by soft-sensors a difficult problem. The essential idea in this paper is to build an inferential estimation model for each type of feed oil and use an on-line feed oil classifier to determine the feed oil type. Bootstrap aggregated neural networks are used in developing the on-line feed oil classifier and a bootstrap aggregated partial least square regression model is developed for each data group corresponding to each type of feed crude oil. The amount of training data in crude oil distillation is usually small and this brings difficulties for classification and estimation modelling. In order to enhance model reliability and robustness, bootstrap aggregated models are developed. The inferential estimation results of kerosene dry point on both simulated data and industrial data show that the proposed method can significantly improve the overall inferential estimation performance.     

Modified fast double sieve bootstraps for ADF tests

This paper studies the finite sample performance of the sieve bootstrap augmented Dickey-Fuller (ADF) unit root test. It is well known that this test’s accuracy in terms of rejection probability under the null depends greatly on the underlying DGP. Through extensive simulations, we find that it also depends on the number of lags employed in the bootstrap DGP and in the bootstrap ADF regression. Based on this finding and using some well established theoretical results, we propose a simple modification that significantly improves the test’s accuracy. We also introduce different versions of the fast double bootstrap, each modified according to the same theoretical basis. According to our simulations, these new testing procedures have lower error in rejection probability under the null while retaining good power.     

An empirical study of a test for polynomial relationships in randomly right censored regression models

In this paper, a test statistic is constructed to test polynomial relationships in randomly right censored regression models based on the local polynomial smoothing technique. Two bootstrap procedures, namely the residual-based bootstrap and the naive bootstrap procedures, are suggested to derive the p-value of the test. Some simulations are conducted to empirically assess the performance of the two bootstrap procedures. The results demonstrate that the residual-based bootstrap performs much better than the naive bootstrap and the test method with the residual-based bootstrap to derive the p-value works satisfactorily. Although the limiting distribution of the test statistic and the consistency of the bootstrap approximations remain to be investigated, simulation results indicate that the proposed test method may be of some practical use. As a real example, the proposed test is applied to the Stanford heart transplant data.     

Ensemble classification of paired data

In many medical applications, data are taken from paired organs or from repeated measurements of the same organ or subject. Subject based as opposed to observation based evaluation of these data results in increased efficiency of the estimation of the misclassification rate. A subject based approach for classification in the generation of bootstrap samples of bagging and bundling methods is analyzed. A simulation model is used to compare the performance of different strategies to create the bootstrap samples which are used to grow individual trees. The proposed approach is compared to linear discriminant analysis, logistic regression, random forests and gradient boosting. Finally, the simulation results are applied to glaucoma diagnosis using both eyes of glaucoma patients and healthy controls. It is demonstrated that the proposed subject based resampling reduces the misclassification rate.     

Batch Process Modelling and Optimal Control Based on Neural Network Model

This paper presents several neural network based modelling, reliable optimal control, and iterative learning control methods for batch processes. In order to overcome the lack of robustness of a single neural network, bootstrap aggregated neural networks are used to build reliable data based empirical models. Apart from improving the model generalisation capability, a bootstrap aggregated neural network can also provide model prediction confidence bounds. A reliable optimal control method by incorporating model prediction confidence bounds into the optimisation objective function is presented. A neural network based iterative learning control strategy is presented to overcome the problem due to unknown disturbances and model-plant mismatches. The proposed methods are demonstrated on a simulated batch polymerisation process.     

Batch Process Modelling and Optimal Control Based on Neural Network Models

This paper presents several neural network based modelling, reliable optimal control, and iterative learning control methods for batch processes. In order to overcome the lack of robustness of a single neural network, bootstrap aggregated neural networks are used to build reliable data based empirical models. Apart from improving the model generalisation capability, a bootstrap aggregated neural network can also provide model prediction confidence bounds. A reliable optimal control method by incorporating model prediction confidence bounds into the optimisation objective function is presented. A neural network based iterative learning control strategy is presented to overcome the problem due to unknown disturbances and model-plant mismatches. The proposed methods are demonstrated on a simulated batch polymerisation process.     

GEE type inference for clustered zero-inflated negative binomial regression with application to dental caries

Use of zero-inflated count data models is common in applications where the number of zero counts exceeds that predicted from a traditional count data model such as Poisson or negative binomial. When count data exhibiting inflated zero counts are correlated among subjects, a natural approach will be to fit a marginal model with the help of generalized estimating equations (GEE) that can incorporate subject-to-subject correlations. A GEE based zero-inflated negative binomial (ZINB) model is proposed to fit clustered counts with excessive zeros. However, the corresponding sandwich variance estimator appears to underestimate the true variance. The theoretical reasons for its failure are explained and a correction under additional modeling assumptions is offered. In addition, a clustered resampling (bootstrap) procedure is proposed to estimate the variance and it is shown that the bootstrap procedure captures the correct variance under no additional model assumptions. Utility of this marginal GEE based ZINB model over two other competing models has been assessed using a thorough simulation study. The resulting inference procedure is applied to study the association between the dental caries and fluoride exposures using a dataset extracted from the Iowa Fluoride Study. A number of risk factors of clinical significance are reliably identified using the proposed model.     

Bootstrap estimated true and false positive rates and ROC curve

Diagnostic studies and new biomarkers are assessed by the estimated true and false positive rates of the classification rule. One diagnostic rule is considered for high-dimensional predictor data. Cross-validation and the leave-one-out bootstrap are discussed to estimate true and false positive rates of classifiers by the machine learning methods Adaboost, Bagging, Random Forest, (penalized) logistic regression and support vector machines. The .632+ bootstrap estimation of the misclassification error has been previously proposed to adjust the overfitting of the apparent error. This idea is generalized to the estimation of true and false positive rates. Tree-based simulation models with 8 and 50 binary non-informative variables are analysed to examine the properties of the estimators. Finally, a bootstrap estimation of receiver operating characteristic (ROC) curves is suggested and a .632+ bootstrap estimation of ROC curves is discussed. This approach is applied to high-dimensional gene expression data of leukemia and predictors of image data for glaucoma diagnosis.     

On the use of the bootstrap for estimating functions with functional data

The bootstrap methodology for functional data and functional estimation target is considered. A Monte Carlo study analyzing the performance of the bootstrap confidence bands (obtained with different resampling methods) of several functional estimators is presented. Some of these estimators (e.g., the trimmed functional mean) rely on the use of depth notions for functional data and do not have received yet much attention in the literature. A real data example in cardiology research is also analyzed. In a more theoretical aspect, a brief discussion is given providing some insights on the asymptotic validity of the bootstrap methodology when functional data, as well as a functional parameter, are involved.     

On the efficiency of bootstrap method into the analysis contingency table

The bootstrap method is a computer intensive statistical method that is widely used in performing nonparametric inference. Categorical data analysis, in particular the analysis of contingency tables, is commonly used in applied field. This work considers nonparametric bootstrap tests for the analysis of contingency tables. There are only a few research papers which exploit this field. The p-values of tests in contingency tables are discrete and should be uniformly distributed under the null hypothesis. The results of this article show that corresponding bootstrap versions work better than the standard tests. Properties of the proposed tests are illustrated and discussed using Monte Carlo simulations. This article concludes with an analytical example that examines the performance of the proposed tests and the confidence interval of the association coefficient.     

Performance assessment through bootstrap

A new performance evaluation paradigm for computer vision systems is proposed. In real situation, the complexity of the input data and/or of the computational procedure can make traditional error propagation methods infeasible. The new approach exploits a resampling technique recently introduced in statistics, the bootstrap. Distributions for the output variables are obtained by perturbing the nuisance properties of the input, i.e., properties with no relevance for the output under ideal conditions. From these bootstrap distributions, the confidence in the adequacy of the assumptions embedded into the computational procedure for the given input is derived. As an example, the new paradigm is applied to the task of edge detection. The performance of several edge detection methods is compared both for synthetic data and real images. The confidence in the output can be used to obtain an edgemap independent of the gradient magnitude     

改进的裂变自举粒子滤波算法在GPS导航系统中的应用

为了提高全球定位系统GPS的定位精度和可靠性,提出了一种基于裂变自举粒子滤波FBPF的GPS定位系统算法。根据GPS输出的参数之间的相互联系建立了系统的状态方程,并应用于对GPS接收数据进行滤波处理的改进FBPF算法中。GPS导航系统中采用DSP对GPS接收机的输出信号进行译码和滤波处理。