Applications of the jackknife procedure in ridge regression

Three aspects of the application of the jackknife technique to ridge regression are considered, viz. as a bias estimator, as a variance estimator, and as an indicator of observations influence on parameter estimates. The ridge parameter is considered non-stochastic. The jackknifed ridge estimator is found to be a ridge estimator with a smaller value on the ridge parameter. Hence it has a smaller bias but a larger variance than the ridge estimator. The variance estimator is expected to be robust against heteroscedastic error variance as well as against outliers. A measure of observations influence on the estimates of regression parameters is proposed.     

Error analysis of modeling and bias errors in continuous time state estimation

This paper derives new exact and approximate algorithms for the computation of modeling and bias errors in linear minimum error variance estimation. The primary difference between the exact algorithms and those previously presented is their form. A result concerning orthogonal projections for suboptimal estimation leads to “delta” error analysis algorithms for the difference between the true error variance and the optimum system error variance. These algorithms often simplify computational problems considerably compared to previously obtained algorithms and adapt easily to sensitivity analysis accurate regardless of parameter variation magnitude.     

A quantile estimation for massive data with generalized Pareto distribution

This paper proposes a new method of estimating extreme quantiles of heavy-tailed distributions for massive data. The method utilizes the Peak Over Threshold (POT) method with generalized Pareto distribution (GPD) that is commonly used to estimate extreme quantiles and the parameter estimation of GPD using the empirical distribution function (EDF) and nonlinear least squares (NLS). We first estimate the parameters of GPD using EDF and NLS and then, estimate multiple high quantiles for massive data based on observations over a certain threshold value using the conventional POT. The simulation results demonstrate that our parameter estimation method has a smaller Mean square error (MSE) than other common methods when the shape parameter of GPD is at least 0. The estimated quantiles also show the best performance in terms of root MSE (RMSE) and absolute relative bias (ARB) for heavy-tailed distributions.     

Development and implementation of the population Fisher information matrix for the evaluation of population pharmacokinetic designs

In population pharmacokinetic studies, the precision of parameter estimates is dependent on the population design. Methods based on the Fisher information matrix have been developed and extended to population studies to evaluate and optimize designs. In this paper we propose simple programming tools to evaluate population pharmacokinetic designs. This involved the development of an expression for the Fisher information matrix for nonlinear mixed-effects models, including estimation of the variance of the residual error. We implemented this expression as a generic function for two software applications: S-PLUS and MATLAB. The evaluation of population designs based on two pharmacokinetic examples from the literature is shown to illustrate the efficiency and the simplicity of this theoretic approach. Although no optimization method of the design is provided, these functions can be used to select and compare population designs among a large set of possible designs, avoiding a lot of simulations.     

Error growth in position estimation from noisy relative pose measurements

We examine how the estimation error grows with time when a mobile robot estimates its location from relative pose measurements without global position or orientation sensors. We show that, in both two-dimensional and three-dimensional space, both the bias and the variance of the position estimation error grows at most linearly with time asymptotically. Non-asymptotic bounds on the bias and variance are obtained, which provide insight into the mechanism of error growth. The bias is crucially dependent on the trajectory of the robot. Conclusions on the asymptotic growth rate of the bias continue to hold even with unbiased measurements or error-free translation measurements. Exact formulas for the bias and the variance of the position estimation error are provided for two specific two-dimensional trajectories–straight line and periodic. Experiments with a P3-DX wheeled robot and Monte Carlo simulations are provided to verify the theoretical predictions. A method to reduce the bias is proposed based on the lessons learned.     

An improved strategy for regression of biophysical variables and Landsat ETM+ data

Empirical models are important tools for relating field-measured biophysical variables to remote sensing data. Regression analysis has been a popular empirical method of linking these two types of data to provide continuous estimates for variables such as biomass, percent woody canopy cover, and leaf area index (LAI). Traditional methods of regression are not sufficient when resulting biophysical surfaces derived from remote sensing are subsequently used to drive ecosystem process models. Most regression analyses in remote sensing rely on a single spectral vegetation index (SVI) based on red and near-infrared reflectance from a single date of imagery. There are compelling reasons for utilizing greater spectral dimensionality, and for including SVIs from multiple dates in a regression analysis. Moreover, when including multiple SVIs and/or dates, it is useful to integrate these into a single index for regression modeling. Selection of an appropriate regression model, use of multiple SVIs from multiple dates of imagery as predictor variables, and employment of canonical correlation analysis (CCA) to integrate these multiple indices into a single index represent a significant strategic improvement over existing uses of regression analysis in remote sensing.To demonstrate this improved strategy, we compared three different types of regression models to predict LAI for an agro-ecosystem and live tree canopy cover for a needleleaf evergreen boreal forest: traditional (Y on X) ordinary least squares (OLS) regression, inverse (X on Y) OLS regression, and an orthogonal regression method called reduced major axis (RMA). Each model incorporated multiple SVIs from multiple dates and CCA was used to integrate these. For a given dataset, the three regression-modeling approaches produced identical coefficients of determination and intercepts, but different slopes, giving rise to divergent predictive characteristics. The traditional approach yielded the lowest root mean square error (RMSE), but the variance in the predictions was lower than the variance in the observed dataset. The inverse method had the highest RMSE and the variance was inflated relative to the variance of the observed dataset. RMA provided an intermediate set of predictions in terms of the RMSE, and the variance in the observations was preserved in the predictions. These results are predictable from regression theory, but that theory has been essentially ignored within the discipline of remote sensing.     

Determination of stoichiometric association constants by a non-iterative computational method

Computational methods for the estimation of stoichiometric association constants for multiple-ligand binding systems are currently based on non-linear least-squares regression analysis. These computational methods require sophisticated, iterative algorithms to assure convergence to a solution, as well as initial parameter and error estimates. A simple procedure, called lambda-invariance testing (LIT), was developed that provides a single-pass (non-iterative) estimation of stoichiometric association constants. The LIT method was applied to simulated binding data containing Gaussian error and to real data drawn from the literature. This method provided parameter estimates essentially equivalent to those obtained by least-squares regression analysis, with no initial parameter or error estimates required.     

Asymptotic variance estimation for the misclassification SIMEX

Most epidemiological studies suffer from misclassification in the response and/or the covariates. Since ignoring misclassification induces bias on the parameter estimates, correction for such errors is important. For measurement error, the continuous analog to misclassification, a general approach for bias correction is the SIMEX (simulation extrapolation) method. This approach has been recently extended to regression models with a possibly misclassified categorical response and/or the covariates and is called the MC-SIMEX approach. In order to assess the importance of a regressor not only its (corrected) estimate is needed, but also its standard error. Based on the original SIMEX approach a method which uses asymptotic expansions to estimate the asymptotic variance is developed. The asymptotic variance estimators for the MC-SIMEX approach are derived. The case when the misclassification probabilities are estimated by a validation study is also included. An extensive simulation study shows the good performance of the new approach. It is illustrated by an example in caries research including a logistic regression model, where the response and a binary covariate are possibly misclassified.     

The determination of the absolute bioavailability for drug substances with long elimination half-lives (with PC-programs for the method of truncated areas).

A survey is given of the methods for the determination of the absolute bioavailability for drug substances with long elimination half-lives. All methods depend on the assumption of linear pharmacokinetics. Emphasis is laid on Urso and Aarons' regression method of truncated areas-under-the-curve (AUC) to obtain estimates of the absolute bioavailability, F, and the absorption rate constant, ka, without obeying the otherwise general rule of following blood or plasma levels for at least three times the terminal half-life. It is shown by simulation when the linear and when the logarithmic trapezoidal rule is to be used for the calculation of the AUC. Two PC-programs for the regression method of truncated areas are presented: one program (SIMF&KA.EXE) simulates, based on tentative pharmacokinetic parameters, whether a blood sampling scheme, together with the precision of the analytical assay, is adequate to obtain accurate and precise estimates of F and ka; the method is insensitive to large variations in the tentative value of the elimination half-life. The other program (ESTF&KA.EXE) provides estimates of F and ka, with standard deviations of the estimates, from empirical data.     

图像中广义高斯噪声形状参数的近似估计

针对图像中广义高斯噪声的形状参数p通常为未知的问题,提出一种用于计算p的近似矩估计算法。从原图像中提取噪声样本,采用分段函数对p的比率函数进行数值拟合,从而得到p的近似表达式。实验结果表明,当噪声样本值准确时,p的估计值能精确到小数点后2位,尤其适用于均匀噪声的情况,估计误差比传统算法小0.3;当噪声样本值不准确时,估计精度与门限K的选择有关。     

Spectral estimation of Green leaf area index of oats

Green leaf area index (LAI) is a measure of vegetative growth and development and is frequently used as an input parameter in yield estimation and evapotranspiration models. Extensive destructive sampling is usually required to achieve accurate estimates of green LAI in natural situations. In this investigation, a statistical modeling approach was used to predict the green LAI of oats from bidirectional reflectance data collected with multiband radiometers. Stepwise multiple regression models based on two sets of spectral reflectance factors accounted for 73% and 65% of the variance in green LAI of oats. Exponential models of spectral data transformations of greenness, normalized difference, and near-infrared/red ratio accounted for more of the variance in green LAI than the multiple regression models.     

Trading variance reduction with unbiasedness: the regularized subspace information criterion for robust model selection in kernel regression

A well-known result by Stein (1956) shows that in particular situations, biased estimators can yield better parameter estimates than their generally preferred unbiased counterparts. This letter follows the same spirit, as we will stabilize the unbiased generalization error estimates by regularization and finally obtain more robust model selection criteria for learning. We trade a small bias against a larger variance reduction, which has the beneficial effect of being more precise on a single training set. We focus on the subspace information criterion (SIC), which is an unbiased estimator of the expected generalization error measured by the reproducing kernel Hilbert space norm. SIC can be applied to the kernel regression, and it was shown in earlier experiments that a small regularization of SIC has a stabilization effect. However, it remained open how to appropriately determine the degree of regularization in SIC. In this article, we derive an unbiased estimator of the expected squared error, between SIC and the expected generalization error and propose determining the degree of regularization of SIC such that the estimator of the expected squared error is minimized. Computer simulations with artificial and real data sets illustrate that the proposed method works effectively for improving the precision of SIC, especially in the high-noise-level cases. We furthermore compare the proposed method to the original SIC, the cross-validation, and an empirical Bayesian method in ridge parameter selection, with good results.     

不等精度数据融合的最优权值与参数估计方法

对测量数据进行融合处理是提高数据处理精度的一个十分有效的方法。本文使用引入加权因子的方法,对不等精度测量数据融合处理的最优加权与参数估计的问题进行了研究。对于线性回归模型,从理论上证明了,最优融合权值由数据的精度唯一确定,这与经典的Gauss-Markov定理是一致的;对于非线性回归模型,在理论上获得了最优融合权值的计算方法,给出了实际数据融合处理的最优权值与参数估计算法,并且证明了非线性模型的不等精度数据融合的最优权值,不但与数据本身精度相关联,而且与模型的结构、模型的导数相关联,则在此时经典Gauss-Markov定理不再成立。通过算例进行对比验证,证明了所提方法的有效性。     

Estimation of timber volume and stem density based on scanning laser altimetry and expected tree size distribution functions

Laser scanners of small footprint diameter and high sampling density provide possibility to obtain accurate height information on the forest canopy. When applying tree crown segmentation methods, individual single trees can be recognised and tree height as well as crown area can be detected. Detection of suppressed trees from a height model based on laser scanning is difficult; however, it is possible to predict these trees by using theoretical distribution functions. In this study, two different methods are used to predict small trees. In the first method, the parameter prediction method is utilised with the complete Weibull distribution, the parameters of which are predicted with separate parameter prediction models; thus, small trees are determined from the predicted tree height distribution. In the second method, the two-parameter left-truncated Weibull distribution is fitted to the detected tree height distribution.The results are presented by using timber volume and stem density as predicted stand characteristics. The results showed that the root mean square error (RMSE) for the timber volume is about 25% when using only information obtained from laser scanning, whereas the RMSE for the number of stems per ha is about 75%. Predictions for both characteristics are also highly biased and the underestimates are 24% and 62%, respectively. The use of the parameter prediction method to describe small trees improved the accuracy considerably; the RMSE figures for estimates of timber volume and number of stems are 16.0% and 49.2%, respectively. The bias for the estimates is also decreased to 6.3% for timber volume and 8.2% for the number of stems. When a left-truncated height distribution is used to predict the heights of the missing small trees, the RMSEs for the estimates of timber volume and number of stems are 22.5% and 72.7%, respectively. In the case of the timber volume, the reliability figures for both the original laser scanning-based estimates and for the estimates that also contain small trees are comparable to those obtained by conventional compartment-wise Finnish field inventories.     

An improved method to estimate reference cloud-free images for the visible band of geostationary satellites

Geostationary images have been used frequently in the past 50 years to derive geophysical information. As a complement to all-sky observations, clear-sky counterparts play an important role in the derivation of cloud properties. We investigated ways to improve estimates of top-of-atmosphere (TOA) visible clear-sky images, over the full spatial and temporal resolution of Meteosat First Generation (MFG) satellites. Estimation was based on TOA measurements in MFG’s visible channel, collected for a specific time of the day over the span of several days. In addition, a cloud climatology aided estimation.

Parameter optimization and the introduction of a spatial filter over ocean resulted in a bias of ?1.0 to ?2.0 digital counts (DC) and a root mean square error (RMSE) of 2.0–3.0 DC when averaged over the complete field of view. This excludes the Spring period which has up to ?3.5 DC bias and up to 5.5 DC RMSE. Reasons for these exceptional differences were found in rapid greenness change, affecting reflectances over vegetated surfaces, and dust storms, with an effect over tropical land and ocean surfaces. Similarly, sea ice and snow affected polar regions seasonally. Applied to 24 years of MFG imagery, we successfully used improved clear-sky estimates to stably detect clouds. Additionally, these clear-sky estimates may prove useful for characterization of instrument degradation as well as cloud feedback studies.     

Transformation-based estimation

To alleviate the computational burden of making the relevant estimation algorithms stable for nonlinear and semiparametric regression models with, particularly, high-dimensional data, a transformation-based method combining sufficient dimension reduction approach is proposed. To this end, model-independent transformations are introduced to models under study. This generic methodology can be applied to transformation models; generalized linear models; and their corresponding quantile regression variants. The constructed estimates almost have closed forms in certain sense such that the above goals can be achieved. Simulation results show that, in finite sample cases with high-dimensional predictors and long-tailed distributions of error, the new estimates often exhibit a smaller degree of variance, and have much less computational burden than the classical methods such as the classical least squares and quantile regression estimation.     

Maximum posterior probability estimators of map accuracy

Assessing the accuracy of land cover maps is often prohibitively expensive because of the difficulty of collecting a statistically valid probability sample from the classified map. Even when post-classification sampling is undertaken, cost and accessibility constraints may result in imprecise estimates of map accuracy. If the map is constructed via supervised classification, then the training sample provides a potential alternative source of data for accuracy assessment. Yet unless the training sample is collected by probability sampling, the estimates are, at best, of uncertain quality, and may be substantially biased. This article discusses a new approach to map accuracy assessment based on maximum posterior probability estimators. Maximum posterior probability estimators are resistant to bias induced by non-representative sampling, and so are intended for situations in which the training sample is collected without using a statistical sampling design. The maximum posterior probability approach may also be used to increase the precision of estimates obtained from a post-classification sample. In addition to discussing maximum posterior probability estimators, this article reports on a simulation study comparing three approaches to estimating map accuracy: 1) post-classification sampling, 2) resampling the training sample via cross-validation, and 3) maximum posterior probability estimation. The simulation study showed substantial reductions in bias and improvements in precision in comparisons of maximum posterior probability and cross-validation estimators when the training sample was not representative of the map. In addition, combining an ordinary post-classification estimator and the maximum posterior probability estimator produced an estimator that was at least, and usually more precise than the ordinary post-classification estimator.     

基于Lorenz-63模型的状态与参数同时估计方法对比研究

模型状态同化精度受多种方面因素的影响,针对状态同化中模型参数的不确定性问题,状态与参数同时估计为此提供了一种较好的解决方案,即在进行状态同化的过程中得到合理的参数估计值。在Lorenz-63模型的基础上构建状态与参数同时估计框架,比较分析增广集合卡尔曼滤波(AEnKF,Augmented Ensemble Kalman Filter)、双重集合卡尔曼滤波(DEnKF,Dual Ensemble Kalman Filter)和同时优化与同化方法(SODA,Simultaneous Optimization and Data Assimilation)在集合数、观测误差和观测数不同时的参数和状态估计结果差异,由此探讨3种方法的优劣及适用性。研究结果表明:3种方法都能较好地估计模型的状态和参数,AEnKF的误差在集合数不大于20时最大,随着集合数增加降低的速率最小;3种方法的RMSE值随观测误差的增大而增大,但算法间差异不大;观测数变为1时3种方法的结果都变差,其中AEnKF最明显。     

A sampling algorithm for bandwidth estimation in a nonparametric regression model with a flexible error density

The unknown error density of a nonparametric regression model is approximated by a mixture of Gaussian densities with means being the individual error realizations and variance a constant parameter. Such a mixture density has the form of a kernel density estimator of error realizations. An approximate likelihood and posterior for bandwidth parameters in the kernel-form error density and the Nadaraya–Watson regression estimator are derived, and a sampling algorithm is developed. A simulation study shows that when the true error density is non-Gaussian, the kernel-form error density is often favored against its parametric counterparts including the correct error density assumption. The proposed approach is demonstrated through a nonparametric regression model of the Australian All Ordinaries daily return on the overnight FTSE and S&P 500 returns. With the estimated bandwidths, the one-day-ahead posterior predictive density of the All Ordinaries return is derived, and a distribution-free value-at-risk is obtained. The proposed algorithm is also applied to a nonparametric regression model involved in state-price density estimation based on S&P 500 options data.