On the estimation of the general parameter

In this paper, we first discuss the origin, developments and various thoughts by several researchers on the generalized linear regression estimator (GREG) due to Deville and Särndal [Deville, J.C., Särndal, C.E., 1992. Calibration estimators in survey sampling. J. Amer. Statist. Assoc. 87, 376-382]. Then, the problem of estimation of the general parameter of interest considered by Rao [Rao, J.N.K., 1994. Estimating totals and distribution functions using auxiliary information at the estimation stage. J. Official Statist. 10 (2), 153-165], and Singh [Singh, S., 2001. Generalized calibration approach for estimating the variance in survey sampling. Ann. Inst. Statist. Math. 53 (2), 404-417; Singh, S., 2004. Golden and Silver Jubilee Year-2003 of the linear regression estimators. In: Proceedings of the Joint Statistical Meeting, Toronto (Available on the CD), 4382-4380; Singh, S., 2006. Survey statisticians celebrate Golden Jubilee Year-2003 of the linear regression estimator. Metrika 1-18] is further investigated. In addition to that it is shown that the Farrell and Singh [Farrell, P.J., Singh, S., 2005. Model-assisted higher order calibration of estimators of variance. Australian & New Zealand J. Statist. 47 (3), 375-383] estimators are also a special case of the proposed methodology. Interestingly, it has been noted that the single model assisted calibration constraint studied by Farrell and Singh [Farrell, P.J., Singh, S., 2002. Re-calibration of higher order calibration weights. Presented at Statistical Society of Canada conference, Hamilton (Available on CD); Farrell, P.J., Singh, S., 2005. Model-assisted higher order calibration of estimators of variance. Australian & New Zealand J. Statist. 47 (3), 375-383] and Wu [Wu, C., 2003. Optimal calibration estimators in survey sampling. Biometrika 90, 937-951] is not helpful for calibrating the Sen [Sen, A.R., 1953. On the estimate of the variance in sampling with varying probabilities. J. Indian Soc. Agril. Statist. 5, 119-127] and Yates and Grundy [Yates, F., Grundy, P.M., 1953. Selection without replacement from within strata with probability proportional to size. J. Roy. Statist. Soc. Ser. 15, 253-261] estimator of the variance of the linear regression estimator under the optimal designs of Godambe and Joshi [Godambe, V.P., Joshi, V.M., 1965. Admissibility and Bayes estimation in sampling finite populations—I. Ann. Math. Statist. 36, 1707-1722]. Three new estimators of the variance of the proposed linear regression type estimator of the general parameters of interest are introduced and compared with each other. The newly proposed two-dimensional linear regression models are found to be useful, unlike a simulation based on a couple of thousands of random samples, in comparing the estimators of variance. The use of knowledge of the model parameters in assisting the estimators of variance has been found to be beneficial. The most attractive feature is that it has been shown theoretically that the proposed method of calibration always remains more efficient than the GREG estimator.     

Averaging, maximum penalized likelihood and Bayesian estimation forimproving Gaussian mixture probability density estimates

We apply the idea of averaging ensembles of estimators to probability density estimation. In particular, we use Gaussian mixture models which are important components in many neural-network applications. We investigate the performance of averaging using three data sets. For comparison, we employ two traditional regularization approaches, i.e., a maximum penalized likelihood approach and a Bayesian approach. In the maximum penalized likelihood approach we use penalty functions derived from conjugate Bayesian priors such that an expectation maximization (EM) algorithm can be used for training. In all experiments, the maximum penalized likelihood approach and averaging improved performance considerably if compared to a maximum likelihood approach. In two of the experiments, the maximum penalized likelihood approach outperformed averaging. In one experiment averaging was clearly superior. Our conclusion is that maximum penalized likelihood gives good results if the penalty term in the cost function is appropriate for the particular problem. If this is not the case, averaging is superior since it shows greater robustness by not relying on any particular prior assumption. The Bayesian approach worked very well on a low-dimensional toy problem but failed to give good performance in higher dimensional problems.     

A new approach to distributed fusion filtering for networked systems with random parameter matrices and correlated noises

This paper is concerned with the distributed filtering problem for a class of discrete-time stochastic systems over a sensor network with a given topology. The system presents the following main features: (i) random parameter matrices in both the state and observation equations are considered; and (ii) the process and measurement noises are one-step autocorrelated and two-step cross-correlated. The state estimation is performed in two stages. At the first stage, through an innovation approach, intermediate distributed least-squares linear filtering estimators are obtained at each sensor node by processing available output measurements not only from the sensor itself but also from its neighboring sensors according to the network topology. At the second stage, noting that at each sampling time not only the measurement but also an intermediate estimator is available at each sensor, attention is focused on the design of distributed filtering estimators as the least-squares matrix-weighted linear combination of the intermediate estimators within its neighborhood. The accuracy of both intermediate and distributed estimators, which is measured by the error covariance matrices, is examined by a numerical simulation example where a four-sensor network is considered. The example illustrates the applicability of the proposed results to a linear networked system with state-dependent multiplicative noise and different network-induced stochastic uncertainties in the measurements; more specifically, sensor gain degradation, missing measurements and multiplicative observation noises are considered as particular cases of the proposed observation model.     

Multi-rate optimal state estimation

This article formulates a multi-rate linear minimum mean squared error (LMMSE) state estimation problem, which includes four rates as follows: the state updating rate in the model, the measurement sampling rate, the estimate updating rate and the estimate output rate. This formulation is unique in two ways. First, the rate ratio between state measurement and state estimate is more general (a rational number), instead of just an integer or its reciprocal as considered in the existing literature. Second, state estimates are produced in blocks, which have never been considered before in the multi-rate estimator design. The multi-rate LMMSE estimation problem is solved by examining several distinctive cases for single-rate state estimation, obtained through the lifting technique. Also, sufficient conditions are given for asymptotic stability of the proposed multi-rate LMMSE estimators. An example in tracking a manoeuvering target is given to illustrate the proposed multi-rate state estimators.     

Domain selection for the varying coefficient model via local polynomial regression

In this article, we consider the varying coefficient model, which allows the relationship between the predictors and response to vary across the domain of interest, such as time. In applications, it is possible that certain predictors only affect the response in particular regions and not everywhere. This corresponds to identifying the domain where the varying coefficient is nonzero. Towards this goal, local polynomial smoothing and penalized regression are incorporated into one framework. Asymptotic properties of our penalized estimators are provided. Specifically, the estimators enjoy the oracle properties in the sense that they have the same bias and asymptotic variance as the local polynomial estimators as if the sparsity is known as a priori. The choice of appropriate bandwidth and computational algorithms are discussed. The proposed method is examined via simulations and a real data example.     

Bayesian inference and prediction of the inverse Weibull distribution for Type-II censored data

This paper describes the Bayesian inference and prediction of the inverse Weibull distribution for Type-II censored data. First we consider the Bayesian inference of the unknown parameter under a squared error loss function. Although we have discussed mainly the squared error loss function, any other loss function can easily be considered. A Gibbs sampling procedure is used to draw Markov Chain Monte Carlo (MCMC) samples, and they have in turn, been used to compute the Bayes estimates and also to construct the corresponding credible intervals with the help of an importance sampling technique. We have performed a simulation study in order to compare the proposed Bayes estimators with the maximum likelihood estimators. We further consider one-sample and two-sample Bayes prediction problems based on the observed sample and provide appropriate predictive intervals with a given coverage probability. A real life data set is used to illustrate the results derived. Some open problems are indicated for further research.     

Optimal mean-square-error calibration of classifier error estimators under Bayesian models

A recently proposed Bayesian modeling framework for classification facilitates both the analysis and optimization of error estimation performance. The Bayesian error estimator is then defined to have optimal mean-square error performance, but in many situations closed-form representations are unavailable and approximations may not be feasible. To address this, we present a method to optimally calibrate arbitrary error estimators for minimum mean-square error performance within a supposed Bayesian framework. Assuming a fixed sample size, classification rule and error estimation rule, as well as a fixed Bayesian model, the calibration is done by first computing a calibration function that maps error estimates to their optimally calibrated values off-line. Once found, this calibration function may be easily applied to error estimates on the fly whenever the assumptions apply. We demonstrate that calibrated error estimators offer significant improvement in performance relative to classical error estimators under Bayesian models with both linear and non-linear classification rules.     

On instrumental variable and total least squares approaches for identification of noisy systems

Many practical system identification problems can be formulated as linear regression problems. The parameter estimates can be computed using instrumental variables (IV) or total least squares (TLS) estimators, both of which have moderate computational complexity. In this work, explicit expressions for the asymptotic covariance matrix of the TLS estimates is derived and is shown to be same as that of the IV method. The accuracy of the parameter estimates for an errors-in-variables model using the above methods has been treated in particular, as standard analysis does not apply. The results obtained from the numerical simulations show that the practical behaviour of the estimators is well predicted by the theoretical results. We provide an explanation why for finite samples, the IV approach is found to be somewhat more robust than the TLS approach. On the other hand, the TLS approach has lower computational load than the IV method.     

Adaptive observers for a class of uniformly observable systems with nonlinear parametrization and sampled outputs

In this paper, we propose an adaptive observer for a class of uniformly observable nonlinear systems with nonlinear parametrization and sampled outputs. A high gain adaptive observer is first designed under the assumption that the output is continuously measured and its exponential convergence is investigated, thanks to a well defined persistent excitation condition. Then, we address the case where the output is available only at (non uniformly spaced) sampling instants. To this end, the continuous-time output observer is redesigned leading to an impulsive observer with a corrective term involving instantaneous state impulses corresponding to the measured samples and their estimates. Moreover, it is shown that the proposed impulsive observer can be put under the form of a hybrid system composed of a continuous-time observer coupled with an inter-sample output predictor. Two design features are worth to be emphasized. Firstly, the observer calibration is achieved through the tuning of a scalar design parameter. Secondly, the exponential convergence to zero of the observation and parameter estimation errors is established under a well defined condition on the maximum value of the sampling partition diameter. More specifically, the observer design is firstly carried out in the case of linear parametrization before being extended to the nonlinear one. The theoretical results are corroborated through simulation results involving a typical bioreactor.     

Design-based approach to k-nearest neighbours technique for coupling field and remotely sensed data in forest surveys

The statistical properties of the k-NN estimators are investigated in a design-based framework, avoiding any assumption about the population under study. The issue of coupling remotely sensed digital imagery with data arising from forest inventories conducted using probabilistic sampling schemes is considered. General results are obtained for the k-NN estimator at the pixel level. When averages (or totals) of forest attributes for the whole study area or sub-areas are of interest, the use of the empirical difference estimator is proposed. The estimator is shown to be approximately unbiased with a variance admitting unbiased or conservative estimators. The performance of the empirical difference estimator is evaluated by an extensive simulation study performed on several populations whose dimensions and covariate values are taken from a real case study. Samples are selected from the populations by means of simple random sampling without replacement. Comparisons with the generalized regression estimator and Horvitz-Thompson estimators are also performed. An application to a local forest inventory on a test area of central Italy is considered.     

考虑有界噪声的摄象机内部参数标定方法

指出摄象机指标定问题中存在的观测误差适于用未知但有界误差（ＵＢＢＥ）模型描述,标定问题可用集内不确定（ＳＭＵ）估计方法解决。针对有关文献研究的一种摄象机内部参数标定问题,提出了利用ＳＭＵ估计方法进行参数标定的新算法。仿真实验结果表明,新算法不权可以在给出标定结果的同时给出标定误差确定的上界,而且可获得较好的标定精度,具有一定的使用价值。     

A robust estimator for the tail index of Pareto-type distributions

In extreme value statistics, the extreme value index is a well-known parameter to measure the tail heaviness of a distribution. Pareto-type distributions, with strictly positive extreme value index (or tail index) are considered. The most prominent extreme value methods are constructed on efficient maximum likelihood estimators based on specific parametric models which are fitted to excesses over large thresholds. Maximum likelihood estimators however are often not very robust, which makes them sensitive to few particular observations. Even in extreme value statistics, where the most extreme data usually receive most attention, this can constitute a serious problem. The problem is illustrated on a real data set from geopedology, in which a few abnormal soil measurements highly influence the estimates of the tail index. In order to overcome this problem, a robust estimator of the tail index is proposed, by combining a refinement of the Pareto approximation for the conditional distribution of relative excesses over a large threshold with an integrated squared error approach on partial density component estimation. It is shown that the influence function of this newly proposed estimator is bounded and through several simulations it is illustrated that it performs reasonably well at contaminated as well as uncontaminated data.     

Smoothed additive estimators for non-error rates in multiple discriminant analysis

A generalization of smoothed additive estimators for non-error rates to the case of more than two groups is discussed. Several properties the smoothing should have are shown to be satisfied. The problem of choosing a smoothing parameter is considered and a parameter choice depending on the sample is proposed. In simulation experiments with normal, uniform and discrete distributions the smoothed additive estimators with fixed and variable smoothing parameter are compared to the leaving-one out method and the resubstitution method with respect to bias and variance.     

A robust estimator for the tail index of Pareto-type distributions

In extreme value statistics, the extreme value index is a well-known parameter to measure the tail heaviness of a distribution. Pareto-type distributions, with strictly positive extreme value index (or tail index) are considered. The most prominent extreme value methods are constructed on efficient maximum likelihood estimators based on specific parametric models which are fitted to excesses over large thresholds. Maximum likelihood estimators however are often not very robust, which makes them sensitive to few particular observations. Even in extreme value statistics, where the most extreme data usually receive most attention, this can constitute a serious problem. The problem is illustrated on a real data set from geopedology, in which a few abnormal soil measurements highly influence the estimates of the tail index. In order to overcome this problem, a robust estimator of the tail index is proposed, by combining a refinement of the Pareto approximation for the conditional distribution of relative excesses over a large threshold with an integrated squared error approach on partial density component estimation. It is shown that the influence function of this newly proposed estimator is bounded and through several simulations it is illustrated that it performs reasonably well at contaminated as well as uncontaminated data.     

Quasi-Monte Carlo estimation in generalized linear mixed models

Generalized linear mixed models (GLMMs) are useful for modelling longitudinal and clustered data, but parameter estimation is very challenging because the likelihood may involve high-dimensional integrals that are analytically intractable. Gauss-Hermite quadrature (GHQ) approximation can be applied but is only suitable for low-dimensional random effects. Based on the Quasi-Monte Carlo (QMC) approximation, a heuristic approach is proposed to calculate the maximum likelihood estimates of parameters in the GLMM. The QMC points scattered uniformly on the high-dimensional integration domain are generated to replace the GHQ nodes. Compared to the GHQ approximation, the proposed method has many advantages such as its affordable computation, good approximation and fast convergence. Comparisons to the penalized quasi-likelihood estimation and Gibbs sampling are made using a real dataset and a simulation study. The real dataset is the salamander mating dataset whose modelling involves six 20-dimensional intractable integrals in the likelihood.     

Quasi-Monte Carlo estimation in generalized linear mixed models

Generalized linear mixed models (GLMMs) are useful for modelling longitudinal and clustered data, but parameter estimation is very challenging because the likelihood may involve high-dimensional integrals that are analytically intractable. Gauss–Hermite quadrature (GHQ) approximation can be applied but is only suitable for low-dimensional random effects. Based on the Quasi-Monte Carlo (QMC) approximation, a heuristic approach is proposed to calculate the maximum likelihood estimates of parameters in the GLMM. The QMC points scattered uniformly on the high-dimensional integration domain are generated to replace the GHQ nodes. Compared to the GHQ approximation, the proposed method has many advantages such as its affordable computation, good approximation and fast convergence. Comparisons to the penalized quasi-likelihood estimation and Gibbs sampling are made using a real dataset and a simulation study. The real dataset is the salamander mating dataset whose modelling involves six 20-dimensional intractable integrals in the likelihood.     

Parameter estimation in a model for misclassified Markov data — a Bayesian approach

We estimate parameters in the context of a discrete-time hidden Markov model with two latent states and two observed states through a Bayesian approach. We provide a Gibbs sampling algorithm for longitudinal data that ensures parameter identifiability. We examine two approaches to start the algorithm for estimation. The first approach generates the initial latent data from transition probability estimates under the false assumption of perfect classification. The second approach requires an initial guess of the classification probabilities and obtains bias-adjusted approximated estimators of the latent transition probabilities based on the observed data. These probabilities are then used to generate the initial latent data set based on the observed data set. Both approaches are illustrated on medical data and the performance of estimates is examined through simulation studies. The approach using bias-adjusted estimators is the best choice of the two options, since it generates a plausible initial latent data set. Our situation is particularly applicable to diagnostic testing, where specifying the range of plausible classification rates may be more feasible than specifying initial values for transition probabilities.     

Efficient direct sampling MCEM algorithm for latent variable models with binary responses

While latent variable models have been successfully applied in many fields and underpin various modeling techniques, their ability to incorporate categorical responses is hindered due to the lack of accurate and efficient estimation methods. Approximation procedures, such as penalized quasi-likelihood, are computationally efficient, but the resulting estimators can be seriously biased for binary responses. Gauss-Hermite quadrature and Markov Chain Monte Carlo (MCMC) integration based methods can yield more accurate estimation, but they are computationally much more intensive. Estimation methods that can achieve both computational efficiency and estimation accuracy are still under development. This paper proposes an efficient direct sampling based Monte Carlo EM algorithm (DSMCEM) for latent variable models with binary responses. Mixed effects and item factor analysis models with binary responses are used to illustrate this algorithm. Results from two simulation studies and a real data example suggest that, as compared with MCMC based EM, DSMCEM can significantly improve computational efficiency as well as produce equally accurate parameter estimates. Other aspects and extensions of the algorithm are discussed.     

Spline smoothing in small area trend estimation and forecasting

Semiparametric models combining both non-parametric trends and small area random effects are now currently being investigated in small area estimation (SAE). These models can prevent bias when the functional form of the relationship between the response and the covariates is unknown. Furthermore, penalized spline regression can be a good tool to incorporate non-parametric regression models into the SAE techniques, as it can be represented as a mixed effects model. A penalized spline model is considered to analyze trends in small areas and to forecast future values of the response. The prediction mean squared error (MSE) for the fitted and the predicted values, together with estimators for those quantities, are derived. The procedure is illustrated with real data consisting of average prices per squared meter of used dwellings in nine neighborhoods of the city of Vitoria, Spain, during the period 1993-2007. Dwelling prices for the next five years are also forecast. A simulation study is conducted to assess the performance of both the small area trend estimator and the prediction MSE estimators. The results confirm a good behavior of the proposed estimators in terms of bias and variability.