Combining Unbiased Ridge and Principal Component Regression Estimators

In the presence of multicollinearity problem, ordinary least squares (OLS) estimation is inadequate. To circumvent this problem, two well-known estimation procedures often suggested are the unbiased ridge regression (URR) estimator given by Crouse et al. (1995 Crouse , R. , Jin , C. , Hanumara , R. ( 1995 ). Unbiased ridge estimation with prior information and ridge trace . Commun. Statist. Theor. Meth. 24 : 2341 – 2354 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and the (r, k) class estimator given by Baye and Parker (1984 Baye , M. , Parker , D. ( 1984 ). Combining ridge and principal component regression: a money demand illustration . Commun. Statist. Theor. Meth. 13 : 197 – 205 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). In this article, we proposed a new class of estimators, namely modified (r, k) class ridge regression (MCRR) which includes the OLS, the URR, the (r, k) class, and the principal components regression (PCR) estimators. It is based on a criterion that combines the ideas underlying the URR and the PCR estimators. The standard properties of this new class estimator have been investigated and a numerical illustration is done. The conditions under which the MCRR estimator is better than the other two estimators have been investigated.     

A New Two-Parameter Estimator in Linear Regression

This article is concerned with the parameter estimation in linear regression model. To overcome the multicollinearity problem, a new two-parameter estimator is proposed. This new estimator is a general estimator which includes the ordinary least squares (OLS) estimator, the ridge regression (RR) estimator, and the Liu estimator as special cases. Necessary and sufficient conditions for the superiority of the new estimator over the OLS, RR, Liu estimators, and the two-parameter estimator proposed by Ozkale and Kaciranlar (2007 Ozkale , M. R. , Kaciranlar , S. ( 2007 ). The restricted and unrestricted two-parameter estimators . Commun. Statist. Theor. Meth. 36 : 2707 – 2725 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) in the mean squared error matrix (MSEM) sense are derived. Furthermore, we obtain the estimators of the biasing parameters and give a numerical example to illustrate some of the theoretical results.     

The Research on Two Kinds of Restricted Biased Estimators Based on Mean Squared Error Matrix

Sakall?oglu et al. (2001 Sakall?oglu , Kaç?ranlar , Akdeniz ( 2001 ). Mean squared error comparisons of some biased estimators . Commun. Statist. Theor. Meth. 30 : 347 – 361 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) dealt with the comparisons among the ridge estimator, Liu estimator, and iteration estimator. Akdeniz and Erol (2003 Akdeniz , F. , Erol , H. ( 2003 ). Mean squared error matrix comparisons of some biased estimators in linear regression . Commun. Statist. Theor. Meth. 32 : 2389 – 2413 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) have compared the (almost unbiased) generalized ridge regression estimator with the (almost unbiased) generalized Liu estimator in the matrix mean squared error sense. In this article, we study the ridge estimator and Liu estimator with respect to linear equality restriction, and establish some sufficient conditions for the superiority of the restricted ridge estimator over the restricted Liu estimator and the superiority of the restricted Liu estimator over the restricted ridge estimator under mean squared error matrix, respectively. Furthermore, we give a numerical example.     

MSE Performance and Minimax Regret Significance Points for a HPT Estimator when each Individual Regression Coefficient is Estimated

In this article, we consider a heterogeneous preliminary test (HPT) estimator whose components are the OLS and feasible ridge regression (FRR) estimators, and derive the exact formulae for the moments of the HPT estimator using mathematical method. Since we cannot examine the MSE of the HPT estimator analytically, we execute the numerical evaluation to investigate the MSE performance of the HPT estimator, and compare the MSE performance of the HPT estimator with those of the FRR estimator and the usual OLS estimator. Furthermore, using the minimax regret criterion proposed by Sawa and Hiromatsu (1973 Sawa , T. , Hiromatsu , T. ( 1973 ). Minimax regret significance points for a preliminary test in regression analysis . Econometrica 41 : 1093 – 1101 .[Crossref], [Web of Science ®] , [Google Scholar]), we derive the optimal critical points of the preliminary F test. Our results show that the optimal significance points are greater than 19% and the optimal signicance points decrease as the denominator degrees of freedom of the preliminary F test statistic increases.     

Efficient Ratio and Product Estimators in Stratified Random Sampling

This paper suggests an efficient class of ratio and product estimators for estimating the population mean in stratified random sampling using auxiliary information. It is interesting to mention that, in addition to many, Koyuncu and Kadilar (2009 Koyuncu , N. , Kadilar , C. ( 2009 ). Ratio and product estimators in stratified random sampling . J. Statist. Plann. Infer. 139 : 2552 – 2558 .[Crossref], [Web of Science ®] , [Google Scholar]), Kadilar and Cingi (2003 Kadilar , C. , Cingi , H. ( 2003 ). Ratio estimator in stratified sampling . Biometr. J. 45 : 218 – 225 .[Crossref], [Web of Science ®] , [Google Scholar], 2005 Kadilar , C. , Cingi , H. ( 2005 ). A new estimator in stratified random sampling . Commun. Statist. Theor. Meth. 34 : 597 – 602 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), and Singh and Vishwakarma (2007 Singh , H. P. , Vishwakarma , G. K. ( 2007 ). Modified exponential ratio and product estimators for finite population mean in double sampling . Austr. J. Statist. 36 ( 3 ): 217 – 225 . [Google Scholar]) estimators are identified as members of the proposed class of estimators. The expressions of bias and mean square error (MSE) of the proposed estimators are derived under large sample approximation in general form. Asymptotically optimum estimator (AOE) in the class is identified alongwith its MSE formula. It has been shown that the proposed class of estimators is more efficient than combined regression estimator and Koyuncu and Kadilar (2009 Koyuncu , N. , Kadilar , C. ( 2009 ). Ratio and product estimators in stratified random sampling . J. Statist. Plann. Infer. 139 : 2552 – 2558 .[Crossref], [Web of Science ®] , [Google Scholar]) estimator. Moreover, theoretical findings are supported through a numerical example.     

Comment on Ridge Estimation to the Restricted Linear Model

In this note, we show that the estimator and the following results given by Zhong and Yang (2007 Zhong , Z. , Yang , H. ( 2007 ). Ridge estimation to the restricted linear model . Commun. Statist. Theor. Meth. 36 : 2099 – 2115 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) are the same with that of Groß (2003 Groß , J. ( 2003 ). Restricted ridge estimation . Statist. Probab. Lett. 65 : 57 – 64 .[Crossref], [Web of Science ®] , [Google Scholar]).     

Tuning Parameter Selection and Various Good Fitting Characteristics for the Liu-Type Estimator in Linear Regression

Liu (2003 Liu , K. ( 2003 ). Using Liu-Type estimator to combat collinearity . Commun. Statist. Theor. Meth. 32 ( 5 ): 1009 – 1020 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) proposed the Liu-Type estimator (LTE) to combat the well-known multicollinearity problem in linear regression. In this article, various better fitting characteristics of the LTE than those of the ordinary ridge regression estimator (Hoerl and Kennard, 1970 Hoerl , A. E. , Kennard , R. W. ( 1970 ). Ridge regression: Biased estimation for non-orthogonal problems . Technometrics 12 : 55 – 67 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) are considered. In particular, we derived two methods to determine the parameter d for the LTE and find that the ridge parameter k could serve for regularization of an ill-conditioned design matrix, while the other parameter d could be used for tuning the fit quality. In addition, the coefficients of regression, coefficient of multiple determination, residual error variance, and generalized cross validation (GCV) of the prediction quality are very stable, and as the ridge parameter increases they eventually reach asymptotic levels, which produces robust regression models. Furthermore, a Monte Carlo evaluation of these features is also given to illustrate some of the theoretical results.     

From Ordinary to Shrinkage Square-Root Estimators

Consider a skewed population. Suppose an intelligent guess could be made about an interval that contains the population mean. There may exist biased estimators with smaller mean squared error than the arithmetic mean within such an interval. This article indicates when it is advisable to shrink the arithmetic mean towards a guessed interval using root estimators. The goal is to obtain an estimator that is better near the average of natural origins. An estimator proposed. This estimator contains the Thompson (1968 Thompson , J. R. ( 1968 ). Accuracy borrowing in the estimation of the mean by shrinkage towards an interval . J. Amer. Statist. Assoc. 63 : 953 – 963 . [CSA] [CROSSREF] [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) ordinary shrinkage estimator, the Jenkins et al. (1973 Jenkins , O. C. , Ringer , L. J. , Hartley , H. O. ( 1973 ). Root estimators . J Amer. Statist. Assoc. 68 : 414 – 419 . [CSA] [CROSSREF] [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) square-root estimator, and the arithmetic sample mean as special cases. The bias and the mean squared error of the proposed more general estimator is compared with the three special cases. Shrinkage coefficients that yield minimum mean squared error estimators are obtained. The proposed estimator is considerably more efficient than the three special cases. This remains true for highly skewed populations. The merits of the proposed shrinkage square-root estimator are supported by the results of numerical and simulation studies.     

A New Estimator of Population Mean in Stratified Sampling

Kadilar and Cingi (2005 Kadilar , C. , Cingi , H. ( 2005 ). A new ratio estimator in stratified sampling . Comm. Statist. Theory Meth. 34 : 1 – 6 . [CSA] [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) have suggested a new ratio estimator in stratified sampling. The efficiency of this estimator is compared with the traditional combined ratio estimator on the basis of mean square error (MSE). We propose another estimator by utilizing a simple transformation introduced by Bedi (1996 Bedi , P. K. ( 1996 ). Efficient utilization of auxiliary information at estimation stage . Biomet. J. 38 ( 8 ): 973 – 976 . [CSA] [Crossref], [Web of Science ®] , [Google Scholar]). The proposed estimator is found to be more efficient than the traditional combined ratio estimator as well as the Kadilar and Cingi (2005 Kadilar , C. , Cingi , H. ( 2005 ). A new ratio estimator in stratified sampling . Comm. Statist. Theory Meth. 34 : 1 – 6 . [CSA] [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) ratio estimator.     

More on the two-parameter estimation in the restricted regression

ABSTRACT

In this paper, we introduce a new restricted two-parameter (RTP) estimator for the vector of parameters in a linear model when additional linear restrictions on the parameter vector are assumed to hold. We show that our new biased estimator is superior in the matrix mean square error criterion to the restricted ridge estimator proposed by Groß (2003 Groß, J. (2003). Restricted ridge estimation. Stat. Probab. Lett. 65:57–64.[Crossref], [Web of Science ®] , [Google Scholar]), restricted Liu estimator introduced by Kaçiranlar et al. (1999 Kaçiranlar, S., Sakall?oglus, S., Akdeniz, F., Styan, G.P.H., Werner, H.J. (1999). A new biased estimator in linear regression and a detailed analysis of the widely-analysed dataset on Portland cement. Sankhya Ser. B., Ind. J. Stat. 61:443–459. [Google Scholar]), and RTP estimator introduced by Özkale and Kaçiranlar (2007 Özkale, M., Kaçiranlar, S. (2007). The restricted and unrestricted two-parameter estimators. Commun. Stat. Theory Methods 36:2707–2725.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). A numerical example and a Monte Carlo simulation have been analyzed to illustrate some of the theoretical results.     

An Asymmetric Kernel Estimator of Density Function for Stationary Associated Sequences

Here, we apply the smoothing technique proposed by Chaubey et al. (2007 Chaubey , Y. P. , Sen , A. , Sen , P. K. ( 2007 ). A new smooth density estimator for non-negative random variables. Technical Report No. 1/07. Department of Mathematics and Statistics, Concordia University, Montreal, Canada . [Google Scholar]) for the empirical survival function studied in Bagai and Prakasa Rao (1991 Bagai , I. , Prakasa Rao , B. L. S. ( 1991 ). Estimation of the survival function for stationary associated processes . Statist. Probab. Lett. 12 : 385 – 391 .[Crossref], [Web of Science ®] , [Google Scholar]) for a sequence of stationary non-negative associated random variables.The derivative of this estimator in turn is used to propose a nonparametric density estimator. The asymptotic properties of the resulting estimators are studied and contrasted with some other competing estimators. A simulation study is carried out comparing the recent estimator based on the Poisson weights (Chaubey et al., 2011 Chaubey , Y. P. , Dewan , I. , Li , J. ( 2011 ). Smooth estimation of survival and density functions for a stationary associated process using poisson weights . Statist. Probab. Lett. 81 : 267 – 276 .[Crossref], [Web of Science ®] , [Google Scholar]) showing that the two estimators have comparable finite sample global as well as local behavior.     

On the Preliminary Test Backfitting and Speckman Estimators in Partially Linear Models and Numerical Comparisons

Przystalski and Krajewski (2007 Przystalski , M. , Krajewski , P. ( 2007 ). Constrained estimators of treatment parameters in semiparametric models . Statist. Probab. Lett. 77 : 914 – 919 .[Crossref], [Web of Science ®] , [Google Scholar]) proposed the restricted backfitting (RBCF) estimator and restricted Speckman (RSPC) estimator for the treatment effects in a partially linear model when some additional exact linear restrictions are assumed to hold. In this article, we introduce the preliminary test backfitting (PTBCF) estimator and preliminary test Speckman (PTSPC) estimator when the validity of the restrictions is suspected. Performances of the proposed estimators are examined with respect to the mean squared error (MSE) criterion. In addition, numerical behaviors of the proposed estimators are illustrated and compared via a Monte Carlo simulation study.     

Robust Bayesian inference via γ-divergence

Abstract

This paper presents the robust Bayesian inference based on the γ-divergence which is the same divergence as “type 0 divergence” in Jones et al. (2001 Jones, M. C., N. L. Hjort, I. R. Harris, and A. Basu. 2001. A comparison of related density-based minimum divergence estimators. Biometrika 88(3):865–73.[Crossref], [Web of Science ®] , [Google Scholar]) on the basis of Windham (1995 Windham, M. P. 1995. Robustifying model fitting. Journal of the Royal Statistical Society B57:599–609. [Google Scholar]). It is known that the minimum γ-divergence estimator works well to estimate the probability density for heavily contaminated data, and to estimate the variance parameters. In this paper, we propose a robust posterior distribution against outliers based on the γ-divergence and show the asymptotic properties of the proposed estimator. We also discuss some robustness properties of the proposed estimator and illustrate its performances in some simulation studies.     

The Product-Limit Estimate as an Inverse-Probability-Weighted Average

Abstract

For randomly censored data, (Satten, G. A., Datta S. (2001 Satten, G. A. and Datta, S. 2001. The Kaplan–Meier estimator as an inverse-probability-of-censoring weighted average. Amer. Statist. Ass., 55: 207–210. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). The Kaplan–Meier estimator as an inverse-probability-of-censoring weighted average. Amer. Statist. Ass. 55:207–210) showed that the Kaplan–Meier estimator (product-limit estimator (PLE)) can be expressed as an inverse-probability-weighted average. In this article, we consider the other two PLEs: the truncation PLE and the censoring-truncation PLE. For the data subject to left-truncation or both left-truncation and right-censoring, it is shown that these two PLEs can be expressed as inverse-probability-weighted averages.     

Estimators for the Population Mean in the Case of Missing Data

We propose a class of estimators for the population mean when there are missing data in the data set. Obtaining the mean square error equations of the proposed estimators, we show the conditions where the proposed estimators are more efficient than the sample mean, ratio-type estimators, and the estimators in Singh and Horn (2000 Singh , S. , Horn , S. ( 2000 ). Compromised imputation in survey sampling . Metrika 51 : 267 – 276 .[Crossref], [Web of Science ®] , [Google Scholar]) and Singh and Deo (2003 Singh , S. , Deo , B. (2003). Imputation by power transformation. Statist. Pap. 44:555–579.[Crossref], [Web of Science ®] , [Google Scholar]) in the case of missing data. These conditions are also supported by a numerical example.     

Relative Errors of Difference-Based Variance Estimators in Nonparametric Regression

Difference-based estimators for the error variance are popular since they do not require the estimation of the mean function. Unlike most existing difference-based estimators, new estimators proposed by Müller et al. (2003 Müller , U. , Schick , A. , Wefelmeyer , W. ( 2003 ). Estimating the error variance in nonparametric regression by a covariate-matched U-statistic . Statistics 37 : 179 – 188 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and Tong and Wang (2005 Tong , T. , Wang , Y. ( 2005 ). Estimating residual variance in nonparametric regression using least squares . Biometrika 92 : 821 – 830 .[Crossref], [Web of Science ®] , [Google Scholar]) achieved the asymptotic optimal rate as residual-based estimators. In this article, we study the relative errors of these difference-based estimators which lead to better understanding of the differences between them and residual-based estimators. To compute the relative error of the covariate-matched U-statistic estimator proposed by Müller et al. (2003 Müller , U. , Schick , A. , Wefelmeyer , W. ( 2003 ). Estimating the error variance in nonparametric regression by a covariate-matched U-statistic . Statistics 37 : 179 – 188 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), we develop a modified version by using simpler weights. We further investigate its asymptotic property for both equidistant and random designs and show that our modified estimator is asymptotically efficient.     

Statistical Analysis of the Log-Normal Distribution under Type-II Progressive Hybrid Censoring Schemes

In this article, we obtain the maximum likelihood estimators (MLEs) and approximate maximum likelihood estimators (AMLEs) of the parameters, from a two-parameter log-normal distribution based on the adaptive Type-II progressive hybrid censoring scheme, which was introduced by Ng et al. (2009 Ng , H. K. T. , Kundu , D. , Chan , P. S. ( 2009 ). Statistical analysis of exponential lifetimes under an adaptive Type-II progressively censoring scheme . Naval Research Logistics 56 : 687 – 698 .[Crossref], [Web of Science ®] , [Google Scholar]) for life testing or reliability experiment. In order to compare the results, we calculate corresponding estimators of the Type-II progressive hybrid censoring scheme. In particular, we provide computational formulas of the expected total test time and the expected number of failures for each scheme. We also compute the observed Fisher information matrix and use them to obtain the asymptotic confidence intervals. A simulation study carries out to evaluate the bias and mean square error of the MLEs and AMLEs from the two above-mentioned schemes. Finally, we present a numerical example to illustrate the methods of inference discussed here.     

Modified Ridge Parameters for Seemingly Unrelated Regression Model

In this article, we modify a number of new biased estimators of seemingly unrelated regression (SUR) parameters which are developed by Alkhamisi and Shukur (2008 Alkhamisi , M. A. , Shukur , G. ( 2008 ). Developing ridge parameters for SUR model . Commun. Statist. Theor. Meth. 37 ( 4 ): 544 – 564 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), AS, when the explanatory variables are affected by multicollinearity. Nine estimators of the ridge parameters have been modified and compared in terms of the trace mean squared error (TMSE) and (PR) criterion. The results from this extended study are the also compared with those founded by AS. A simulation study has been conducted to compare the performance of the modified estimators of the ridge parameters. The results showed that under certain conditions the performance of the multivariate ridge regression estimators based on SUR ridge R _MSmax is superior to other estimators in terms of TMSE and PR criterion. In large samples and when the collinearity between the explanatory variables is not high, the unbiased SUR, estimator produces a smaller TMSEs.     

Estimation of the Mean of a Sensitive Variable in the Presence of Auxiliary Information

Sousa et al. (2010 Sousa , R. , Shabbir , J. , Real , P. C. , Gupta , S. ( 2010 ). Ratio estimation of the mean of a sensitive variable in the presence of auxiliary information . J. Statist. Theor. Prac. 4 ( 3 ): 495 – 507 .[Taylor & Francis Online] , [Google Scholar]) introduced a ratio estimator for the mean of a sensitive variable and showed that this estimator performs better than the ordinary mean estimator based on a randomized response technique (RRT). In this article, we introduce a regression estimator that performs better than the ratio estimator even for modest correlation between the primary and the auxiliary variables. The underlying assumption is that the primary variable is sensitive in nature but a non sensitive auxiliary variable exists that is positively correlated with the primary variable. Expressions for the Bias and MSE (Mean Square Error) are derived based on the first order of approximation. It is shown that the proposed regression estimator performs better than the ratio estimator and the ordinary RRT mean estimator (that does not utilize the auxiliary information). We also consider a generalized regression-cum-ratio estimator that has even smaller MSE. An extensive simulation study is presented to evaluate the performances of the proposed estimators in relation to other estimators in the study. The procedure is also applied to some financial data: purchase orders (a sensitive variable) and gross turnover (a non sensitive variable) in 2009 for a population of 5,336 companies in Portugal from a survey on Information and Communication Technologies (ICT) usage.