Efficient use of higher-lag autocorrelations for estimating autoregressive processes

The Yule–Walker estimator is commonly used in time-series analysis, as a simple way to estimate the coefficients of an autoregressive process. Under strong assumptions on the noise process, this estimator possesses the same asymptotic properties as the Gaussian maximum likelihood estimator. However, when the noise is a weak one, other estimators based on higher-order empirical autocorrelations can provide substantial efficiency gains. This is illustrated by means of a first-order autoregressive process with a Markov-switching white noise. We show how to optimally choose a linear combination of a set of estimators based on empirical autocorrelations. The asymptotic variance of the optimal estimator is derived. Empirical experiments based on simulations show that the new estimator performs well on the illustrative model.     

Effects of outliers on the identification and estimation of GARCH models

Abstract. This paper analyses how outliers affect the identification of conditional heteroscedasticity and the estimation of generalized autoregressive conditionally heteroscedastic (GARCH) models. First, we derive the asymptotic biases of the sample autocorrelations of squared observations generated by stationary processes and show that the properties of some conditional homoscedasticity tests can be distorted. Second, we obtain the asymptotic and finite sample biases of the ordinary least squares (OLS) estimator of ARCH(p) models. The finite sample results are extended to generalized least squares (GLS), maximum likelihood (ML) and quasi‐maximum likelihood (QML) estimators of ARCH(p) and GARCH(1,1) models. Finally, we show that the estimated asymptotic standard deviations are biased estimates of the sample standard deviations.     

Subsampling inference for the autocovariances and autocorrelations of long‐memory heavy‐ tailed linear time series

We provide a self‐normalization for the sample autocovariances and autocorrelations of a linear, long‐memory time series with innovations that have either finite fourth moment or are heavy‐tailed with tail index 2 < α < 4. In the asymptotic distribution of the sample autocovariance there are three rates of convergence that depend on the interplay between the memory parameter d and α, and which consequently lead to three different limit distributions; for the sample autocorrelation the limit distribution only depends on d. We introduce a self‐normalized sample autocovariance statistic, which is computable without knowledge of α or d (or their relationship), and which converges to a non‐degenerate distribution. We also treat self‐normalization of the autocorrelations. The sampling distributions can then be approximated non‐parametrically by subsampling, as the corresponding asymptotic distribution is still parameter‐dependent. The subsampling‐based confidence intervals for the process autocovariances and autocorrelations are shown to have satisfactory empirical coverage rates in a simulation study. The impact of subsampling block size on the coverage is assessed. The methodology is further applied to the log‐squared returns of Merck stock.     

Multivariate Portmanteau Test For Autoregressive Models with Uncorrelated but Nonindependent Errors

Abstract. We study the asymptotic behaviour of the least squares estimator, of the residual autocorrelations and of the Ljung–Box (or Box–Pierce) portmanteau test statistic for multiple autoregressive time series models with nonindependent innovations. Under mild assumptions, it is shown that the asymptotic distribution of the portmanteau tests is that of a weighted sum of independent chi‐squared random variables. When the innovations exhibit conditional heteroscedasticity or other forms of dependence, this asymptotic distribution can be quite different from that of models with independent and identically distributed innovations. Consequently, the usual chi‐squared distribution does not provide an adequate approximation to the distribution of the Box–Pierce goodness‐of‐fit portmanteau test in the presence of nonindependent innovations. Hence we propose a method to adjust the critical values of the portmanteau tests. Monte carlo experiments illustrate the finite sample performance of the modified portmanteau test.     

ON WEIGHTED PORTMANTEAU TESTS FOR TIME‐SERIES GOODNESS‐OF‐FIT

Recent work in the literature has shown weighted variants of the classic portmanteau test for time series can be more powerful in many situations. In this article, we study the asymptotic distribution of weighted sums of the squared residual autocorrelations where both the sample size n and maximum lag of the statistic m grow large. Several weighting schemes are introduced, including a data‐adaptive statistic in which the weights are determined by a function of the sample partial autocorrelations. These statistics can provide more power than other portmanteau tests found in the literature and are much less sensitive to the choice of the maximum correlation lag. The efficacy of the proposed methods is further demonstrated through an analysis of Australian red wine sales.     

ON THE SQUARED RESIDUAL AUTOCORRELATIONS IN NON-LINEAR TIME SERIES WITH CONDITIONAL HETEROSKEDASTICITY

Abstract. Time series with a changing conditional variance have been found useful in many applications. Residual autocorrelations from traditional autoregressive moving-average models have been found useful in model diagnostic checking. By analogy, squared residual autocorrelations from fitted conditional heteroskedastic time series models would be useful in checking the adequacy of such models. In this paper, a general class of squared residual autocorrelations is defined and their asymptotic distribution is obtained. The result leads to some useful diagnostic tools for statisticians using conditional heteroskedastic time series models. Some simulation results and an illustrative example are also reported.     

OPTIMALITY OF THE MAXIMUM LIKELIHOOD ESTIMATOR IN FIRST-ORDER AUTOREGRESSIVE PROCESSES

Abstract. Let ρρ* be the maximum likelihood estimator (MLE) of the parameter ρ in the first-order autoregressive process with normal errors. The problem of optimality in the sense of weighted squared error is considered rather than moments of asymptotic distributions. Many unbiased estimators can be constructed, but the two-dimensional sufficient statistic is incomplete. It is shown that dEρ* - ρ→ 0 uniformly in ρ, and that dE (ρ*)/ d ρ→ 1 for all |ρ| > 1. For |ρ| > 1, it is known that the asymptotic distribution of {I(ρ)}¹²⁽ρ* - ρ), where I (ρ) is the Fisher information, is Cauchy. It follows that the Cramèr-Rao inequality will not yield useful results for investigating the limit of exact efficiency of any asymptotically unbiased estimator, including the MLE. For all |ρ| < 1, ρ* is asymptotically optimal in the sense of minimizing expected weighted squared error. In addition, for |ρ| < 1, ρ* minimizes the variance asymptotically.     

A decision theoretic approach to change point estimation for binomial CUSUM control charts

Detecting when the process has changed is a classical problem in sequential analysis and is an important practical issue in statistical process control. This article is concerned about the binomial cumulative sum (CUSUM) control chart, which is extensively applied to industrial process control, health care, public health surveillance, and other fields. For the binomial CUSUM, a maximum likelihood estimator has been proposed to estimate the change point. In our article, following a decision theoretic approach, we develop a new estimator that aims to improve the existing methods. For interval estimation, we propose a parametric bootstrap procedure to construct the confidence set of the change point. We compare our proposed method with the maximum likelihood estimator and Page's last zero estimator in terms of mean squared error by simulations. We find that the proposed method gives more unbiased and robust results than the existing procedures under various parameter designs. We analyze jewelry manufacturing data for illustration.     

EFFICIENT METHOD OF MOMENTS ESTIMATORS FOR INTEGER TIME SERIES MODELS

The parameters of integer autoregressive models with Poisson, or negative binomial innovations can be estimated by maximum likelihood where the prediction error decomposition, together with convolution methods, is used to write down the likelihood function. When a moving average component is introduced this is not the case. To address this problem an efficient method of moment estimator is proposed where the estimated standard errors for the parameters are obtained using subsampling methods. The small sample properties of the estimator are investigated using Monte Carlo methods, while the approach is demonstrated using two well‐known examples from the time series literature.     

Improved sequential and accelerated sequential Procedures for estimating the scale parameter in a aniform distribution

Ghosh and Mukhopadhyay (1975a) and more recently Mukhopadhyay et al.(1983) have consideredsequential minimum risk point estimation of θ in a uniform (0,θ)population.The loss function has been squared error plus linear cost and θ has beencustomarily estimated by the sample maximum.In this paper,we consider instead the best scalarmultiple of the sample aximum as the estimator of θ.The percentage saving in the fictitious optimal fixed sample size as well as the percentage reduction in the corresponding minimum risk ar seen to be about 20.6X when compared with the Mukhopadhyay et al.(1983) cedure.Since thisamount of saving is quite substantial,we set out to develop     

The mean squared error of Geweke and Porter-Hudak's estimator of the memory parameter of a long-memory time series

We establish some asymptotic properties of a log-periodogram regression estimator for the memory parameter of a long-memory time series. We consider the estimator originally proposed by Geweke and Porter-Hudak (The estimation and application of long memory time series models. Journal of Time Ser. Anal. 4 (1983), 221–37). In particular, we do not omit any of the low frequency periodogram ordinates from the regression. We derive expressions for the estimator's asymptotic bias, variance and mean squared error as functions of the number of periodogram ordinates, m , used in the regression. Consistency of the estimator is obtained as long as m ←∞ and n ←∞ with ( m log m )/ n ← 0, where n is the sample size. Under these and the additional conditions assumed in this paper, the optimal m , minimizing the mean squared error, is of order O( n ^4/5). We also establish the asymptotic normality of the estimator. In a simulation study, we assess the accuracy of our asymptotic theory on mean squared error for finite sample sizes. One finding is that the choice m = n ^1/2, originally suggested by Geweke and Porter-Hudak (1983), can lead to performance which is markedly inferior to that of the optimal choice, even in reasonably small samples.     

On the limit theory of the Gaussian SQMLE in the EGARCH(1,1) model

We derive the limit theory of the Gaussian stable quasi maximum likelihood estimator for the stationary EGARCH(1,1) model when the squared innovation process has marginals with regularly varying tails. We derive regularly varying rates and limiting stable distributions. We perform Monte Carlo experiments to assess the extent of the parameter space corresponding to the invertibility condition, and the quality of the asymptotic approximation.     

Sequential point estimation for branching processes i,subcritical case

For a subcritical branching process with observable immigration, we consider the problem of finding a sample size for which the mean squared error of a natural estimator of the offspring mean is less than a prescribed value u. If the parameters of the process are known, then a minimum sample size achieving the above objective (approximately) exists. When the parameters are unknown, we propose a stopping rule and show that the resulting sequential procedure is asymptotically risk efficient as u approaches zero. Furthermore, we establish the asymptotic efficiency and normality of our stopping rule.     

Asymptotic Relative Efficiency of Goodness‐Of‐Fit Tests Based on Inverse and Ordinary Autocorrelations

Abstract. We compare the performance of the inverse and ordinary (partial) autocorrelations for time series model identification. It is found that, both in terms of Bahadur's slope and Pitman's asymptotic relative efficiency, the inverse partial autocorrelations are more efficient than the ordinary autocorrelations for identification of moving‐average models. By duality, the partial autocorrelations turn out to be more powerful than the inverse autocorrelations to identify autoregressive models. Numerical experiments on both simulated and real data sets are presented to highlight the theoretical results.     

Estimation of the long‐memory stochastic volatility model parameters that is robust to level shifts and deterministic trends

I provide conditions under which the trimmed FDQML estimator, advanced by McCloskey (2010) in the context of fully parametric short‐memory models, can be used to estimate the long‐memory stochastic volatility model parameters in the presence of additive low‐frequency contamination in log‐squared returns. The types of low‐frequency contamination covered include level shifts as well as deterministic trends. I establish consistency and asymptotic normality in the presence or absence of such low‐frequency contamination under certain conditions on the growth rate of the trimming parameter. I also provide theoretical guidance on the choice of trimming parameter by heuristically obtaining its asymptotic MSE‐optimal rate under certain types of low‐frequency contamination. A simulation study examines the finite sample properties of the robust estimator, showing substantial gains from its use in the presence of level shifts. The finite sample analysis also explores how different levels of trimming affect the parameter estimates in the presence and absence of low‐frequency contamination and long‐memory.     

Minimum α‐divergence estimation for arch models

Abstract. This paper considers a minimum α‐divergence estimation for a class of ARCH(p) models. For these models with unknown volatility parameters, the exact form of the innovation density is supposed to be unknown in detail but is thought to be close to members of some parametric family. To approximate such a density, we first construct an estimator for the unknown volatility parameters using the conditional least squares estimator given by Tjøstheim [Stochastic processes and their applications (1986) Vol. 21, pp. 251–273]. Then, a nonparametric kernel density estimator is constructed for the innovation density based on the estimated residuals. Using techniques of the minimum Hellinger distance estimation for stochastic models and residual empirical process from an ARCH(p) model given by Beran [Annals of Statistics (1977) Vol. 5, pp. 445–463] and Lee and Taniguchi [Statistica Sinica (2005) Vol. 15, pp. 215–234] respectively, it is shown that the proposed estimator is consistent and asymptotically normal. Moreover, a robustness measure for the score of the estimator is introduced. The asymptotic efficiency and robustness of the estimator are illustrated by simulations. The proposed estimator is also applied to daily stock returns of Dell Corporation.     

Least tail‐trimmed squares for infinite variance autoregressions

We develop a robust least squares estimator for autoregressions with possibly heavy tailed errors. Robustness to heavy tails is ensured by negligibly trimming the squared error according to extreme values of the error and regressors. Tail‐trimming ensures asymptotic normality and super‐‐convergence with a rate comparable to the highest achieved amongst M‐estimators for stationary data. Moreover, tail‐trimming ensures robustness to heavy tails in both small and large samples. By comparison, existing robust estimators are not as robust in small samples, have a slower rate of convergence when the variance is infinite, or are not asymptotically normal. We present a consistent estimator of the covariance matrix and treat classic inference without knowledge of the rate of convergence. A simulation study demonstrates the sharpness and approximate normality of the estimator, and we apply the estimator to financial returns data. Finally, tail‐trimming can be easily extended beyond least squares estimation for a linear stationary AR model. We discuss extensions to quasi‐maximum likelihood for GARCH, weighted least squares for a possibly non‐stationary random coefficient autoregression, and empirical likelihood for robust confidence region estimation, in each case for models with possibly heavy tailed errors.     

A Nonparametric Prewhitened Covariance Estimator

This paper proposes a new nonparametric spectral density estimator for time series models with general autocorrelation. The conventional nonparametric estimator that uses a positive kernel has mean squared error no better than n^?4/5. We show that the best implementation of our estimator has mean squared error of order n^?8/9, provided there is sufficient smoothness present in the spectral density. This is, of course, achieved by bias reduction; however, unlike most other bias reduction methods, like the kernel method with higher‐order kernels, our procedure ensures a positive definite estimate. Our method is a generalization of the well‐known prewhitening method of spectral estimation; we argue that this can best be interpreted as multiplicative bias reduction. Higher‐order expansions for the proposed estimator are derived, providing an improved bandwidth choice that minimizes the mean squared error to the second order. A simulation study shows that the recommended prewhitened kernel estimator reduces bias and mean squared error in spectral density estimation.     

ARMA MODELLING WITH NON-GAUSSIAN INNOVATIONS

Abstract. The problem of modelling time series driven by non-Gaussian innovations is considered. The asymptotic normality of the maximum likelihood estimator is established under some general conditions. The distribution of the residual autocorrelations is also obtained. This gives rise to a potentially useful goodness-of-fit statistic. Applications of the results to two important cases are discussed. Two real examples are considered.     

Efficient estimation and inference in cointegrating regressions with structural change

Abstract. This paper investigates an efficient estimation method for a cointegrating regression model with structural change. Our proposal is that we first estimate the break point by minimizing the sum of squared residuals and then, by replacing the break fraction with the estimated one, we estimate the regression model by the canonical cointegrating regression (CCR) method proposed by Park [ Econometrica (1992 ) Vol. 60, pp. 119–143]. We show that the estimator of the break fraction has the same convergence rate as obtained in Bai, Lumsdaine and Stock [ Review of Economic Studies (1998 ) Vol. 65, pp. 395–432] and that the CCR estimator with the estimated break fraction has the same asymptotic property as the estimator with the known break point. However, we also show that our method breaks down when the magnitude of structural change is very small. Simulation experiments reveal how the finite sample distribution approaches the limiting distribution as the magnitude of the break and or the sample size increases.