Recursive Prediction and Likelihood Evaluation for Periodic ARMA Models

This paper explores recursive prediction and likelihood evaluation techniques for periodic autoregressive moving-average (PARMA) time series models. The innovations algorithm is used to develop a simple recursive scheme for computing one-step-ahead predictors and their mean squared errors. The asymptotic form of this recursion is explored. The prediction results are then used to develop an efficient (and exact) PARMA likelihood evaluation algorithm for Gaussian series. We then show how a multivariate autoregressive moving average (ARMA) likelihood can be evaluated by writing the multivariate ARMA model in PARMA form. Explicit calculations for PARMA(1, 1) models and periodic autoregressions are included.     

Some Results on Specification Search and Pre-testing in an ARMA(1,1) Process

In this study we consider simple autoregressive moving-average (ARMA) models of order at most 1. Pre-testing, on the moving-average coefficient θ, is used to choose between an ARMA(1,1) and an AR(1) in a Monte Carlo design. We find that the pre-test estimator is not always dominated by the others, and that the bias and the mean square error of the estimate of the autoregressive coefficient φ very often depend on the sign of the autoregressive and moving-average parameters of the ARMA(1,1) model in the data-generating process. Further, we note that the degrees of size and power distortion of the t test on φ, after pre-testing for θ, are generally associated with model misspecification.     

SUBSET THRESHOLD AUTOREGRESSION WITH APPLICATIONS

Abstract. In this paper we define subset threshold autoregressive models and suggest a simple algorithm for fitting them. The suggested algorithm is applied to simulated as well as real data. Two well-known time series are studied:the Canadian lynx data and Wolf's sunspot numbers. The fitted models are compared with the threshold models of Tong and Lim and with the subset bilinear models of Gabr and Subba Rao. Some statistical properties of these models are also studied. An examination of forecasting performance is included.     

ESTIMATION OF AUTOREGRESSIVE MOVING-AVERAGE MODELS VIA HIGH-ORDER AUTOREGRESSIVE APPROXIMATIONS

Abstract. In this paper the problem of estimating autoregressive moving-average (ARMA) models is dealt with by first estimating a high-order autoregressive (AR) approximation and then using the AR estimate to form the ARMA estimate. We show how to obtain an efficient ARMA estimate by allowing the order of the AR estimate to tend to infinity as the number of observations tends to infinity. This approach is closely related to the work of Durbin. By transforming the approach into the frequency domain, we can view it as an L ²-norm model approximation of the relative error of the spectral factors. It can also be seen as replacing the periodogram estimate in the Whittle approach by a high-order AR spectral density estimate. Since L ²-norm approximation is a difficult task, we replace it by a modification of a recent model approximation technique called balanced model reduction. By an example, we show that this technique gives almost efficient ARMA estimates without the use of numerical optimization routines.     

PERIODIC CORRELATION IN STRATOSPHERIC OZONE DATA

Abstract. A 50-year time series of monthly stratospheric ozone readings from Arosa, Switzerland, is analyzed. The time series exhibits the properties of a periodically correlated (PC) random sequence with annual periodicities. Spectral properties of PC random sequences are reviewed and a test to detect periodic correlation is presented. An autoregressive moving-average (ARMA) model with periodically varying coefficients (PARMA) is fitted to the data in two stages. First, a periodic autoregressive model is fitted to the data. This fit yields residuals that are stationary but non-white. Next, a stationary ARMA model is fitted to the residuals and the two models are combined to produce a larger model for the data. The combined model is shown to be a PARMA model and yields residuals that have the correlation properties of white noise.     

Testing for a Unit Root in Autoregressive Moving-average Models with Missing Data

Testing for a single autoregressive unit root in an autoregressive moving-average (ARMA) model is considered in the case when data contain missing values. The proposed test statistics are based on an ordinary least squares type estimator of the unit root parameter which is a simple approximation of the one-step Newton–Raphson estimator. The limiting distributions of the test statistics are the same as those of the regression statistics in AR(1) models tabulated by Dickey and Fuller (Distribution of the estimators for autoregressive time series with a unit root. J. Am. Stat. Assoc . 74 (1979), 427–31) for the complete data situation. The tests accommodate models with a fitted intercept and a fitted time trend.     

ON THE SELECTION OF SUBSET AUTOREGRESSIVE TIME SERIES MODELS

Abstract. The estimation of subset autoregressive time series models has been a difficult problem because of the large number of possible alternative models involved. However, with the advent of model selection criteria based on the maximum likelihood, subset model fitting has become feasible. Using an efficient technique for evaluating the residual variance of all possible subset models, a method is proposed for the fitting of subset autoregressive models. The application of the method is illustrated by means of real and simulated data.     

ASYMPTOTIC INFERENCE FOR NON-INVERTIBLE MOVING-AVERAGE TIME SERIES

Abstract. This paper is concerned with statistical inference of nonstationary and non-invertible autoregressive moving-average (ARMA) processes. It makes use of the fact that a derived process of an ARMA( p, q ) model follows an AR( q ) model with an autoregressive (AR) operator equivalent to the moving-average (MA) part of the original ARMA model. Asymptotic distributions of least squares estimates of MA parameters based on a constructed derived process are obtained as corresponding analogs of a nonstationary AR process. Extensions to the nearly non-invertible models are considered and the limiting distributions are obtained as functionals of stochastic integrals of Brownian motions and Ornstein-Uhlenbeck processes. For application, a two-stage procedure is proposed for testing unit roots in the MA polynomial. Examples are given to illustrate the application.     

Multi-variate t Autoregressions: Innovations, Prediction Variances and Exact Likelihood Equations

Abstract.  The multi-variate t distribution provides a viable framework for modelling volatile time-series data; it includes the multi-variate Cauchy and normal distributions as special cases. For multi-variate t autoregressive models, we study the nature of the innovation distribution and the prediction error variance; the latter is nonconstant and satisfies a kind of generalized autoregressive conditionally heteroscedastic model. We derive the exact likelihood equations for the model parameters, they are related to the Yule–Walker equations and involve simple functions of the data, the model parameters and the autocovariances up to the order of the model. The maximum likelihood estimators are obtained by alternately solving two linear systems and illustrated using the lynx data. The simplicity of these equations contributes greatly to our theoretical understanding of the likelihood function and the ensuing estimators. Their range of applications are not limited to the parameters of autoregressive models; in fact, they are applicable to the parameters of ARMA models and covariance matrices of stochastic processes whose finite-dimensional distributions are multi-variate t .     

DIFFERENTIAL GEOMETRY OF ARMA MODELS

Abstract. A general approach for the development of a statistical inference on autoregressive moving-average (ARMA) models is presented based on geometric arguments. ARMA models are characterized as members of the curved exponential family. Geometric properties of ARMA models are computed and used to suggest parameter transformations that satisfy predetermined properties. In particular, the effect on the asymptotic bias of the maximum likelihood estimator of model parameters is illustrated. Hypothesis testing of parameters is discussed through the application of a modified form of the likelihood ratio test statistic.     

ESTIMATION OF AUTOREGRESSIVE MOVING-AVERAGE ORDER GIVEN AN INFINITE NUMBER OF MODELS AND APPROXIMATION OF SPECTRAL DENSITIES

Abstract. A modification of the minimum Akaike information criterion (AIC) procedure (and of related procedures like the Bayesian information criterion (BIC)) for order estimation in autoregressive moving-average (ARMA) models is introduced. This procedure has the advantage that consistency for the order estimators obtained via this procedure can be established without restricting attention to only a finite number of models. The behaviour of these newly introduced order estimators is also analysed for the case when the data-generating process is not an ARMA process (transfer function/spectral density approximation). Furthermore, the behaviour of the order estimators obtained via minimization of BIC (or of related criteria) is investigated for a non-ARMA data-generating process.     

AN APPLICATION OF THE SCHUR-COHN ALGORITHM TO TIME SERIES ANALYSIS

Abstract. Standard least squares analysis of autoregressive moving-average (ARMA) processes with errors-in-variables entails the construction of a new set of parameters which are functions of the original ARMA parameters, and requires that derivatives of these new parameters of order three or less with respect to the ARMA parameters exist and be bounded. The boundedness of these derivatives in turn depends critically on the nonsingularity of a matrix B which is a function of the ARMA parameters via the new parameters in the model. A particular version of the classical Schur–Cohn algorithm enables us to establish this nonsingularity.     

ON THE UNBIASEDNESS PROPERTY OF AIC FOR EXACT OR APPROXIMATING LINEAR STOCHASTIC TIME SERIES MODELS

Abstract. A rigorous analysis is given of the asymptotic bias of the log maximum likelihood as an estimate of the expected log likelihood of the maximum likelihood model, when a linear model, such as an invertible, gaussian ARMA ( p, q ) model, with or without parameter constraints, is fit to stationary, possibly non-gaussian observations. It is assumed that these data arise from a model whose spectral density function either (i) coincides with that of a member of the class of models being fit, or, that failing, (ii) can be well-approximated by invertible ARMA ( p, q ) model spectral density functions in the class, whose ARMA coefficients are parameterized separately from the innovations variance. Our analysis shows that, for the purpose of comparing maximum likelihood models from different model classes, Akaike's AIC is asymptotically unbiased, in case (i), under gaussian or separate parametrization assumptions, but is not necessarily unbiased otherwise. In case (ii), its asymptotic bias is shown to be of the order of a number less than unity raised to the power max { p, q } and so is negligible if max { p, q } is not too small. These results extend and complete the somewhat heuristic analysis given by Ogata (1980) for exact or approximating autoregressive models.     

Nonlinear ARMA models with functional MA coefficients

Abstract. In the present article, we propose and study a new class of nonlinear autoregressive moving‐average (ARMA) models, in which each moving‐average (MA) coefficient is enlarged to an arbitrary univariate function. We first provide a sufficient condition for the existence of the stationary solution and further discuss the moment structure. We investigate the estimation method to the proposed models. The global estimates of parameters and local linear estimates of functional coefficients are obtained by using a back‐fitting algorithm. For testing whether the functional coefficients are some specified parametric forms, a bootstrap test approach is provided. The proposed models are illustrated by both simulated and real data examples.     

An Extended Portmanteau Test for VARMA Models With Mixing Nonlinear Constraints

Abstract. The portmanteau test is a widely used diagnostic tool for univariate and multivariate time‐series models. Its asymptotic distribution is known for the unconstrained vector autoregressive moving‐average (VARMA) case and for VAR models with constraints on the autoregressive coefficients. In this article, we give conditions under which the test can be applied to constrained VARMA models. Unfortunately, it cannot generally be applied to models with constraints that simultaneously affect the ARMA polynomial coefficients and the covariance matrix of the innovations (mixing constraints). This happens in latent‐variable models such as dynamic factor models (DFM). In addition, when there are constraints on the covariance matrix it seems convenient to check the goodness of fit using the zero‐lag residual covariances. We propose an extended portmanteau test that not only checks the autocorrelations of the residuals but also whether their covariance matrix is consistent with the constraints. We prove that the statistic is asymptotically distributed as a chi‐square for ARMA models under the assumption that the innovations have Gaussian‐like fourth‐order moments. We also show that the test is appropriate for the DFM, Peña–Box model and factor‐structural vector autoregression (FSVAR).     

On Parameter Estimation for Exponential Dispersion Arma Models

Abstract. A class of autoregressive moving‐average (ARMA) models proposed by Jørgensen and Song [Journal of Applied Probability (1998), vol. 35, pp. 78–92] with exponential dispersion model margins are useful to deal with non‐normal stationary time series with high‐order autocorrelation. One property associated with the class of models is that the projection process takes the exact form of the classical Box and Jenkins ARMA representation, leading to considerable ease to establish theories. This paper focuses on the issue of parameter estimation for such models, which has not been thoroughly investigated in Jørgensen and Song's paper. The key of the proposed approach is to treat the residual process associated with the projection essentially as a measurement error, which enables us to formulate directly an ARMA representation for the observed time series. The parameter estimation therefore becomes straightforward using the existing methods for the Box and Jenkins ARMA models such as the quasi‐likelihood method. The approach is illustrated by simulation studies and by an analysis of myoclonic seizure counts.     

RECURSIVE COMPUTATION OF THE PARAMETERS OF PERIODIC AUTOREGRESSIVE MOVING-AVERAGE PROCESSES

Abstract. An algorithm for recursive computation of the parameters of periodic autoregressive moving-average (ARMA) processes is given. It also provides recursions for stationary multivariate ARMA processes. A procedure for simultaneous estimation of the order and the parameters of a periodic ARMA process is outlined.     

RESULTS ON ESTIMATION AND TESTING FOR A UNIT ROOT IN THE NONSTATIONARY AUTOREGRESSIVE MOVING-AVERAGE MODEL

Abstract. We review the limiting distribution theory for Gaussian estimation of the univariate autoregressive moving-average (ARMA) model in the presence of a unit root in the autoregressive (AR) operator, and present the asymptotic distribution of the associated likelihood ratio (LR) test statistic for testing for a unit root in the ARMA model. The finite sample properties of the LR statistic as well as other unit root test procedures for the ARMA model are examined through a limited simulation study. We conclude that, for practical empirical work that relies on standard computations, the LR test procedure generally performs better than other standard procedures in the presence of a substantial moving-average component in the ARMA model.     

ON THE ASYMPTOTIC DISTRIBUTION OF THE GENERALIZED PARTIAL AUTOCORRELATION FUNCTION IN AUTOREGRESSIVE MOVING-AVERAGE PROCESSES

Abstract. It has been conjectured and illustrated that the estimate of the generalized partial autocorrelation function (GPAC), which has been used for the identification of autoregressive moving-average (ARMA) models, has a thick-tailed asymptotic distribution. The purpose of this paper is to investigate the asymptotic behaviour of the GPAC in detail. It will be shown that the GPAC can be represented as a ratio of two functions, known as the θ function and the Λ function, each of which itself has a useful pattern for ARMA model identification. We shall show the consistencies of the extended Yule-Walker estimates of the three functions and present their asymptotic distributions.     

A Note on Non‐Negative Arma Processes

Abstract. Recently, there has been much research on developing models suitable for analysing the volatility of a discrete‐time process. Since the volatility process, like many others, is necessarily non‐negative, there is a need to construct models for stationary processes which are non‐negative with probability one. Such models can be obtained by driving autoregressive moving average (ARMA) processes with non‐negative kernel by non‐negative white noise. This raises the problem of finding simple conditions under which an ARMA process with given coefficients has a non‐negative kernel. In this article, we derive a necessary and sufficient condition. This condition is in terms of the generating function of the ARMA kernel which has a simple form. Moreover, we derive some readily verifiable necessary and sufficient conditions for some ARMA processes to be non‐negative almost surely.