EFFICIENT NON‐PARAMETRIC ESTIMATION OF THE SPECTRAL DENSITY IN THE PRESENCE OF MISSING OBSERVATIONS

The problem of non‐parametric spectral density estimation for discrete‐time series in the presence of missing observations has a long history. In particular, the first consistent estimators of the spectral density have been developed at about the same time as consistent estimators for non‐parametric regression. On the other hand, while for now, the theory of efficient (under the minimax mean integrated squared error criteria) and adaptive nonparametric regression estimation with missing data is well developed, no similar results have been proposed for the spectral density of a time series whose observations are missed according to an unknown stochastic process. This article develops the theory of efficient and adaptive estimation for a class of spectral densities that includes classical causal autoregressive moving‐average time series. The developed theory shows how a missing mechanism affects the estimation and what penalty it imposes on the risk convergence. In particular, given costs of a single observation in time series with and without missing data and a desired accuracy of estimation, the theory allows one to choose the cost‐effective time series. A numerical study confirms the asymptotic theory.     

Sequential estimation of the autoregressive parameter in a first order autoregressive process

This paper considers the problem of sequential point estimation of the autoregressive parameter in a first order autoregressive model. The sequential estimator proposed here is based on the least squares estimator and is shown to be asymtotically risk efficient as the cost of estimation error tends to infinity, under certain regularity conditions. Furthermore, nonlinear renewal theory is used to obtain a second order approximation to the expected stopping time. The asymptotic normality and uniform integrability of the standardized stopping time are also established.     

BIAS-CORRECTED NONPARAMETRIC SPECTRAL ESTIMATION

Abstract. The theory of nonparametric spectral density estimation based on an observed stretch X₁,…, X_N from a stationary time series has been studied extensively in recent years. However, the most popular spectral estimators, such as the ones proposed by Bartlett, Daniell, Parzen, Priestley and Tukey, are plagued by the problem of bias, which effectively prohibits ?N-convergence of the estimator. This is true even in the case where the data are known to be m-dependent, in which case ?N-consistent estimation is possible by a simple plug-in method. In this report, an intuitive method for the reduction in the bias of a nonparametric spectral estimator is presented. In fact, applying the proposed methodology to Bartlett's estimator results in bias-corrected estimators that are related to kernel estimators with lag-windows of trapezoidal shape. The asymptotic performance (bias, variance, rate of convergence) of the proposed estimators is investigated; in particular, it is found that the trapezoidal lag-window spectral estimator is ?N-consistent in the case of moving-average processes, and ?(N/log/N)-consistent in the case of autoregressive moving-average processes. The finite-sample performance of the trapezoidal lag-window estimator is also assessed by means of a numerical simulation.     

LOCALLY ADAPTIVE LAG-WINDOW SPECTRAL ESTIMATION

Abstract. We propose a procedure for the locally optimal window width in nonparametric spectral estimation, minimizing the asymptotic mean square error at a fixed frequency Λ of a lag-window estimator. Our approach is based on an iterative plug-in scheme. Besides the estimation of a spectral density at a fixed frequency, e.g. at frequency Λ = 0, our procedure allows to perform nonparametric spectral estimation with variable window width which adapts to the smoothness of the true underlying density.     

Plug-in Selection of the Number of Frequencies in Regression Estimates of the Memory Parameter of a Long-memory Time Series

We consider the problem of selecting the number of frequencies, m , in a log-periodogram regression estimator of the memory parameter d of a Gaussian long-memory time series. It is known that under certain conditions the optimal m , minimizing the mean squared error of the corresponding estimator of d , is given by m ^(opt)= Cn ^4/5, where n is the sample size and C is a constant. In practice, C would be unknown since it depends on the properties of the spectral density near zero frequency. In this paper, we propose an estimator of C based again on a log-periodogram regression and derive its consistency. We also derive an asymptotically valid confidence interval for d when the number of frequencies used in the regression is deterministic and proportional to n ^4/5. In this case, squared bias cannot be neglected since it is of the same order as the variance. In a Monte Carlo study, we examine the performance of the plug-in estimator of d , in which m is obtained by using the estimator of C in the formula for m ^(opt) above. We also study the performance of a bias-corrected version of the plug-in estimator of d . Comparisons with the choice m = n ^1/2 frequencies, as originally suggested by Geweke and Porter-Hudak (The estimation and application of long memory time series models. Journal of Time Ser. Anal. 4 (1983), 221–37), are provided.     

A Nonparametric Prewhitened Covariance Estimator

This paper proposes a new nonparametric spectral density estimator for time series models with general autocorrelation. The conventional nonparametric estimator that uses a positive kernel has mean squared error no better than n^?4/5. We show that the best implementation of our estimator has mean squared error of order n^?8/9, provided there is sufficient smoothness present in the spectral density. This is, of course, achieved by bias reduction; however, unlike most other bias reduction methods, like the kernel method with higher‐order kernels, our procedure ensures a positive definite estimate. Our method is a generalization of the well‐known prewhitening method of spectral estimation; we argue that this can best be interpreted as multiplicative bias reduction. Higher‐order expansions for the proposed estimator are derived, providing an improved bandwidth choice that minimizes the mean squared error to the second order. A simulation study shows that the recommended prewhitened kernel estimator reduces bias and mean squared error in spectral density estimation.     

Comparison of Stopping Rules in Sequential Estimation of the Number of Classes in a Population

Abstract

In sequential analysis, investigation of stopping rules is important, as they govern the sampling cost and derivation and accuracy of frequentist inference. We study stopping rules in sampling from a population comprised of an unknown number of classes where all classes are equally likely to occur in each selection. We adopt Blackwell's criterion for a “more informative experiment” to compare stopping rules in our context and derive certain complete class results, which provide some guidance for selecting a stopping rule. We show that it suffices to let the stopping probability, at any time, depend only on the number of selections and the number of discovered classes up to that time. A more informative stopping rule costs a higher expected sample size, and conversely, any given stopping rule can be improved with an increment in expected sample size. Admissibility within all stopping rules with a uniform upper bound on average sample size is also discussed. Any fixed-sample-size rule is shown to be admissible within an appropriate class. Finally, we show that for the minimal sufficient statistic to be complete, which is useful for unbiased estimation, the stopping rule must be nonrandomized.     

Inference for the Fourth‐Order Innovation Cumulant in Linear Time Series

The rescaled fourth‐order cumulant of the unobserved innovations of linear time series is an important parameter in statistical inference. This article deals with the problem of estimating this parameter. An existing nonparametric estimator is first discussed, and its asymptotic properties are derived. It is shown how the autocorrelation structure of the underlying process affects the behaviour of the estimator. Based on our findings and on an important invariance property of the parameter of interest with respect to linear filtering, a pre‐whitening‐based nonparametric estimator of the same parameter is proposed. The estimator is obtained using the filtered time series only; that is, an inversion of the pre‐whitening procedure is not required. The asymptotic properties of the new estimator are investigated, and its superiority is established for large classes of stochastic processes. It is shown that for the particular estimation problem considered, pre‐whitening can reduce the variance and the bias of the estimator. The finite sample performance of both estimators is investigated by means of simulations. The new estimator allows for a simple modification of the multiplicative frequency domain bootstrap, which extends its considerable range of validity. Furthermore, the problem of testing hypotheses about the rescaled fourth‐order cumulant of the unobserved innovations is also considered. In this context, a simple test for Gaussianity is proposed. Some real‐life data applications are presented.     

PEAK-INSENSITIVE NON-PARAMETRIC SPECTRUM ESTIMATION

Abstract. We study the problem of non-parametric spectrum estimation of a stationary time series that might contain periodic components. In this case the periodogram ordinates have a significant amplitude at frequencies near the frequencies of the periodic components. These can be regarded as outliers in an asymptotically exponential sample. We develop a non-parametric estimator for the spectral density that is insensitive to these outliers in the frequency domain. This is done by robustifying the usual kernel estimator (smoothed periodogram) by means of M-estimation in the frequency domain. We propose to use data-tapered periodograms, which yield a drastic improvement of the procedure, typically for the contaminated situation. This is both shown theoretically and supported by means of simulation. We show consistency of the resulting estimator in the general case, and asymptotic normality in the special case of a Gaussian time series, whether contamination is present or not. Finally we illustrate the finite sample performance of the estimating procedure by some simulation results and by application to the Canadian lynx trappings data.     

Exact Risks of Sequential Point Estimators of the Exponential Parameter

Abstract

Under purely sequential sampling schemes, a theory is developed for the exact determination of the distributions of two classes of stopping variables (rules) in order to handle point estimation problems for the parametric functionals in an exponential distribution. Explicit formulae are derived for the expected value and risks of sequential estimators of the mean, failure rate, and reliability function of an exponential distribution. These are utilized to compare performances of several competing estimators of the mean and the failure rate.     

A Simple Test for White Noise in Functional Time Series

We propose a new procedure for white noise testing of a functional time series. Our approach is based on an explicit representation of the L²‐distance between the spectral density operator and its best (L²‐)approximation by a spectral density operator corresponding to a white noise process. The estimation of this distance can be easily accomplished by sums of periodogram kernels, and it is shown that an appropriately standardized version of the estimator is asymptotically normal distributed under the null hypothesis (of functional white noise) and under the alternative. As a consequence, we obtain a very simple test (using the quantiles of the normal distribution) for the hypothesis of a white noise functional process. In particular, the test does not require either the estimation of a long‐run variance (including a fourth order cumulant) or resampling procedures to calculate critical values. Moreover, in contrast to all other methods proposed in the literature, our approach also allows testing for ‘relevant’ deviations from white noise and constructing confidence intervals for a measure that measures the discrepancy of the underlying process from a functional white noise process.     

Variable Bandwidth Kernel Estimators of the Spectral Density

In this paper we investigate the theoretical properties of a nonparametric kernel regression estimator of the spectral density when the bandwidth is selected locally. We also analyze the relationship between the global and the locally selected bandwidths, presenting some simulation results and an application to the estimation of the spectral density of the Spanish money multiplier.     

A superharmonic prior for the autoregressive process of the second‐order

Abstract. The Bayesian estimation of the spectral density of the AR(2) process is considered. We propose a superharmonic prior on the model as a non‐informative prior rather than the Jeffreys prior. Theoretically, the Bayesian spectral density estimator based on it dominates asymptotically the one based on the Jeffreys prior under the Kullback–Leibler divergence. In the present article, an explicit form of a superharmonic prior for the AR(2) process is presented and compared with the Jeffreys prior in computer simulation.     

A Robbins–Monro Algorithm for Non‐Parametric Estimation of NAR Process with Markov Switching: Consistency

We approach the problem of non‐parametric estimation for autoregressive Markov switching processes. In this context, the Nadaraya–Watson‐type regression functions estimator is interpreted as a solution of a local weighted least‐square problem, which does not admit a closed‐form solution in the case of hidden Markov switching. We introduce a non‐parametric recursive algorithm to approximate the estimator. Our algorithm restores the missing data by means of a Monte Carlo step and estimates the regression function via a Robbins–Monro step. We prove that non‐parametric autoregressive models with Markov switching are identifiable when the hidden Markov process has a finite state space. Consistency of the estimator is proved using the strong α‐mixing property of the model. Finally, we present some simulations illustrating the performances of our non‐parametric estimation procedure.     

Hidden Frequency Estimation with Data Tapers

The detection and estimation of hidden frequencies has long been recognized as an important problem in time series. In this paper we study the asymptotic theory for two methods of high-precision estimation of hidden frequencies (the secondary analysis method and the maximum periodogram method) using a data taper. In ordinary situations, a data taper may reduce the estimation precision slightly. However, when there are high peaks in the spectral density of the noise or other strong hidden periodicities with frequencies close to the hidden frequency of interest, the procedures for detection of the existence of and estimation of the hidden frequency of interest fail if data are nontapered whereas they may work well if the data are tapered. The theoretical results are verified by some simulated examples.     

CONSISTENT ESTIMATION OF THE FOURTH-ORDER CUMULANT SPECTRAL DENSITY

Abstract. In this paper we consider the estimation of the fourth-order cumulant spectral density. Indeed this is the first case where the cumulant depends on lower-order product moments for a mean-zero stationary process. The proposed estimator of the fourth-order cumulant spectral density is constructed by replacing product moments with appropriately weighted estimates of product moments according to the definition of the fourth-order cumulant spectral density. Asymptotic unbiasedness and consistency are shown to hold for these estimators under stationarity and absolute summability of cumulants up to various orders with no restrictions on the frequencies. An expression for the asymptotic variance is also obtained.     

Broadband Semiparametric Estimation of the Memory Parameter of a Long-Memory Time Series Using Fractional Exponential Models

We consider a fractional exponential, or FEXP estimator of the memory parameter of a stationary Gaussian long-memory time series. The estimator is constructed by fitting a FEXP model of slowly increasing dimension to the log periodogram at all Fourier frequencies by ordinary least squares, and retaining the corresponding estimated memory parameter. We do not assume that the data were necessarily generated by a FEXP model, or by any other finite-parameter model. We do, however, impose a global differentiability assumption on the spectral density except at the origin. Because of this, and its use of all Fourier frequencies, we refer to the FEXP estimator as a broadband semiparametric estimator. We demonstrate the consistency of the FEXP estimator, and obtain expressions for its asymptotic bias and variance. If the true spectral density is sufficiently smooth, the FEXP estimator can strongly outperform existing semiparametric estimators, such as the Geweke–Porter-Hudak (GPH) and Gaussian semiparametric estimators (GSE), attaining an asymptotic mean squared error proportional to (log n )/ n , where n is the sample size. In a simulation study, we demonstrate the merits of using a finite-sample correction to the asymptotic variance, and we also explore the possibility of automatically selecting the dimension of the exponential model using Mallows' C_L criterion.     

Minimum risk sequential point estimation of the stress-strength reliability parameter for exponential distribution

Abstract

In this article, using purely and two-stage sequential procedures, the problem of minimum risk point estimation of the reliability parameter (R) under the stress–strength model, in case the loss function is squared error plus sampling cost, is considered when the random stress (X) and the random strength (Y) are independent and both have exponential distributions with different scale parameters. The exact distribution of the total sample size and explicit formulas for the expected value and mean squared error of the maximum likelihood estimator of the reliability parameter under the stress–strength model are provided under the two-stage sequential procedure. Using the law of large numbers and Monte Carlo integration, the exact distribution of the stopping rule under the purely sequential procedure is approximated. Moreover, it is shown that both proposed sequential procedures are finite and for special cases the exact distribution of stopping times has a degenerate distribution at the initial sample size. The performances of the proposed methodologies are investigated with the help of simulations. Finally, using a real data set, the procedures are clearly illustrated.     

On the Spectral Density of the Wavelet Coefficients of Long‐Memory Time Series with Application to the Log‐Regression Estimation of the Memory Parameter

Abstract. In recent years, methods to estimate the memory parameter using wavelet analysis have gained popularity in many areas of science. Despite its widespread use, a rigorous semi‐parametric asymptotic theory, comparable with the one developed for Fourier methods, is still lacking. In this article, we adapt to the wavelet setting, the classical semi‐parametric framework introduced by Robinson and his co‐authors for estimating the memory parameter of a (possibly) non‐stationary process. Our results apply to a class of wavelets with bounded supports, which include but are not limited to Daubechies wavelets. We derive an explicit expression of the spectral density of the wavelet coefficients and show that it can be approximated, at large scales, by the spectral density of the continuous‐time wavelet coefficients of fractional Brownian motion. We derive an explicit bound for the difference between the spectral densities. As an application, we obtain minimax upper bounds for the log‐scale regression estimator of the memory parameter for a Gaussian process and we derive an explicit expression of its asymptotic variance.     

An Asymptotic F Test for Uncorrelatedness in the Presence of Time Series Dependence

We propose a simple asymptotically F-distributed Portmanteau test for zero autocorrelations in an otherwise dependent time series. By employing the orthonormal series variance estimator of the variance matrix of sample autocovariances, our test statistic follows an F distribution asymptotically under fixed-smoothing asymptotics. The asymptotic F theory accounts for the estimation error in the underlying variance estimator, which the asymptotic chi-squared theory ignores. Monte Carlo simulations reveal that the F approximation is much more accurate than the corresponding chi-squared approximation in finite samples. The asymptotic F test is as easy to use as the chi-squared test: there is no need to obtain critical values by simulations. Furthermore, it has more accurate empirical sizes and substantial power advantages, comparing to other competitors.