Facts and fiction in spectral analysis

This analysis is limited to the spectral analysis of stationary stochastic processes with unknown spectral density. The main spectral estimation methods are: parametric with time series models, or nonparametric with a windowed periodogram. A single time series model will be chosen with a statistical criterion from three previously estimated and selected models: the best autoregressive (AR) model, the best moving average (MA) model, and the best combined ARMA model. The accuracy of the spectrum, computed from this single selected time series model, is compared with the accuracy of some windowed periodogram estimates. The time series model generally gives a spectrum that is better than the best possible windowed periodogram. It is a fact that a single good time series model can be selected automatically for statistical data with unknown spectral density. It is fiction that objective choices between windowed periodograms can be made     

Let the Data Speak for Themselves

An estimation algorithm for stationary random data automatically selects a single time-series (TS) model for a given number of observations. The parameters of that model accurately represent the spectral density and the autocovariance function of the data. The increased computational speed has given the possibility to compute hundreds of TS models and to select only one. The computer program uses a selection criterion to determine the best model type and model order from a large number of candidates. That selected model includes all statistically significant details that are present in the data, and no more. The spectral density of high-order TS models is the same as the raw periodogram, and the autocorrelation function can be the same as the lagged product (LP) estimate. Therefore, the periodogram and the LP autocorrelation function are very high-order TS candidates. However, those high-order models are never selected in practice because they contain many insignificant details. The automatic selection with the algorithm lets the data speak for themselves: a single model is selected without user interaction. The automatic program can be implemented in measurement instruments for maintenance or in radar, by automatically detecting differences in signal properties     

Time-series analysis if data are randomly missing

Maximum-likelihood (ML) theory presents an elegant asymptotic solution for the estimation of the parameters of time-series models. Unfortunately, the performance of ML algorithms in finite samples is often disappointing, especially in missing-data problems. The likelihood function is symmetric with respect to the unit circle for the estimated zeros of time-series models. As a consequence, the unit circle is either a local maximum or a local minimum in the likelihood of moving-average (MA) models. This is a trap for nonlinear optimization algorithms that often converge to poor models, with estimated zeros precisely on the unit circle. With ML estimation, it is much easier to estimate a long autoregressive (AR) model with only poles. The parameters of that long AR model can then be used to estimate MA and autoregressive moving-average (ARMA) models for different model orders. The accuracy of the estimated AR, MA, and ARMA spectra is very good. The robustness is excellent as long as the AR order is less than 10 or 15. For still-higher AR orders until about 60, the possible convergence to a useful model will depend on the missing fraction and on the specific properties of the data at hand.     

Historical Misconceptions in Autocorrelation Estimation

The sample autocovariance function, which is estimated as mean lagged products (LPs) of random observations, can also be obtained as the inverse Fourier transform of the periodogram of the data that are augmented with zeros. Hence, the quality of the sample autocovariance as a representation of stochastic data is the same as that of a raw periodogram. The LP estimate is not based on any efficient estimation principle for random data. The spectral density and the autocovariance function can be estimated much more accurately with parametric time series models. A recent development in time series analysis gives the possibility to select automatically the type and the order of the best time series model for any set of random observations. The spectral accuracy of the selected model is better than the accuracy of modified periodograms. In addition, the accuracy of the parametric estimate of the autocovariance function is the same or better for every individual lag than what can be achieved by the nonparametric mean-LP estimates. Perhaps even more important, the estimated time series parameters define the autocovariance as a complete function, for all lags together.     

The Quality of Lagged Products and Autoregressive Yule–Walker Models as Autocorrelation Estimates

The sample autocorrelation function is defined by the mean lagged products (LPs) of random observations. It is the inverse Fourier transform of the raw periodogram. Both contain the same information, and the quality of the full-length sample autocorrelation to represent random data is as poor as that of a raw periodogram. The autoregressive (AR) Yule-Walker method uses LP autocorrelation estimates to compute AR parameters as a parametric model for the autocorrelation. The order of the AR model can be taken as the full LP length, or it can be determined with an order selection criterion. However, the autocorrelation function can more accurately be estimated with a general parametric time-series method. This parametric estimate of the autocorrelation function always has better accuracy than the LP estimates. The LP autocorrelation function is as long as the observation window, but parametric estimates will eventually die out. They allow an objective answer to the question of how long the autocorrelation function really is.     

Finite sample properties of ARMA order selection

The cost of order selection is defined as the loss in model quality due to selection. It is the difference between the quality of the best of all available candidate models that have been estimated from a finite sample of N observations and the quality of the model that is actually selected. The order selection criterion itself has an influence on the cost because of the penalty factor for each additionally selected parameter. Also, the number of competitive candidate models for the selection is important. The number of candidates is, of itself, small for the nested and hierarchical autoregressive/moving average (ARMA) models. However, intentionally reducing the number of selection candidates can be beneficial in combined ARMA(p,q) models, where two separate model orders are involved: the AR order p and the MA order q. The selection cost can be diminished by creating a nested sequence of ARMA(r,r-1) models. Moreover, not evaluating every combination (p,q) of the orders considerably reduces the required computation time. The disadvantage may be that the true ARMA(p,q) model is no longer among the nested candidate models. However, in finite samples, this disadvantage is largely compensated for by the reduction in the cost of order selection by considering fewer candidates. Thus, the quality of the selected model remains acceptable with only hierarchically nested ARMA(r,r-1) models as candidates.     

Reliable LDA-spectra by resampling and ARMA-modeling

Laser-Doppler Anemometry (LDA) is used to measure the velocity of gases and liquids with observations irregularly spaced in time. Equidistant resampling turns out to be better than slotting techniques. After resampling, two ways of spectral estimation are compared. The first estimate is a windowed periodogram and the second is the spectrum of a time series model. That is an estimated autoregressive moving average (ARMA) process whose orders are automatically selected from the data with an objective statistical criterion. Typically, the ARMA spectrum is better than the best windowed periodogram     

On the usefulness of singular value decomposition-ARMA models in Doppler ultrasound

The singular value decomposition (SVD) autoregressive moving average, (ARMA) procedure is applied to computer-generated synthetic Doppler signals as well as in-vivo Doppler data recorded in the carotid artery. Two essential algorithmic parameters (the initially proposed model order and the number of overdetermined equations used) prove difficult to choose. The resulting spectra are very dependent on these two parameters. For the simulated data models orders of (3, 3) provide good results. However, when applying the SVD-ARMA algorithm to in-vivo Doppler signals no single set of model orders was capable of producing consistent spectral estimates throughout the cardiac cycle. Altering the model orders also necessitates the selection of new algorithmic parameters. Hence, the SVD-ARMA approach cannot be considered suitable as a spectral estimation technique, for real-time Doppler ultrasound systems     

Time series analysis in a frequency subband

Standard time series analysis estimates the power spectral density over the full frequency range, until half the sampling frequency. In several input-output identification problems, frequency selective model estimation is desirable. Processing of a time series in a subband may also be useful if observations of a stochastic process are analyzed for the presence or multiplicity of spectral peaks. If two close spectral peaks are present, a minimum number of observations is required to observe two separate narrow peaks with sufficient statistical reliability. Otherwise, with less data, a model with one single broad peak might be selected. A high order autoregressive model will always indicate the separate peaks in the power spectral density, together with many other similar details that are not significant. However, order selection among full-range models may select a model with a single peak. By using subband order selection, it is sometimes possible to detect the presence of two peaks from the same data. Therefore, spectral details can be analyzed from fewer observations with a subband analysis.     

Automatic spectral analysis with time series models

The increased computational speed and developments in the robustness of algorithms have created the possibility to identify automatically a well-fitting time series model for stochastic data. It is possible to compute more than 500 models and to select only one, which certainly is one of the better models, if not the very best. That model characterizes the spectral density of the data. Time series models are excellent for random data if the model type and the model order are known. For unknown data characteristics, a large number of candidate models have to be computed. This necessarily includes too low or too high model orders and models of the wrong types, thus requiring robust estimation methods. The computer selects a model order for each of the three model types. From those three, the model type with the smallest expectation of the prediction error is selected. That unique selected model includes precisely the statistically significant details that are present in the data     

Experimental comparison of fourier transformation and model-fitting methods of spectral analysis

A summary of the results of an extensive comparative experimental study of Fourier transformation and model-fitting methods of spectral analysis of random time-series data is presented. It is illustrated that Fourier transformation methods can be an essential companion to model-fitting methods even for short data segments with underlying sharp spectral peaks. The best spectrum estimates can be obtained by taking advantage of the strengths of both types of methods. For example, it is shown that detection and estimation of the frequencies of spectral lines for short data segments can be best accomplished using certain parametric methods in conjunction with Fourier transformation methods to aid in model-order selection and identification of spurious peaks in the parametric spectrum estimate, and that estimation of amplitude and phase for sine-wave removal, given frequency estimates, and spectrum estimation after sine-wave removal can often be best accomplished with Fourier transformation methods alone.     

Bandlimited rational models for signal reconstruction

A rational model is proposed for reconstruction of bandlimited signals from successive observations, bandlimits, and knowledge of signal peak location. The parameters of the model may be estimated from the singular vectors of a certain double-Hankel matrix constructed from this information. In contrast to the currently popular minimum-energy approach, which has limited spectral resolution and is sensitive to noise, high spectral resolution is possible with the rational model because of its pole-zero nature. To reduce the sensitivity of the rational reconstruction to noise and modelling errors, rank reduction of the double-Hankel matrix via singular value decomposition is suggested. The singular values of the matrix provide estimates of the error in modelling the data with rational models of different orders, and so the singular value distribution can be used to select model order. This work was partially supported by SDIO/IST under contract DAAL 03-86-0111 administered by the US Army Research Office, by the Bell Communication Research’s Graduate Study Program, and by Cray Research Inc. under grants DMS9880010SC and ECS890002SC and utilized by the Cray X-MP/48 at the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign.     

Generating data with prescribed power spectral density

Data generation is straightforward if the parameters of a time series model define the prescribed spectral density or covariance function. Otherwise, a time series model has to be determined. An arbitrary prescribed spectral density will be approximated by a finite number of equidistant samples in the frequency domain. This approximation becomes accurate by taking more and more samples. Those samples can be inversely Fourier transformed into a covariance function of finite length. The covariance in turn is used to compute a long autoregressive (AR) model with the Yule-Walker relations. Data can be generated with this long AR model. The long AR model can also be used to estimate time series models of different types to search for a parsimonious model that attains the required accuracy with less parameters. It is possible to derive objective rules to choose a preferred type with a minimal order for the generating time series model. That order will generally depend on the number of observations to be generated. The quality criterion for the generating time series model is that the spectrum estimated from the generated number of observations cannot be distinguished from the prescribed spectrum.     

Parallel factor analysis (PARAFAC) of target analytes in GC x GC-TOFMS data: automated selection of a model with an appropriate number of factors

PARAFAC (parallel factor analysis) is a powerful chemometric method that has been demonstrated as a useful deconvolution technique in dealing with data obtained using comprehensive two-dimensional gas chromatography combined with time-of-flight mass spectrometry (GC x GC-TOFMS). However, selection of a PARAFAC model having an appropriate number of factors can be challenging, especially at low S/N or for analytes in the presence of chromatographic and spectral overlapping compounds (interferences). Herein, we present a method for the automated selection of a PARAFAC model with an appropriate number of factors in GC x GC-TOFMS data, demonstrated for a target analyte of interest. The approach taken in the methodology is as follows. PARAFAC models are automatically generated having an incrementally higher number of factors until mass spectral matching of the corresponding loadings in the model against a target analyte mass spectrum indicates overfitting has occurred. Then, the model selected simply has one less factor than the overfit model. Results indicate this model selection approach is viable across the detection range of the instrument from overloaded analyte signal down to low S/N analyte signal (total ion current signal intensity at analyte peak maximum S/N < 1). While the methodology is generally applicable to comprehensive two-dimensional separations using multichannel spectral detection, we evaluated it with several target analytes using GC x GC-TOFMS. For brevity in this report, only results for bromobenzene as target analyte are presented. Alternatively, instead of using the model with one less factor than the overfit model, one can select the model with the highest mass spectral match for the target analyte from among all the models generated (excluding the overfit model). Both model selection approaches gave essentially identical results.     

Modified Durbin Method for Accurate Estimation of Moving-Average Models

Spectra with narrow valleys can accurately be described with moving-average (MA) models by using only a small number of parameters. Durbin's MA method uses the estimated parameters of a long autoregressive (AR) model to calculate the MA parameters. Probably all the pejorative remarks on the quality of Durbin's method in the literature are based on suboptimal or wrong choices for the method of AR estimation or for the order of the intermediate AR model. Generally, the AR order should considerably be higher than the order of the best predicting AR model, and it should grow with the sample size. Furthermore, the Burg estimates for the AR parameters give the best results because they have the smallest variance of all the AR methods with a small bias. A modified Durbin MA method uses a properly defined number of AR parameters, which was estimated with Burg's method, and outperforms all the other known MA estimation methods, asymptotically as well as in finite samples. The accuracy is generally close to the Cramer-Rao bound.     

Doppler angle estimation using AR modeling

The transit time spectrum broadening effect has long been explored for Doppler angle estimation. Given acoustic beam geometry, the Doppler angle can be derived based on the mean Doppler frequency and the Doppler bandwidth. Spectral estimators based on the fast Fourier transform (FFT) are typically used. One problem with this approach is that a long data acquisition time is required to achieve adequate spectral resolution, with typically 32-128 flow samples being needed. This makes the method unsuitable for real-time two-dimensional Doppler imaging. This paper proposes using an autoregressive (AR) model to obtain the Doppler spectrum using a small number (e.g., eight) of flow samples. The flow samples are properly selected, then extrapolated to ensure adequate spectral resolution. Because only a small number of samples are used, the data acquisition time is significantly reduced and real-time, two-dimensional Doppler angle estimation becomes feasible. The approach was evaluated using both simulated and experimental data. Flows with various degrees of velocity gradient were simulated, with the Doppler angle ranging from 20° to 75°. The results indicate that the AR method generally provided accurate Doppler bandwidth estimates. In addition, the AR method outperformed the FFT method at smaller Doppler angles. The experimental data for Doppler angles, ranging from 33° to 72°, showed that the AR method using only eight flow samples had an average estimation error of 3.6°, which compares favorably to the average error of 4.7° for the FFT method using 64 flow samples. Because accurate estimates can be obtained using a small number of flow samples, it is concluded that real-time, two-dimensional estimation of the Doppler angle over a wide range of angles is possible using the AR method     

Error Correction of Rainfall-Runoff Models With the ARMAsel Program

Improved predictions can be based on recent observed differences or errors between the best available model predictions and the actual measured data. This is possible in the predicted amount of supplies, services, sewage, transportation, power, water, heat, or gas, as well as in the predicted level of rivers. As an example, physical modeling of the dynamics of a catchment area produces models with a limited forecasting accuracy for the discharge of rivers. The discrepancies between the model and the actually observed past discharges can be used as information for error correction. With a time-series model of the error signal, an improved discharge forecast can be made for the next few days. The best type and order of the forecasting time-series model can be automatically selected. Adaptive modeling in data assimilation calculates updates of the time-series model estimated from the error data of only the last few weeks. The use of variable updated models has advantages in periods with the largest discharges, which are most important in flood forecasting.     

Identification of patterns in control charts for processes with statistically correlated noise

In real industrial scenarios, if the quality characteristics of a continuous or batch production process are monitored using Shewhart control charts, there could be a large number of false alarms about the process going out of control. This is because these control charts assume that the inherent noise of the monitored process is normally, independently and identically distributed, although the assumption of independence is not always correct for continuous and batch production processes. This paper presents three control chart pattern recognition systems where the inherent disturbance is assumed to be stationary. The systems use the first-order autoregressive (AR(1)), moving-average (MA(1)) and autoregressive moving-average (ARMA(1,1)) models. A special pattern generation scheme is adopted to ensure generality, randomness and comparability, as well as allowing the further categorisation of the studied patterns. Two different input representation techniques for the recognition systems were studied. These gave nearly the same performance for the MA(1) and ARMA(1,1) models, while the raw data yielded the highest accuracies when AR(1) was used. The effect of autocorrelation on the pattern recognition capabilities of the developed models was studied. It was observed that Normal and Upward Shift patterns were the most affected.     

Discrimination between accelerated life models via Approximate Bayesian Computation

Accelerated life testing (ALT) is widely used in high-reliability product estimation to get relevant information about an item's performance and its failure mechanisms. To analyse the observed ALT data, reliability practitioners need to select a suitable accelerated life model based on the nature of the stress and the physics involved. A statistical model consists of (i) a lifetime distribution that represents the scatter in product life and (ii) a relationship between life and stress. In practice, several accelerated life models could be used for the same failure mode and the choice of the best model is far from trivial. For this reason, an efficient selection procedure to discriminate between a set of competing accelerated life models is of great importance for practitioners. In this paper, accelerated life model selection is approached by using the Approximate Bayesian Computation (ABC) method and a likelihood-based approach for comparison purposes. To demonstrate the efficiency of the ABC method in calibrating and selecting accelerated life model, an extensive Monte Carlo simulation study is carried out using different distances to measure the discrepancy between the empirical and simulated times of failure data. Then, the ABC algorithm is applied to real accelerated fatigue life data in order to select the most likely model among five plausible models. It has been demonstrated that the ABC method outperforms the likelihood-based approach in terms of reliability predictions mainly at lower percentiles particularly useful in reliability engineering and risk assessment applications. Moreover, it has shown that ABC could mitigate the effects of model misspecification through an appropriate choice of the distance function.     

Investigating the effects of the fixed and varying dispersion parameters of Poisson-gamma models on empirical Bayes estimates

Traditionally, transportation safety analysts have used the empirical Bayes (EB) method to improve the estimate of the long-term mean of individual sites; to correct for the regression-to-the-mean (RTM) bias in before-after studies; and to identify hotspots or high risk locations. The EB method combines two different sources of information: (1) the expected number of crashes estimated via crash prediction models, and (2) the observed number of crashes at individual sites. Crash prediction models have traditionally been estimated using a negative binomial (NB) (or Poisson-gamma) modeling framework due to the over-dispersion commonly found in crash data. A weight factor is used to assign the relative influence of each source of information on the EB estimate. This factor is estimated using the mean and variance functions of the NB model. With recent trends that illustrated the dispersion parameter to be dependent upon the covariates of NB models, especially for traffic flow-only models, as well as varying as a function of different time-periods, there is a need to determine how these models may affect EB estimates. The objectives of this study are to examine how commonly used functional forms as well as fixed and time-varying dispersion parameters affect the EB estimates. To accomplish the study objectives, several traffic flow-only crash prediction models were estimated using a sample of rural three-legged intersections located in California. Two types of aggregated and time-specific models were produced: (1) the traditional NB model with a fixed dispersion parameter and (2) the generalized NB model (GNB) with a time-varying dispersion parameter, which is also dependent upon the covariates of the model. Several statistical methods were used to compare the fitting performance of the various functional forms. The results of the study show that the selection of the functional form of NB models has an important effect on EB estimates both in terms of estimated values, weight factors, and dispersion parameters. Time-specific models with a varying dispersion parameter provide better statistical performance in terms of goodness-of-fit (GOF) than aggregated multi-year models. Furthermore, the identification of hazardous sites, using the EB method, can be significantly affected when a GNB model with a time-varying dispersion parameter is used. Thus, erroneously selecting a functional form may lead to select the wrong sites for treatment. The study concludes that transportation safety analysts should not automatically use an existing functional form for modeling motor vehicle crashes without conducting rigorous analyses to estimate the most appropriate functional form linking crashes with traffic flow.