Parameter change test for random coefficient integer-valued autoregressive processes with application to polio data analysis

Abstract.  In this paper, we consider the problem of testing for a parameter change in a first-order random coefficient integer-valued autoregressive [RCINAR(1)] model. We employ the cumulative sum (CUSUM) test based on the conditional least-squares and modified quasi-likelihood estimators. It is shown that under regularity conditions, the CUSUM test has the same limiting distribution as the supremum of the squares of independent Brownian bridges. The CUSUM test is then applied to the analysis of the monthly polio counts data set.     

A STATE SPACE TIME SERIES MODELLING METHOD WITHOUT INDIVIDUAL DETRENDING

Abstract. A state space method for building time series models without detrending each component of data vectors individually is presented. The method uses the recent algorithm based on the singular-value decomposition of the Hankel matrix and a two-step sequential procedure suggested by the notion of dynamic aggregation. Some asymptotic properties of the estimators of the model parameter and error estimates are also presented.     

Parameter Estimation Under Failure-Censored Constant-Stress Life Testing Model: A Bayesian Approach

Abstract

This article compares likelihood and Bayesian estimations for partially accelerated constant-stress life test model under type II censoring assuming Pareto distribution of the second kind. Both maximum likelihood and Bayesian estimators of the model parameters are derived. The posterior means and posterior variances are obtained under the squared error loss function using Lindley's approximation procedure. The advantages of this proposed procedure are shown. Monte Carlo simulations are conducted under different samples sizes and different parameter values to assess and compare the proposed methods of estimation. A noninformative prior on the model parameters is used to make the comparison more meaningful. It has been observed that Lindley's method usually provides posterior variances and mean squared errors smaller than those of the maximum likelihood estimators. That is, Lindley's method produces improved estimates, which is an advantage of this method.     

Estimation in Functional Lagged Regression

The paper introduces a functional time series (lagged) regression model. The impulse‐response coefficients in such a model are operators acting on a separable Hilbert space, which is the function space L² in applications. A spectral approach to the estimation of these coefficients is proposed and asymptotically justified under a general nonparametric condition on the temporal dependence of the input series. Since the data are infinite‐dimensional, the estimation involves a spectral‐domain dimension‐reduction technique. Consistency of the estimators is established under general data‐dependent assumptions on the rate of the dimension‐reduction parameter. Their finite‐sample performance is evaluated by a simulation study that compares two ad hoc approaches to dimension reduction with an alternative, asymptotically justified method.     

Redescending estimators for data reconciliation and parameter estimation

Gross error detection is crucial for data reconciliation and parameter estimation, as gross errors can severely bias the estimates and the reconciled data. Robust estimators significantly reduce the effect of gross errors (or outliers) and yield less biased estimates. An important class of robust estimators are maximum likelihood estimators or M-estimators. These are commonly of two types, Huber estimators and Hampel estimators. The former significantly reduces the effect of large outliers whereas the latter nullifies their effect. In particular, these two estimators can be evaluated through the use of an influence function, which quantifies the effect of an observation on the estimated statistic. Here, the influence function must be bounded and finite for an estimator to be robust. For the Hampel estimators the influence function becomes zero for large outliers, nullifying their effect. On the other hand, Huber estimators do not reject large outliers; their influence function is simply bounded. As a result, we consider the three part redescending estimator of Hampel and compare its performance with a Huber estimator, the Fair function. A major advantage to redescending estimators is that it is easy to identify outliers without having to perform any exploratory data analysis on the residuals of regression. Instead, the outliers are simply the rejected observations. In this study, the redescending estimators are also tuned to the particular observed system data through an iterative procedure based on the Akaike information criterion, (AIC). This approach is not easily afforded by the Huber estimators and this can have a significant impact on the estimation. The resulting approach is incorporated within an efficient non-linear programming algorithm. Finally, all of these features are demonstrated on a number of process and literature examples for data reconciliation.     

Nonlinear stochastic modeling to improve state estimation in process monitoring and control

State estimation from plant measurements plays an important role in advanced monitoring and control technologies, especially for chemical processes with nonlinear dynamics and significant levels of process and sensor noise. Several types of state estimators have been shown to provide high‐quality estimates that are robust to significant process disturbances and model errors. These estimators require a dynamic model of the process, including the statistics of the stochastic disturbances affecting the states and measurements. The goal of this article is to introduce a design method for nonlinear state estimation including the following steps: (i) nonlinear process model selection, (ii) stochastic disturbance model selection, (iii) covariance identification from operating data, and (iv) estimator selection and implementation. Results on the implementation of this design method in nonlinear examples (CSTR and large dimensional polymerization process) show that the linear time‐varying autocovariance least‐squares technique accurately estimates the noise covariances for the examples analyzed, providing a good set of such covariances for the state estimators implemented. On the estimation implementation, a case study of a chemical reactor demonstrates the better capabilities of MHE when compared with the extended Kalman filter. © 2010 American Institute of Chemical Engineers AIChE J, 2011     

On Residual Variance Estimation in Autoregressive Models

In this paper we consider time series models belonging to the autoregressive (AR) family and deal with the estimation of the residual variance. This is important because estimates of the variance are involved in, for example, confidence sets for the parameters of the model, estimation of the spectrum, expressions for the estimated error of prediction and sample quantities used to make inferences about the order of the model. We consider the asymptotic biases for moment and least squares estimators of the residual variance, and compare them with known results when available and with those for maximum likelihood estimators under normality. Simulation results are presented for finite samples     

SOME PROPERTIES OF CONDITIONAL QUASI-LIKELIHOOD FUNCTIONS FOR TIME SERIES MODEL FITTING

Abstract. We consider fitting a parametric model to a time series and obtain the maximum likelihood estimates of unknown parameters included in the model by regarding the time series as a Gaussian process satisfying the model. We evaluate the asymptotic value of the conditional quasi-likelihood function when the number of observations tends to infinity. We show what properties of the time series we can find by examining the behaviour of the conditional quasi-likelihood function, even when the time series does not necessarily satisfy the model and is not necessarily Gaussian.     

An Estimating Method for Parametric Spectral Densities of Gaussian Time Series

An estimating method for spectral densities of Gaussian time series that belong to a parametric model is proposed. Spectral density estimators are evaluated by using average Kullback–Leibler divergence from the true spectral density to estimated spectral densities. In the classical approach, unknown spectral densities are estimated by replacing the unknown parameters by asymptotically efficient estimates. In the alternative method introduced in the present paper, spectral density estimates usually do not belong to the model. The alternative spectral density estimators asymptotically dominate the classical ones. The difference in average Kullback–Leibler divergence between them can be regarded as the mixture mean curvature of the model in the space of all spectral densities. The explicit expression for the proposed estimators of spectral densities of autoregressive processes is obtained. The accuracy of prediction can be improved by using predictors that correspond to the alternative spectral density estimators.     

An Approximate Innovation Method For The Estimation Of Diffusion Processes From Discrete Data

Abstract. In this paper, an approximate innovation method is introduced for the estimation of diffusion processes, given a set of discrete and noisy observations of some of their components. The method is based on a recent extension of local linearization filters to the general case of continuous–discrete state–space models with multiplicative noise. This filtering method provides adequate approximations for the prediction and filter estimates that are required by the innovation method in the estimation of the unknown parameters and the unobserved component of the diffusion process. The performance of approximate innovation estimators is illustrated by means of numerical simulations.     

A NOTE ON ASYMPTOTIC INFERENCE IN AUTOREGRESSIVE MODELS WITH ROOTS ON THE UNIT CIRCLE

Abstract. An estimation and inference procedure is proposed for parameters of the p th order autoregressive model with roots both on the unit circle and outside the unit circle. The procedure is motivated by the fact that the parameter estimates of the nonstationary part of the model have higher order consistency properties than the parameter estimates of the stationary part. The procedure allows the use of the known asymptotic distributional results of purely nonstationary models and purely stationary models. Only ordinary least squares routines are needed.     

Nonlinear parameter estimation through particle swarm optimization

Parameter estimation procedures are very important in the chemical engineering field for development of mathematical models, since design, optimization and advanced control of chemical processes depend on model parameter values obtained from experimental data. Model nonlinearity makes the estimation of parameter and the statistical analysis of parameter estimates more difficult and more challenging. In this work, it is shown that many of these difficulties can be overcome with the use of heuristic optimization methods, such as the particle swarm optimization (PSO) method. Parameter estimation problems are solved here with PSO and it is shown that the PSO method is efficient for both minimization and construction of the confidence region of parameter estimates. Moreover, it is shown that the elliptical approximation of confidence regions of nonlinear model parameters can be very poor sometimes and that more accurate likelihood confidence regions can be constructed with PSO, allowing for more reliable statistical analysis of the significance of parameter estimates.     

Nonlinear inferential control

Due to unmeasured distrubances and nonlinearities typical for chemical processes the performance of state reconstruction schemes based on linearized system is often unsatisfactory. Unmeasured disturbances yield biased state estimates because generally only proportional feedback is used in the estimators. Set point changes or large disturbances make linear estimators invalid for most chemical processes. Static and dynamic nonlinear estimation schemes are derived in this paper when persistent or slowly varying disturbances (nonstationary noise) affect the system. Because an analytical solution for general nonlinear systems is impossible, approximate methods are suggested to reduce the computational effort necessary for evaluating the estimate. The method is applied to two CSTR's in series where concentration estimates are obtained from temperature measurements. The results are significantly better than those obtained by linear estimation techniques. A convenient measurement selection criterion is also derived which aims at minimizing the sensitivity of the estimate to unmeasured disturbances.     

Asymptotic self‐similarity and wavelet estimation for long‐range dependent fractional autoregressive integrated moving average time series with stable innovations

Abstract. Methods for parameter estimation in the presence of long‐range dependence and heavy tails are scarce. Fractional autoregressive integrated moving average (FARIMA) time series for positive values of the fractional differencing exponent d can be used to model long‐range dependence in the case of heavy‐tailed distributions. In this paper, we focus on the estimation of the Hurst parameter H = d + 1/α for long‐range dependent FARIMA time series with symmetric α‐stable (1 < α < 2) innovations. We establish the consistency and the asymptotic normality of two types of wavelet estimators of the parameter H. We do so by exploiting the fact that the integrated series is asymptotically self‐similar with parameter H. When the parameter α is known, we also obtain consistent and asymptotically normal estimators for the fractional differencing exponent d = H ? 1/α. Our results hold for a larger class of causal linear processes with stable symmetric innovations. As the wavelet‐based estimation method used here is semi‐parametric, it allows for a more robust treatment of long‐range dependent data than parametric methods.     

Sequential design including initial experimental runs for a kinetic study of propylene oxidation

A sequential design strategy for selecting experimental runs to obtain model discrimination and precise parameter estimation is tested via a simulation study of propylene oxidation kinetics. The strategy is used to design all runs including the preliminary ones which were arbitrarily chosen by earlier researchers. To design initial runs, crude initial parameter guesses may be used in the rival models until least squares estimates can be calculated. Even under conditions of very bad initial guesses and high error variances, this procedure selects whichever model is the correct one and estimates with precision its parameters, in fewer runs than previously reported.     

Estimation of the location and exponent of the spectral singularity of a long memory process

Abstract.  We consider the estimation of the location of the pole and memory parameter ω ₀ and d of a covariance stationary process with spectral density
We investigate optimal rates of convergence for the estimators of ω ₀ and d , and the consequence that the lack of knowledge of ω ₀ has on the estimation of the memory parameter d . We present estimators which achieve the optimal rates. A small Monte-Carlo study is included to illustrate the finite sample performance of our estimators.     

FAST LINEAR ESTIMATION METHODS FOR VECTOR AUTOREGRESSIVE MOVING-AVERAGE MODELS

Abstract. Three linear methods for estimating parameter values of vector auto-regressive moving-average (VARMA) models which are in general at least an order of magnitude faster than maximum likelihood estimation are developed in this paper. Simulation results for different model structures with varying numbers of component series and observations suggest that the accuracy of these procedures is in most cases comparable with maximum likelihood estimation. Procedures for estimating parameter standard error are also discussed and used for identification of nonzero elements in the VARMA polynomial structures. These methods can also be used to establish the order of the VARMA structure. We note, however, that the primary purpose of these estimates is to generate initial estimates for the nonzero parameters in order to reduce subsequent computational time of more efficient estimation procedures such as exact maximum likelihood.     

ESTIMATION OF AUTOCOVARIANCE MATRICES FOR INFINITE DIMENSIONAL VECTOR LINEAR PROCESS

Consider an infinite dimensional vector linear process. Under suitable assumptions on the parameter space, we provide consistent estimators of the autocovariance matrices. In particular, under causality, this includes the infinite‐dimensional vector autoregressive (IVAR) process. In that case, we obtain consistent estimators for the parameter matrices. An explicit expression for the estimators is obtained for IVAR(1), under a fairly realistic parameter space. We also show that under some mild restrictions, the consistent estimator of the marginal large dimensional variance–covariance matrix has the same convergence rate as that in case of i.i.d. samples.     

Parameter Estimation in the Error‐in‐Variables Models Using the Gibbs Sampler

Least squares and maximum likelihood techniques have long been used in parameter estimation problems. However, those techniques provide only point estimates with unknown or approximate uncertainty information. Bayesian inference coupled with the Gibbs Sampler is an approach to parameter estimation that exploits modern computing technology. The estimation results are complete with exact uncertainty information. The Error‐in‐Variables model (EVM) approach is investigated in this study. In it, both dependent and independent variables contain measurement errors, and the true values and uncertainties of all measurements are estimated. This EVM set‐up leads to unusually large dimensionality in the estimation problem, which makes parameter estimation very difficult with classical techniques. In this paper, an innovative way of performing parameter estimation is introduced to chemical engineers. The paper shows that the method is simple and efficient; as well, complete and accurate uncertainty information about parameter estimates is readily available. Two real‐world EVM examples are demonstrated: a large‐scale linear model and an epidemiological model. The former is simple enough for most readers to understand the new concepts without difficulty. The latter has very interesting features in that a Poisson distribution is assumed, and a parameter with known distribution is retained while other unknown parameters are estimated. The Gibbs Sampler results are compared with those of the least squares.