A Space-Time Bilinear Model and its Identification

In this paper we propose a class of space–time bilinear (STBL) models which can be used to model space–time series which exhibit bilinear behavior. The STBL model is shown to be an extension of a space–time autoregressive moving-average model and a special form of the multiple bilinear model. We focus on the identification procedure of the models. Some results about stationarity and the covariance structure of these models are also discussed. An identification procedure based on the squared observations is established for the simplest pure bilinear model and some illustrative examples are provided.     

Maximum likelihood estimation of higher‐order integer‐valued autoregressive processes

Abstract. In this article, we extend the earlier work of Freeland and McCabe [Journal of time Series Analysis (2004) Vol. 25, pp. 701–722] and develop a general framework for maximum likelihood (ML) analysis of higher‐order integer‐valued autoregressive processes. Our exposition includes the case where the innovation sequence has a Poisson distribution and the thinning is binomial. A recursive representation of the transition probability of the model is proposed. Based on this transition probability, we derive expressions for the score function and the Fisher information matrix, which form the basis for ML estimation and inference. Similar to the results in Freeland and McCabe (2004) , we show that the score function and the Fisher information matrix can be neatly represented as conditional expectations. Using the INAR(2) specification with binomial thinning and Poisson innovations, we examine both the asymptotic efficiency and finite sample properties of the ML estimator in relation to the widely used conditional least squares (CLS) and Yule–Walker (YW) estimators. We conclude that, if the Poisson assumption can be justified, there are substantial gains to be had from using ML especially when the thinning parameters are large.     

Limiting distributions of unconditional maximum likelihood unit root test statistics in seasonal time–series models

Abstract.  The likelihood function of a seasonal model, Y _t  =  ρ Y _{t − d}  +  e _t as implemented in computer algorithms under the assumption of stationary initial conditions is a function of ρ which is zero at the point ρ  = 1. It is a smooth function for ρ in the above seasonal model with a well-defined maximum regardless of the data-generating mechanism. Gonzalez-Farias (PhD Thesis, North Carolina State University, 1992) proposed tests for unit roots based on maximizing the stationary likelihood function in nonseasonal time series. We extend it to seasonal time series. The limiting distribution of seasonal unit root test statistics based on the unconditional maximum likelihood estimators are shown. Models having a single mean, seasonal means, and a single-trend variable across the seasons are considered.     

An algorithm for the exact likelihood of a stationary vector autoregressive-moving average model

The so-called innovations form of the likelihood function implied by a stationary vector autoregressive-moving average model is considered without directly using a state–space representation. Specifically, it is shown in detail how to compute the exact likelihood by an adaptation to the multivariate case of the innovations algorithm of Ansley (1979 ) for univariate models. Comparisons with other existing methods are also provided, showing that the algorithm described here is computationally more efficient than the fastest methods currently available in many cases of practical interest.     

Extremes of bilinear time series models

The class of bilinear time series models is an obvious generalization of linear ARMA models and has found many applications in time series modeling. It is known that the sample paths of even the simplest bilinear process may have sudden bursts of large negative and positive values that vary in form and amplitude depending on the model parameters. Yet, little is known about the extremal properties of this class. In this paper, we look at the extremal properties of bilinear processes and explain how model parameters affect the extremal behavior.     

On the three-step non-Gaussian quasi-maximum likelihood estimation of heavy-tailed double autoregressive models

This note considers a three-step non-Gaussian quasi-maximum likelihood estimation (TS-NGQMLE) of the double autoregressive model with its asymptotics, which improves efficiency of the GQMLE and circumvents inconsistency of the NGQMLE when the innovation is heavy-tailed. Under mild conditions, the estimator not only can achieve consistency and asymptotic normality regardless of density misspecification of the innovation, but also outperforms the existing estimators, such as the GQMLE and the (weighted) least absolute deviation estimator, when the innovation is indeed heavy-tailed.     

Maximum likelihood estimation of pseudo-stoichiometry in macroscopic biological reaction schemes

Identification of pseudo-stoichiometric (or yield) coefficients is of primary importance for building a bioprocess model. In most of the applications, the estimation of these coefficients has to be performed without any knowledge of the kinetics and on the basis of a few experiments for which noisy discrete measurements of component concentrations are available. This paper proposes maximum likelihood estimators which are able to deal with measurement errors on all the signals, at each sampling time (including the initial one) and with intrinsic sign constraints on the parameters. This kind of realistic hypotheses exclude the use of the usual (weighted) least-squares estimators. The maximum likelihood estimators are proved to be unbiased (provided a first-order approximation) and their estimation error covariance matrix can be computed (at the same level of first-order approximation). The solutions are proposed in a very general framework, dealing with cell cultures (of bacteria, yeasts or animal cells) performed in stirred tank (continuous, semi-batch or batch) reactors, and without any a priori knowledge on the kinetics. The use of the estimators and their statistical properties are illustrated in a simulation case study (fed-batch bacterial cultures) and in a real case one (batch animal cell cultures).     

Maximum likelihood estimation for the three-parameter Weibull cdf of strength in presence of concurrent flaw populations

For a correct strength characterization of brittle materials, not only the maximum stress at fracture, but also the geometry of the specimens has to be considered thus taking into account the variable stress state and the size effect. Additionally, fracture may occur due to different fracture modes, as for example surface or edge defects. The authors propose a maximum likelihood estimator to obtain the cumulative distribution functions of strength for surface and edge flaw populations separately, both being three-parameter Weibull cdfs referred to an elemental surface area or elemental edge length, respectively. The method has been applied to simulated 3-point bending test data. The estimated Weibull parameters have been used to compute the cdfs of strength for specimens with different size, providing also the confidence bounds calculated by means of the bootstrap method. Finally, fracture data of 4-point bending tests on silicon carbide have been evaluated with the proposed method.     

Parameter estimation in models with hidden variables : An application to a biotech process

Biological processes are often characterised by significant nonlinearities, noisy measurements and hidden process variables. The dynamic behaviour of such processes can be represented by stochastic differential equations obtained from physical laws. We propose a Bayesian algorithm for parameter estimation in stochastic nonlinear biological processes with unmeasured (or hidden) variables. The proposed algorithm, involves drawing random samples iteratively from a posterior density functions of the parameters and the hidden variables. A Bayesian sampling techniques is used to approximate these posterior density functions. Both Metropolis–Hastings algorithm and Gibbs sampling are used for sample generation. The algorithm is extended to handle multiple data sets and missing observations. The algorithm is applied to an experimental data set collected from an algal bioreactor system. © 2011 Canadian Society for Chemical Engineering     

Likelihood functions for state space models with diffuse initial conditions

State space models with non‐stationary processes and/or fixed regression effects require a state vector with diffuse initial conditions. Different likelihood functions can be adopted for the estimation of parameters in time‐series models with diffuse initial conditions. In this article, we consider profile, diffuse and marginal likelihood functions. The marginal likelihood function is defined as the likelihood function of a transformation of the data vector. The transformation is not unique. The diffuse likelihood is a marginal likelihood for a data transformation that may depend on parameters. Therefore, the diffuse likelihood cannot be used generally for parameter estimation. The marginal likelihood function is based on an orthonormal data transformation that does not depend on parameters. Here we develop a marginal likelihood function for state space models that can be evaluated by the Kalman filter. The so‐called diffuse Kalman filter is designed for computing the diffuse likelihood function. We show that a minor modification of the diffuse Kalman filter is needed for the evaluation of our marginal likelihood function. Diffuse and marginal likelihood functions have better small sample properties compared with the profile likelihood function for the estimation of parameters in linear time series models. The results in our article confirm the earlier findings and show that the diffuse likelihood function is not appropriate for a range of state space model specifications.     

Regularized maximum likelihood estimation of sparse stochastic monomolecular biochemical reaction networks

A sparse parameter matrix estimation method is proposed for identifying a stochastic monomolecular biochemical reaction network system. Identification of a reaction network can be achieved by estimating a sparse parameter matrix containing the reaction network structure and kinetics information. Stochastic dynamics of a biochemical reaction network system is usually modeled by a chemical master equation (CME) describing the time evolution of probability distributions for all possible states. This paper considers closed monomolecular reaction systems for which an exact analytical solution of the corresponding chemical master equation can be derived. The estimation method presented in this paper incorporates the closed-form solution into a regularized maximum likelihood estimation (MLE) for which model complexity is penalized. A simulation result is provided to verify performance improvement of regularized MLE over least-square estimation (LSE), which is based on a deterministic mass-average model, in the case of a small population size.     

基于LSTM时间序列重建的生产装置异常检测

工业生产装置通常设置传感器报警阈值进行报警，但是对处于报警阈值以下的时间序列异常难以及时捕捉。基于统计的传统检测方法在解决时间序列异常检测上存在很大挑战，因此提出基于long short term memory (LSTM)时间序列重建的方法进行生产装置的异常检测。该算法首先引入一层LSTM网络对传感器数据的时间序列进行向量表示，采用另一层LSTM网络对时间序列进行逆序重建，然后利用重建值与实际值之间的误差，通过极大似然估计方法对该段序列进行异常概率估计，最终通过学习异常报警阈值实现时间序列异常检测。采用ECG测试数据、能源数据与危险品储罐传感器数据进行了仿真实验，验证了所提方法在不同长度的数据上的有效性。     

Empirical likelihood in long‐memory time series models

This article studies the empirical likelihood method for long‐memory time series models. By virtue of the Whittle likelihood, one obtains a score function that can be viewed as an estimating equation of the parameters of a fractional integrated autoregressive moving average (ARFIMA) model. This score function is used to obtain an empirical likelihood ratio which is shown to be asymptotically chi‐square distributed. Confidence regions for the parameters are constructed based on the asymptotic distribution of the empirical likelihood ratio. Bartlett correction and finite sample properties of the empirical likelihood confidence regions are examined.     

Parameter and state estimation in nonlinear stochastic continuous‐time dynamic models with unknown disturbance intensity

Approximate Maximum Likelihood Estimation (AMLE) is an algorithm for estimating the states and parameters of models described by stochastic differential equations (SDEs). In previous work (Varziri et al., Ind. Eng. Chem. Res., 47 (2), 380‐393, (2008); Varziri et al., Comp. Chem. Eng., in press), AMLE was developed for SDE systems in which process‐disturbance intensities and measurement‐noise variances were assumed to be known. In the current article, a new formulation of the AMLE objective function is proposed for the case in which measurement‐noise variance is available but the process‐disturbance intensity is not known a priori. The revised formulation provides estimates of the model parameters and disturbance intensities, as demonstrated using a nonlinear CSTR simulation study. Parameter confidence intervals are computed using theoretical linearization‐based expressions. The proposed method compares favourably with a Kalman‐filter‐based maximum likelihood method. The resulting parameter estimates and information about model mismatch will be useful to chemical engineers who use fundamental models for process monitoring and control.     

Frequency and phase estimation in time series with quasi periodic components

In this article, we consider frequency and phase estimation in a noisy oscillation with potentially non‐constant phase increments resulting from an underlying non‐constant frequency. A maximum periodogram method on segments is used to estimate the time‐varying frequency and a subsequent least squares approach to estimate the phase. A key problem addressed in this article is the question how to set up a meaningful concept of asymptotic statistics for this model. This problem is solved by a special infill asymptotics concept. We use this concept to prove consistency and asymptotic normality of the estimates. Furthermore, the phase estimate is compared to the Hilbert transform in a simulation.     

A mixed portmanteau test for ARMA‐GARCH models by the quasi‐maximum exponential likelihood estimation approach

This paper investigates the joint limiting distribution of the residual autocorrelation functions and the absolute residual autocorrelation functions of ARMA‐GARCH models. This leads a mixed portmanteau test for diagnostic checking of the ARMA‐GARCH model fitted by using the quasi‐maximum exponential likelihood estimation approach in Zhu and Ling (2011) . Simulation studies are carried out to examine our asymptotic theory, and assess the performance of this mixed test and other two portmanteau tests in Li and Li (2008) . A real example is given.     

Comparative study of estimation methods for continuous time stochastic processes

In this paper we investigate the finite sample performances of five estimation methods for a continuous-time stochastic process from discrete observations. Applying these methods to two examples of stochastic differential equations, one with linear drift and state-dependent diffusion coefficients and the other with nonlinear drift and constant diffusion coefficients, Monte Carlo experiments are carried out to evaluate the finite sample performance of each method. The Monte Carlo results indicate that the differences between the methods are large when the discrete- time interval is large. In addition, these differences are noticeable in estimations of the diffusion coefficients.     

Inference for asymmetric exponentially weighted moving average models

The exponentially weighted moving average (EWMA) model in ‘Risk-Metrics’ has been a benchmark for controlling and forecasting risks in financial operations. However, it is incapable of capturing the asymmetric volatility effect and the heavy-tailed innovation, which are two important stylized features of financial returns. We propose a new asymmetric EWMA model driven by the Student's t-distributed innovations to take these two stylized features into account and study its maximum likelihood estimation and model diagnostic checking. The finite-sample performance of the estimation and diagnostic test statistic is examined by the simulated data.     

Maximum likelihood estimation for nearly non‐stationary stable autoregressive processes

The maximum likelihood estimate (MLE) of the autoregressive coefficient of a near‐unit root autoregressive process Y_t = ρ_nY_t?1 + ?_t with α‐stable noise {?_t} is studied in this paper. Herein ρ_n = 1 ? γ/n, γ ≥ 0 is a constant, Y₀ is a fixed random variable and ε_t is an α‐stable random variable with characteristic function φ(t,θ) for some parameter θ. It is shown that when 0 < α < 1 or α > 1 and E?₁ = 0, the limit distribution of the MLE of ρ_n and θ are mixtures of a stable process and Gaussian processes. On the other hand, when α > 1 and E?₁ ≠ 0, the limit distribution of the MLE of ρ_n and θ are normal. A Monte Carlo simulation reveals that the MLE performs better than the usual least squares procedures, particularly for the case when the tail index α is less than 1.     

Testing for a Unit Root in Autoregressive Moving-average Models with Missing Data

Testing for a single autoregressive unit root in an autoregressive moving-average (ARMA) model is considered in the case when data contain missing values. The proposed test statistics are based on an ordinary least squares type estimator of the unit root parameter which is a simple approximation of the one-step Newton–Raphson estimator. The limiting distributions of the test statistics are the same as those of the regression statistics in AR(1) models tabulated by Dickey and Fuller (Distribution of the estimators for autoregressive time series with a unit root. J. Am. Stat. Assoc . 74 (1979), 427–31) for the complete data situation. The tests accommodate models with a fitted intercept and a fitted time trend.