Auxiliary model-based least-squares identification methods for Hammerstein output-error systems

The difficulty in identification of a Hammerstein (a linear dynamical block following a memoryless nonlinear block) nonlinear output-error model is that the information vector in the identification model contains unknown variables—the noise-free (true) outputs of the system. In this paper, an auxiliary model-based least-squares identification algorithm is developed. The basic idea is to replace the unknown variables by the output of an auxiliary model. Convergence analysis of the algorithm indicates that the parameter estimation error consistently converges to zero under a generalized persistent excitation condition. The simulation results show the effectiveness of the proposed algorithms.     

On computing output-error estimates

In this note, we consider a system with rational transfer function from input to output where this transfer function haspparameters. It is also supposed that the input has a spectrum consisting ofpdistinct points (equivalently, we suppose that the design index =p/2). As proved in [2], this implies that the output-error estimator has the same efficiency as the prediction-error estimator. We present a very convenient algorithm for computing the output-error estimate of the transfer function from input to output.     

On the efficiency of output-error estimators

In this correspondence we consider a system with rational transfer function from input to output where the transfer function haspparameters. It is proved in [1] that when the input is a sum of sinusoids and has a two-sided line spectrum consisting ofpdistinct frequencies, then the output-error estimator and the prediction-error estimator are equally efficient. This result, which is at first sight a surprising one, is given an intuitively pleasing proof in this correspondence. This proof is based on the algorithm of [2].     

Neuro-fuzzy methods for nonlinear system identification

Most processes in industry are characterized by nonlinear and time-varying behavior. Nonlinear system identification is becoming an important tool which can be used to improve control performance and achieve robust fault-tolerant behavior. Among the different nonlinear identification techniques, methods based on neuro-fuzzy models are gradually becoming established not only in the academia but also in industrial applications. Neuro-fuzzy modeling can be regarded as a gray-box technique on the boundary between neural networks and qualitative fuzzy models. The tools for building neuro-fuzzy models are based on combinations of algorithms from the fields of neural networks, pattern recognition and regression analysis. In this paper, an overview of neuro-fuzzy modeling methods for nonlinear system identification is given, with an emphasis on the tradeoff between accuracy and interpretability.     

On sparsity-inducing methods in system identification and state estimation

The purpose of this article is to survey some sparsity-inducing methods in system identification and state estimation. Such methods can be divided into two main categories: methods inducing sparsity in the parameters and those sparsifying the prediction error. In the last class we discuss in particular the least absolute deviation estimator and its robustness properties with respect to sparse noise in both cases of univariate and multivariate measurements. We also discuss the application of sparsity-inducing methods to switched system identification and to state estimation for linear systems in the presence sparse and dense measurement noises. While the presentation focuses essentially on bridging some existing results, some technical refinements, and new features are also provided.     

Coupled stochastic gradient identification algorithms for multivariate output-error systems using the auxiliary model

This paper considers the gradient based identification problem of a multivariate output-error system. By using the auxiliary model identification idea and the coupling identification concept, an auxiliary model based stochastic gradient (AM-SG) algorithm and a coupled AM-SG algorithm are presented. The results indicate that the parameter estimation errors converge to zero under the persistent excitation conditions. The simulation examples confirm the theoretical results.     

Instrumental variable methods for closed-loop system identification

In this paper, several instrumental variable (IV) and instrumental variable-related methods for closed-loop system identification are considered and set in an extended IV framework. Extended IV methods require the appropriate choice of particular design variables, as the number and type of instrumental signals, data prefiltering and the choice of an appropriate norm of the extended IV-criterion. The optimal IV estimator achieves minimum variance, but requires the exact knowledge of the noise model. For the closed-loop situation several IV methods are put in an extended IV framework and characterized by different choices of design variables. Their variance properties are considered and illustrated with a simulation example.     

Subspace model identification Part 1. The output-error state-space model identification class of algorithms

In this paper, we present two novel algorithms to realize a finite dimensional, linear time-invariant state-space model from input-output data. The algorithms have a number of common features. They are classified as one of the subspace model identification schemes, in that a major part of the identification problem consists of calculating specially structured subspaces of spaces defined by the input-output data. This structure is then exploited in the calculation of a realization. Another common feature is their algorithmic organization: an RQ factorization followed by a singular value decomposition and the solution of an overdetermined set (or sets) of equations. The schemes assume that the underlying system has an output-error structure and that a measurable input sequence is available. The latter characteristic indicates that both schemes are versions of the MIMO Output-Error State Space model identification (MOESP) approach. The first algorithm is denoted in particular as the (elementary MOESP scheme). The subspace approximation step requires, in addition to input-output data, knowledge of a restricted set of Markov parameters. The second algorithm, referred to as the (ordinary MOESP scheme), solely relies on input-output data. A compact implementation is presented of both schemes. Although we restrict our presentation here to error-free input-output data, a framework is set up in an identification context. The identification aspects of the presented realization schemes are treated in the forthcoming Parts 2 and 3.     

Non-parametric methods of system identification

The non-parametric identification of systems in terms of unparametrized representations such as the impulse response and frequency response is considered. Basic approaches are outlined in a retrospective setting as are the relationships between non-parametric and parametric identification models. The article concludes with an assessment of non-parametric methods which is conducted in terms of typical industrial applications.     

Errors-in-variables methods in system identification

The paper gives a survey of errors-in-variables methods in system identification. Background and motivation are given, and examples illustrate why the identification problem can be difficult. Under general weak assumptions, the systems are not identifiable, but can be parameterized using one degree-of-freedom. Examples where identifiability is achieved under additional assumptions are also provided. A number of approaches for parameter estimation of errors-in-variables models are presented. The underlying assumptions and principles for each approach are highlighted.     

Subspace model identification Part 2. Analysis of the elementary output-error state-space model identification algorithm

The elementary MOESP algorithm presented in the first part of this series of papers is analysed in this paper. This is done in three different ways. First, we study the asymptotic properties of the estimated state-space model when only considering zero-mean white noise perturbations on the output sequence. It is shown that, in this case, the MOESPl implementation yields asymptotically unbiased estimates. An important constraint to this result is that the underlying system must have a finite impulse response and subsequently the size of the Hankel matrices, constructed from the input and output data at the beginning of the computations, depends on the number of non-zero Markov parameters. This analysis, however, leads to a second implementation of the elementary MOESP scheme, namely MOESP2. The latter implementation has the same asymptotic properties without the finite impulse response constraint. Secondly, we compare the MOESP2 algorithm with a classical state space model identification scheme. The latter scheme, referred to as the CLASSIC algorithm, is based on the Ho and Kalman realization scheme and estimated Markov parameters. The comparison is done by a sensitivity study, where the effect is studied of the errors on the data on the calculated column space of the shift-invariant subspace. This study demonstrates that the elementary MOESP2 scheme is more robust with respect to the errors considered than the CLASSIC algorithm. In the third part, the model reduction capabilities of the elementary MOESP schemes are analysed when the observations are error-free. We demonstrate in which sense the reduced order model is optimal when acquired with the MOESP schemes. The optimality is expressed by the difference between the 2-norm of the errors on the state (or output) sequence of the reduced-order model and the 2-norm of the matrix containing the rejected singular values being as small as possible. The insights obtained in these three parts are evaluated in a simulation study, and validated in this paper. They lead to the assertion that the MOESP2 implementation allows identification of a compact, low-dimensional, state-space model accurately describing the input -output behaviour of the system to be identified, while making use of ‘perturbed’ input-output data. This can be done efficiently.     

Nonparametric system identification by kernel methods

A new nonparametric estimate for nonlinear discrete-time dynamic systems is considered. The new algorithm is weakly consistent under a specific condition on the transition probability operator of a stationary Markov process. The estimate is applicable when a parametric state model of the system is difficult to choose.     

Finite sample properties of system identification methods

In this paper we study the quality of system identification models obtained using the standard quadratic prediction error criterion for a general linear model class. The main feature of our results is that they hold true for a finite data sample and they are not asymptotic. The main theorems bound the difference between the expected value of the identification criterion evaluated at the estimated parameters and at the optimal parameters. The bound depends naturally on the model and system order, the pole locations, and the noise variance, and it shows that although these variables often do not enter in asymptotic convergence results, they do play an important role when the data sample is finite.     

On efficient parametric identification methods for linear discrete stochastic systems

We construct a numerically stable algorithm (with respect to machine rounding errors) of adaptive Kalman filtering in order to solve the parametric identification problem for linear stationary stochastic discrete systems. We solve the problem in the state space. The proposed algorithm is formulated in terms of an orthogonal square-root covariance filter which lets us avoid a standard implementation of the Kalman filter.     

On least squares algorithms for system parameter identification

A new least squares solution for obtaining asymptotically unbiased and consistent estimates of unknown parameters in noisy linear systems is presented. The proposed algorithms are in many ways more advantageous than generalized least squares algorithm. Extensions to on-line and multivariable problems can be easily implemented. Examples are given to illustrate the performance of these new algorithms.     

A PWA model identification method based on optimal operating region partition with the output-error minimization for nonlinear systems

When piecewise affine (PWA) model-based control methods are applied to nonlinear systems, the first question is how to get sub-models and corresponding operating regions. Motivated by the fact that the operating region of each sub-model is an important component of a PWA model and the parameters of a sub-model are strongly coupled with the operating region, a new PWA model identification method based on optimal operating region partition with the output-error minimization for nonlinear systems is initiated. Firstly, construct local data sets from input-output data and get local models by using the least square (LS) method. Secondly, cluster local models according to the feature vectors and identify the parameter vectors of sub-models by weighted least squares (WLS) method. Thirdly, get the initial operating region partition by using a normalized exponential function, which is to partition the operating space completely. Finally, simultaneously determine the optimal parameter vectors of sub-models and the optimal operating region partition underlying the output-error minimization, which is executed by particle swarm optimization (PSO) algorithm. Simulation results demonstrate that the proposed method can improve model accuracy compared with two existing methods.     

On methods and tools for rigorous system design

International Journal on Software Tools for Technology Transfer - Full a posteriori verification of the correctness of modern software systems is practically infeasible due to the sheer complexity...     

A user perspective on errors-in-variables methods in system identification

The paper starts with a brief survey of errors-in-variables methods in system identification. Background and motivation are given, and it is illustrated why the identification problem can be difficult. Under general weak assumptions, the system is not identifiable, but can be parameterized using one degree of freedom. Examples where identifiability is achieved under additional assumptions are also provided. A number of approaches for parameter estimation of errors-in-variables models are reviewed. The underlying assumptions and principles for each approach are highlighted. The paper then continues by discussing from a user’s perspective on how to proceed when practical problems are handled.     

Frequency domain versus time domain methods in system identification

The basic techniques of time domain and frequency domain identification, including the maximum entropy methods, are outlined. Then connections and distinctions between the methods are explored. This includes the derivation of some analytic relationships together with a discussion of the restrictions inherent in choosing certain methods, and their ease of use in different experimental conditions. It is concluded that these are complementary rather than competing techniques.     

Compressive system identification: Sequential methods and entropy bounds

In the first part of this work, a novel Kalman filtering-based method is introduced for estimating the coefficients of sparse, or more broadly, compressible autoregressive models using fewer observations than normally required. By virtue of its (unscented) Kalman filter mechanism, the derived method essentially addresses the main difficulties attributed to the underlying estimation problem. In particular, it facilitates sequential processing of observations and is shown to attain a good recovery performance, particularly under substantial deviations from ideal conditions, those which are assumed to hold true by the theory of compressive sensing. In the remaining part of this paper we derive a few information-theoretic bounds pertaining to the problem at hand. The obtained bounds establish the relation between the complexity of the autoregressive process and the attainable estimation accuracy through the use of a novel measure of complexity. This measure is used in this work as a substitute to the generally incomputable restricted isometric property.