The Optimality of Linear Estimation Algorithms in Minimax Identification

The optimality of linear estimates in minimax estimation of a stochastically uncertain vector in a linear observation model by mean-square criterion is studied. In the Gaussian case, a uniformly optimal linear estimate is shown to exist in the class of all unbiased estimates. Moreover, it is minimax in the class of all nonlinear estimates if the nonrandom parameters of the observation model are unbounded. If the a priori information on random parameters are given as constraints on the covariance matrix, linear estimates are shown to be minimax.     

LMI-based minimax estimation and filtering under unknown covariances

In this paper, we consider a minimax approach to the estimation and filtering problems in the stochastic framework, where covariances of the random factors are completely unknown. The term ‘random factors’ refers either to unknown parameters and measurement noise in the estimation problem or to disturbance process and the initial state of a linear discrete-time dynamic system in the filtering problem. We introduce a notion of the attenuation level of random factors as a performance measure for both a linear unbiased estimate and a filter. This is the worst-case variance of the estimation error normalised by the sum of variances of all random factors over all nonzero covariance matrices. It is shown that this performance measure is equal to the spectral norm of the ‘transfer matrix’ and therefore the minimax estimate and filter can be computed in terms of linear matrix inequalities (LMIs). Moreover, the explicit formulae for both the minimax estimate and the minimal value of the attenuation level are presented in the estimation problem. It turns out that the above attenuation level of random factors coincides with the attenuation level of deterministic factors that is the worst-case normalised squared Euclidian norm of the estimation error over all nonzero sample values of random factors. In addition, we demonstrate that the LMI technique can be applied to derive the optimal robust estimator and filter, when there is a priori information about convex polyhedral sets which unknown covariance matrices of random factors belong to. Two illustrative examples show advantages of the minimax approach proposed.     

Optimal estimation and filtration under unknown covariances of random factors

The general schemes of linear estimation and filtration were considered on assumption of the unknown covariance matrix of random factors such as unknown parameters, measurement errors, and initial and external perturbations. A new criterion was introduced for the quality of estimate or filter. It is the level of damping random perturbations which is defined by the maximal value over all covariance matrices of the root-mean-square error normalized by the sum of variances of all random factors. The level of damping random perturbations was shown to be equal to the square of the spectral norm of the matrix relating the error of estimation and the random factors, and the optimal estimate minimizing this criterion was established. In the problem of filtration, it was shown how the filter parameters that are optimal in the level of damping random perturbations are expressed in terms of the linear matrix inequalities.     

State estimation with probability constraints

This paper considers a state estimation problem for a discrete-time linear system driven by a Gaussian random process. The second order statistics of the input process and state initial condition are uncertain. However, the probability that the state and input satisfy linear constraints during the estimation interval is known. A minimax estimation problem is formulated to determine an estimator that minimises the worst-case mean square error criterion, over the uncertain second order statistics, subject to the probability constraints. It is shown that a solution to this constrained state estimation problem is given by a Kalman filter for appropriately chosen input and initial condition models. These models are obtained from a finite dimensional convex optimisation problem. The application of this estimator to an aircraft tracking problem quantifies the improvement in estimation accuracy obtained from the inclusion of probability constraints in the minimax formulation.     

The Wonham filter under uncertainty: A game–theoretic approach

The paper is devoted to a state filtering problem of Markov jump processes given the continuous and/or counting observations. All the transition intensity matrix, observation plan and counting intensity are parameterized by a random vector with uncertain distribution on a known support set.The estimation problem is formulated in minimax settings with a conditional optimality criterion. We reduce the initial minimax problem to a dual problem of constrained quadratic optimization. The corresponding numerical algorithm of minimax filtering is presented as well as its illustrative implementation in the monitoring of a TCP link status under uncertainty.     

Minimax a posteriori estimation in the hidden Markov models

Consideration was given to the minimax estimation in the observation system including a hidden Markov model for continuous and counting observations. The dynamic and observation equations depend on a random finite-dimensional parameter having an unknown distribution with the given support. The conditional expectation of the available observation of some generalized quadratic loss function was used as the risk function. Existence of the saddle point in the formulated minimax problem was proved, and the worst distribution and the minimax estimate as the solution of a simpler dual problem were characterized.     

Optimal and suboptimal solutions to stochastically uncertain problems of quintile optimization

In this paper, problems of stochastic optimization under incomplete information on distribution of random perturbations with the quintile and probability criteria are considered. The minimax approach is used when optimal solutions are chosen. Conditions for equivalency of direct and inverse problems of stochastic optimization under incomplete statistical information are studied. The solution method for statistically uncertain problems of optimization with the quintile criterion basing on the use of generalized confidence sets for statistically uncertain random quantities is proposed. The use of confidence sets for finding suboptimal solutions to the problem of stochastic optimization under incomplete information is considered. Examples of the application of obtained relations are represented.     

Minimax estimation in systems of observation with Markovian chains by integral criterion

A problem of estimation of states and parameters in stochastic dynamic systems of observation with discrete time containing a Markovian chain is studied. Matrices of transient probabilities and observation plans are random with unknown distribution with a given compact carrier. Observations, on the basis of which the estimation is made, are available at a fixed interval of time [0, T]. As a loss function, we have a conditional mathematical expectation with respect to the available observations of ℓ ₂-norm of the estimation error of a signal process on [0, T]. The problem is in constructing an estimate minimizing losses correspondent to the worst distribution of the pair “a matrix of transient probabilities—a matrix of observation plan” form a set of allowable distributions. For a correspondent minimax problem is demonstrated the existence of a saddle point and is obtained a form of the wanted minimax estimation. The applicability of the obtained results is illustrated by a numerical example of the estimation of a state of TCP under the conditions of uncertainty of communication channel parameters.     

Methods for minimax estimation under elementwise covariance uncertainty

We consider the minimax estimation problem in the linear regression model under elementwise constraints imposed on the covariance matrix of the random parameters vector. Minimax estimates are designed using several approaches to the numerical solution of the dual problem, namely, the semidefinite programming method, the conditional gradient method and its modification with the Lagrange multipliers and regularization. The efficiency of the suggested methods is illustrated by the example of path restoration for a maneuvering target with a statistically uncertain acceleration.     

Robust state estimation in uncertain systems: Combined detection-estimation with incremental MSE criterion

Minimax state estimation for uncertain systems is discussed. The conservative performance of the standard minimax estimator in the absence of an intelligent adversary is reduced by a combined detectorestimator structure and an incremental mean-squared error (IMSE) performance criterion. The optimal structure is defined for a wide class of linear and nonlinear systems whose uncertain parameters are elements of some known compact space and is also obtained for convex parameter spaces. Since the complete specification of the optimal estimator detector is problem dependent, a computational procedure is outlined. In an example, the resulting combined detector-estimator is shown to increase the estimation accuracy in the incremental minimax sense by a factor of two over the standard minimax estimator.     

Optimizing Estimation of a Statistically Undefined System

Consideration was given to the optimal choice of the parameters for the best estimation of the phase state of a linear system fallible to the action of the Gaussian perturbation with undefined covariances of increments. The matrices at system perturbation and those in the measurement equation are the parameters to be selected for the choice of the observer player. The undefined increment matrices are selected by the opponent player. Both parameters are limited by compact sets. The problem comes to a differential game for the Riccati equation with a performance criterion in the form of a matrix trace. In a special case, consideration was given to the problem with constant matrices. Used were the methods of minimax optimization, optimal control theory, and the theory of differential games. Examples were considered.     

Uncertain linear regression model and its application

Regression analysis is a statistical process for estimating the relationships among variables based on probability. Because not all the imprecise quantities can be described by random variables, it is necessary to investigate relationships between an uncertain variable and some other variables. In this paper, an uncertain linear regression model is established based on uncertainty theory. Then, the estimators of parameters are obtained in the proposed model by the empirical uncertainty distribution coming from experts’ experimental data. Finally, the uncertain linear regression model is applied to solve an estimate problem.     

Minimax control of a process in a linear uncertain-stochastic system with incomplete data

Consideration is given to the control problem in a linear stochastic differential system where constant noise intensities in equations of state and observation are prescribed only accurate within the membership of some known sets. For control optimization, an integral root-mean-square performance criterion is used. The problem is solved by the transition to a dual one, which makes it possible to prove the existence of a saddle point of the criterion and obtain an explicit expression for the minimax control operator as functions of the solution to the dual problem. To solve the latter, an iteration algorithm is proposed; the convergence of the algorithm is proved and investigated by a model example.     

Reliable Minimax Parameter Estimation

This paper deals with the minimax parameter estimation of nonlinear parametric models from experimental data. Taking advantage of the special structure of the minimax problem, a new efficient and reliable algorithm based on interval constraint propagation is proposed. As an illustration, the ill-conditioned problem of estimating the parameters of a two-exponential model is considered.     

Noncausal minimax linear state estimation for systems with uncertain second-order statistics

This note considers the problem of minimax state estimation of the states of a linear time-invariant system which is driven by and observed in the presence of noise processes with uncertain second-order statistics. When the process noise and observations are scalars, the problem is shown to be equivalent to a scalar minimax estimation problem. The existence of a minimax solution is thereby established, and the minimax filter is shown to be a linear transformation of the minimax filter for the scalar problem.     

Experiment design in guaranteed identification

Consideration was given to the optimal choice of inputs at identification of the control system parameters from the results of measurements under the assumption that the a priori information about the uncertain parameters and measurement errors is confined to the admissible limits of their variations. The problem of identification is considered within the framework of the minimax (guaranteed) approach; optimization of input is oriented to improving the accuracy of estimation of the uncertain system parameters. The integral of the system information (function) is used as a criterion characterizing the quality of estimation.     

Structural parameter estimation in power systems

A new class of algorithms for the estimation of structural parameters of a continuous-time linear system excited by random natural disturbances is presented in the paper. All these algorithms are based on fitting the autocorrelation function of the system output; differences among them arise from the various possible formulations of the fit-criterion. Thus, the asymptotic statistical properties of the estimate are analyzed in order to have a choice tool among the class of algorithms and to compare them with other existing estimation methods. A further relevant subject is the statement of a robust test to verify the correctness of the tentative model assumed for the sake of the estimation procedure. Then the above algorithm is applied to the problem of estimating structural parameters (i.e. natural frequencies and damping factors) of the Italian and Yugoslavian power systems by recording some main electrical quantities during the normal operation of the system.Capability of dealing with structural systems affected by an unknown number of oscillatory modes and simplicity of use by non-statistical people are interesting features of the present approach, emphasized by the application.     

Parameter learning for the nonlinear system described by Hammerstein model with output disturbance

A novel parameter learning scheme using multi-signal processing is developed that aims at estimating parameters of the Hammerstein nonlinear model with output disturbance in this paper. The Hammerstein nonlinear model consists of a static nonlinear block and a dynamic linear block, and the multi-signals are devised to estimate separately the nonlinear block parameters and the linear block parameters; the parameter estimation procedure is greatly simplified. Firstly, in view of the input–output data of separable signals, the linear block parameters are computed through correlation analysis method, thereby the influence of output noise is effectively handled. In addition, model error probability density function technology is employed to estimate the nonlinear block parameters with the help of measurable input–output data of random signals, which not only controls the space state distribution of model error but also makes error distribution tends to normal distribution. The simulation results demonstrate that the developed approach obtains high learning accuracy and small modeling error, which verifies the effectiveness of the developed approach.     

Optimal linear granulometric estimation for random sets

This paper addresses two pattern-recognition problems in the context of random sets. For the first, the random set law is known and the task is to estimate the observed pattern from a feature set calculated from the observation. For the second, the law is unknown and we wish to estimate the parameters of the law. Estimation is accomplished by an optimal linear system whose inputs are features based on morphological granulometries. In the first case these features are granulometric moments; in the second they are moments of the granulometric moments. For the latter, estimation is placed in a Bayesian context by assuming that there exists a prior distribution for the parameters determining the law. A disjoint random grain model is assumed and the optimal linear estimator is determined by using asymptotic expressions for the moments of the granulometric moments. In both cases, the linear approach serves as a practical alternative to previously proposed nonlinear methods. Granulometric pattern estimation has previously been accomplished by a nonlinear method using full distributional knowledge of the random variables determining the pattern and granulometric features. Granulometric estimation of the law of a random grain model has previously been accomplished by solving a system of nonlinear equations resulting from the granulometric asymptotic mixing theorem. Both methods are limited in application owing to the necessity of performing a nonlinear optimization. The new linear method avoids this. It makes estimation possible for more complex models.     

Approximation of linear constant systems

In this paper two problems are considered, the problem of modeling a given constant linear system by a constant linear system of fixed lower order, and the problem of finding a filter of fixed order to estimate a time-invariant random process from a related time-invariant random process. A quadratic criterion is used to select the optimum system in both cases. It is shown that the filtering problem reduces to the problem of modeling the corresponding Wiener filter. Necessary conditions for a solution are developed and stated in terms of standard Wiener filter theory notation. Numerical solution of the equations embodying the necessary conditions is considered and several examples are presented.