Risk-sensitive filtering for jump Markov linear systems

In this paper, a risk-sensitive multiple-model filtering algorithm is derived using the reference probability methods. First, the approximation of the interacting multiple-model (IMM) algorithm is identified in the reference probability domain. Then, the same type of approximation is used to derive the finite-dimensional risk-sensitive filtering algorithm. The derived algorithm reduces to the IMM filter when the risk-sensitive parameter goes to zero and reduces to the risk-sensitive filter for linear Gauss-Markov systems when the number of models is unity. The algorithm performs better in a simulated uncertain parameter scenario than the IMM filter.     

Risk-sensitive decision-theoretic diagnosis

We consider the problem of determining the optimal sequence of tests for the discovery of a faulty component, where there is a random cost associated with testing a component. Our work is motivated by applications in telecommunications networks, e.g., location and isolation of faults (or intruders) in IP networks. A novel feature in our approach is that a risk-sensitive performance criterion is used in order to rank different competing schedules. Risk-sensitivity is incorporated through the use of an exponential utility function, and hence optimal schedules attain a trade-off between minimal expected costs and, e.g., a low variance about the achievable expected costs. We characterize optimal schedules both when the testing sequence is not subject to precedence constraints, and when it is subject to such constraints, given by an arbitrary partial order. For the case with precedence constraints, we show that our models can be analyzed via modular decompositions, as studied by Monma and Sidney (1987)     

Risk-sensitive dual control

In this paper, we develop new results concerning the risk-sensitive dual control problem for output feedback nonlinear systems, with unknown time-varying parameters. These results are not merely immediate specializations of known risk-sensitive control theory for nonlinear systems, but rather, are new formulations which are of interest in their own right. A dynamic programming equation solution is given to an optimal risk-sensitive dual control problem penalizing outputs, rather than the states, for a reasonably general class of nonlinear signal models. This equation, in contrast to earlier formulations in the literature, clearly shows the dual aspects of the risk-sensitive controller regarding control and estimation. The computational task to solve this equation, as has been seen for the risk-neutral dual control problem, suffers from the so-called ‘curse of dimensionality’. This motivates our study of the risk-sensitive version for a suboptimal risk-sensitive dual controller. Explicit controllers are derived for a minimum phase single-input, single-output auto-regressive model with exogenous input and unknown time-varying parameters. Also, simulation studies are carried out for an integrator with a time-varying gain. They show that the risk-sensitive suboptimal dual controller is more robust to uncertain noise environments compared with its risk-neutral counterpart. © 1997 by John Wiley & Sons, Ltd.     

Risk-sensitive optimal control of hidden Markov models: structuralresults

The authors consider a risk-sensitive optimal control problem for (finite state and action spaces) hidden Markov models (HMM). They present results of an investigation on the nature and structure of risk-sensitive controllers for HMM. Several general structural results are presented, as well as a particular case study of a popular benchmark problem. For the latter, they obtain structural results for the optimal risk-sensitive controller and compare it to that of the risk-neutral controller. Furthermore, they show that indeed the risk-sensitive controller and its corresponding information state converge to the known solutions for the risk-neutral situation as the risk factor goes to zero. They also study the infinite and general risk aversion cases     

Risk-sensitive control for a class of nonlinear systems with multiplicative noise

In this paper, we consider the problem of optimal control for a class of nonlinear stochastic systems with multiplicative noise. The nonlinearity consists of quadratic terms in the state and control variables. The optimality criteria are of a risk-sensitive and generalised risk-sensitive type. The optimal control is found in an explicit closed-form by the completion of squares and the change of measure methods. As applications, we outline two special cases of our results. We show that a subset of the class of models which we consider leads to a generalised quadratic–affine term structure model (QATSM) for interest rates. We also demonstrate how our results lead to generalisation of exponential utility as a criterion in optimal investment.     

Average cost criterion induced by the regular utility function for continuous-time Markov decision processes

In this paper we study the average cost criterion induced by the regular utility function (U-average cost criterion) for continuous-time Markov decision processes. This criterion is a generalization of the risk-sensitive average cost and expected average cost criteria. We first introduce an auxiliary risk-sensitive first passage optimization problem and obtain the properties of the corresponding optimal value function under the slight conditions. Then we show that the pair of the optimal value functions of the risk-sensitive average cost criterion and the risk-sensitive first passage criterion is a solution to the optimality equation of the risk-sensitive average cost criterion allowing the risk-sensitivity parameter to take any nonzero value. Moreover, we have that the optimal value function of the risk-sensitive average cost criterion is continuous with respect to the risk-sensitivity parameter. Finally, we give the connections between the U-average cost criterion and the average cost criteria induced by the identity function and the exponential utility function, and prove the existence of a U-average optimal deterministic stationary policy in the class of all randomized Markov policies.     

Discrete-Time Risk-Sensitive Filters with Non-Gaussian Initial Conditions and Their Ergodic Properties

In this paper, we study asymptotic stability properties of risk-sensitive filters with respect to their initial conditions. In particular, we consider a linear time-invariant systems with initial conditions that are not necessarily Gaussian. We show that in the case of Gaussian initial conditions, the optimal risk-sensitive filter asymptotically converges to a suboptimal filter initialized with an incorrect covariance matrix for the initial state vector in the mean square sense provided the incorrect initializing value for the covariance matrix results in a risk-sensitive filter that is asymptotically stable, that is, results in a solution for a Riccati equation that is asymptotically stabilizing. For non-Gaussian initial conditions, we derive the expression for the risk-sensitive filter in terms of a finite number of parameters. Under a boundedness assumption satisfied by the fourth order absolute moment of the initial state variable and a slow growth condition satisfied by a certain Radon-Nikodym derivative, we show that a suboptimal risk-sensitive filter initialized with Gaussian initial conditions asymptotically approaches the optimal risk-sensitive filter for non-Gaussian initial conditions in the mean square sense. Some examples are also given to substantiate our claims.     

Robust filtering of stochastic uncertain systems on an infinite time horizon

In this paper, we consider a robust filtering problem for continuous time stochastic uncertain systems.The uncertainty in the system is characterized in terms of an uncertain probability distribution on the noise input. This uncertainty is assumed to satisfy a certain relative entropy constraint. The solution to a specially parameterized risk-sensitive stochastic filtering problem is used to construct a filter for the original uncertain system which guarantees an optimal worst-case filtering error. The corresponding minimax optimal filter is obtained by solving a pair of algebraic Riccati equations.     

On asymptotic stability of continuous-time risk-sensitive filters with respect to initial conditions

In this paper, we consider the problem of risk-sensitive filtering for continuous-time stochastic linear Gaussian time-invariant systems. In particular, we address the problem of forgetting of initial conditions. Our results show that suboptimal risk-sensitive filters initialized with arbitrary Gaussian initial conditions asymptotically approach the optimal risk-sensitive filter for a linear Gaussian system with Gaussian but unknown initial conditions in the mean square sense at an exponential rate, provided the arbitrary initial covariance matrix results in a stabilizing solution of the (H_∞-like) Riccati equation associated with the risk-sensitive problem. More importantly, in the case of non-Gaussian initial conditions, a suboptimal risk-sensitive filter asymptotically approaches the optimal risk-sensitive filter in the mean square sense under a boundedness condition satisfied by the fourth order absolute moment of the initial non-Gaussian density and a slow growth condition satisfied by a certain Radon–Nikodym derivative.     

Robustness and risk-sensitive filtering

This paper gives a precise meaning to the robustness of risk-sensitive filters for problems in which one is uncertain as to the exact value of the probability model. It is shown that risk-sensitive estimators (including filters) enjoy an error bound which is the sum of two terms, the first of which coincides with an upper bound on the error that one would obtain if one knew exactly the underlying probability model, while the second term is a measure of the distance between the true and design probability models. The paper includes a discussion of several approaches to estimation, including H_∞ filtering     

Design of satisfaction output feedback controls for stochastic nonlinear systems under quadratic tracking risk-sensitive index

In this paper, the design problem of satisfaction output feedback controls for stochastic nonlinear systems in strict feedback form under long-term tracking risk-sensitive index is investigated. The index function adopted here is of quadratic form usually encountered in practice, rather than of quartic one used to beg the essential difficulty on controller design and performance analysis of the closed-loop systems. For any given risk-sensitive parameter and desired index value, by using the integrator backstepping method, an output feedback control is constructively designed so that the closed-loop system is bounded in probability and the risk-sensitive index is upper bounded by the desired value.     

Risk-sensitive control of Markov jump linear systems: Caveats and difficulties

In this technical note, we revisit the risk-sensitive optimal control problem for Markov jump linear systems (MJLSs). We first demonstrate the inherent difficulty in solving the risk-sensitive optimal control problem even if the system is linear and the cost function is quadratic. This is due to the nonlinear nature of the coupled set of Hamilton-Jacobi-Bellman (HJB) equations, stemming from the presence of the jump process. It thus follows that the standard quadratic form of the value function with a set of coupled Riccati differential equations cannot be a candidate solution to the coupled HJB equations. We subsequently show that there is no equivalence relationship between the problems of risk-sensitive control and H ^∞ control of MJLSs, which are shown to be equivalent in the absence of any jumps. Finally, we show that there does not exist a large deviation limit as well as a risk-neutral limit of the risk-sensitive optimal control problem due to the presence of a nonlinear coupling term in the HJB equations.

     

Finite-dimensional risk-sensitive filters and smoothers fordiscrete-time nonlinear systems

Finite-dimensional optimal risk-sensitive filters and smoothers are obtained for discrete-time nonlinear systems by adjusting the standard exponential of a quadratic risk-sensitive cost index to one involving the plant nonlinearity. It is seen that these filters and smoothers are the same as those for a fictitious linear plant with the exponential of squared estimation error as the corresponding risk-sensitive cost index. Such finite-dimensional filters do not exist for nonlinear systems in the case of minimum variance filtering and control     

Robust finite horizon minimax filtering for discrete-time stochastic uncertain systems

We study a finite-horizon robust minimax filtering problem for time-varying discrete-time stochastic uncertain systems. The uncertainty in the system is characterized by a set of probability measures under which the stochastic noises, driving the system, are defined. The optimal minimax filter has been found by applying techniques of risk-sensitive LQG control. The structure and properties of resulting filter are analyzed and compared to H_∞ and Kalman filters.     

Finite Horizon Minimax Optimal Control of Stochastic Partially Observed Time Varying Uncertain Systems

We consider a linear-quadratic problem of minimax optimal control for stochastic uncertain control systems with output measurement. The uncertainty in the system satisfies a stochastic integral quadratic constraint. To convert the constrained optimization problem into an unconstrained one, a special S-procedure is applied. The resulting unconstrained game-type optimization problem is then converted into a risk-sensitive stochastic control problem with an exponential-of-integral cost functional. This is achieved via a certain duality relation between stochastic dynamic games and risk-sensitive stochastic control. The solution of the risk-sensitive stochastic control problem in terms of a pair of differential matrix Riccati equations is then used to establish a minimax optimal control law for the original uncertain system with uncertainty subject to the stochastic integral quadratic constraint. Date received: May 13, 1997. Date revised: March 18, 1998.     

Partially observed non-linear risk-sensitive optimal stopping control for non-linear discrete-time systems

In this paper we introduce and solve the partially observed optimal stopping non-linear risk-sensitive stochastic control problem for discrete-time non-linear systems. The presented results are closely related to previous results for finite horizon partially observed risk-sensitive stochastic control problem. An information state approach is used and a new (three-way) separation principle established that leads to a forward dynamic programming equation and a backward dynamic programming inequality equation (both infinite dimensional). A verification theorem is given that establishes the optimal control and optimal stopping time. The risk-neutral optimal stopping stochastic control problem is also discussed.     

A risk-sensitive control dual approach to a large deviations control problem

We consider a control problem of an ergodic process where the objective is to maximize over long term the probability to overperform a given level. This is formulated as a large deviations control problem for which the standard dynamic programming methods may not be applied directly. We solve this problem by adopting a duality approach leading to a risk-sensitive ergodic control problem. In a continuous-time diffusion setting, we state a verification theorem in terms of partial differential equations for this dual problem. We then turn back to the primal problem by means of large deviations techniques. We derive the optimal rate function and nearly optimal controls for the large deviations optimization problem. Finally, explicit solutions are provided in a linear-quadratic case.     

Risk-sensitive control and dynamic games for partially observeddiscrete-time nonlinear systems

Solves a finite-horizon partially observed risk-sensitive stochastic optimal control problem for discrete-time nonlinear systems and obtains small noise and small risk limits. The small noise limit is interpreted as a deterministic partially observed dynamic game, and new insights into the optimal solution of such game problems are obtained. Both the risk-sensitive stochastic control problem and the deterministic dynamic game problem are solved using information states, dynamic programming, and associated separated policies. A certainty equivalence principle is also discussed. The authors' results have implications for the nonlinear robust stabilization problem. The small risk limit is a standard partially observed risk-neutral stochastic optimal control problem     

Existence and uniqueness of risk-sensitive estimates

Risk-sensitive criteria have been used to derive robust filters, identifiers, and controllers. The fundamental issues of existence and uniqueness of an estimate of a random variable given a random vector with respect to an order-(/spl lambda/, p) risk-sensitive criterion are studied in this note. More precisely, we prove the existence of a unique risk-sensitive estimate provided /spl lambda/>0 and p>1. For the remaining cases, a general existence result is not available at this time. We do, however, prove the existence in certain special cases. Moreover, we present examples with uncountably many optimal risk-sensitive estimates, i.e., exhibiting an extremely high level of nonuniqueness.