Maximal solution to algebraic Riccati equations linked to infinite Markov jump linear systems

In this paper, we deal with a perturbed algebraic Riccati equation in an infinite dimensional Banach space. Besides the interest in its own right, this class of equations appears, for instance, in the optimal control problem for infinite Markov jump linear systems (from now on iMJLS). Here, infinite or finite has to do with the state space of the Markov chain being infinite countable or finite (see Fragoso and Baczynski in SIAM J Control Optim 40(1):270–297, 2001). By using a certain concept of stochastic stability (a sort of L ₂-stability), we prove the existence (and uniqueness) of maximal solution for this class of equation and provide a tool to compute this solution recursively, based on an initial stabilizing controller. When we recast the problem in the finite setting (finite state space of the Markov chain), we recover the result of de Souza and Fragoso (Syst Control Lett 14:233–239, 1999) set to the Markovian jump scenario, now free from an inconvenient technical hypothesis used there, originally introduced in Wonham in (SIAM J Control 6(4):681–697). Research supported by grants CNPq 520367-97-9, 300662/2003-3 and 474653/2003-0, FAPERJ 171384/2002, PRONEX and IM-AGIMB.     

线性Markov 切换系统的随机Nash 微分博弈及混合H2/H∞ 控制

研究线性Markov切换系统的随机Nash微分博弈问题。首先借助线性Markov切换系统随机最优控制的相关结果,得到了有限时域和无线时域Nash均衡解的存在条件等价于其相应微分(代数) Riccati方程存在解,并给出了最优解的显式形式;然后应用相应的微分博弈结果分析线性Markov切换系统的混合H2/H∞控制问题;最后通过数值算例验证了所提出方法的可行性。     

Monte Carlo TD(λ)-methods for the optimal control of discrete-time Markovian jump linear systems

In this paper, we present an iterative technique based on Monte Carlo simulations for deriving the optimal control of the infinite horizon linear regulator problem of discrete-time Markovian jump linear systems for the case in which the transition probability matrix of the Markov chain is not known. We trace a parallel with the theory of TD(λ) algorithms for Markovian decision processes to develop a TD(λ) like algorithm for the optimal control associated to the maximal solution of a set of coupled algebraic Riccati equations (CARE). It is assumed that either there is a sample of past observations of the Markov chain that can be used for the iterative algorithm, or it can be generated through a computer program. Our proofs rely on the spectral radius of the closed loop operators associated to the mean square stability of the system being less than 1.     

Infinite horizon linear quadratic stochastic Nash differential games of Markov jump linear systems with its application

In this paper, we deal with the problem of stochastic Nash differential games of Markov jump linear systems governed by Itô-type equation. Combining the stochastic stabilizability with the stochastic systems, a necessary and sufficient condition for the existence of the Nash strategy is presented by means of a set of cross-coupled stochastic algebraic Riccati equations. Moreover, the stochastic H₂/H_∞ control for stochastic Markov jump linear systems is discussed as an immediate application and an illustrative example is presented.     

Discrete-time LQ-optimal control problems for infinite Markov jumpparameter systems

Optimal control problems for discrete-time linear systems subject to Markovian jumps in the parameters are considered for the case in which the Markov chain takes values in a countably infinite set. Two situations are considered: the noiseless case and the case in which an additive noise is appended to the model. The solution for these problems relies, in part, on the study of a countably infinite set of coupled algebraic Riccati equations (ICARE). Conditions for existence and uniqueness of a positive semidefinite solution to the ICARE are obtained via the extended concepts of stochastic stabilizability (SS) and stochastic detectability (SD), which turn out to be equivalent to the spectral radius of certain infinite dimensional linear operators in a Banach space being less than one. For the long-run average cost, SS and SD guarantee existence and uniqueness of a stationary measure and consequently existence of an optimal stationary control policy. Furthermore, an extension of a Lyapunov equation result is derived for the countably infinite Markov state-space case     

事件驱动的网络化系统最优控制

针对随机事件驱动的网络化控制系统, 研究其中的有限时域和无限时域内最优控制器的设计问题. 首先, 根据执行器介质访问机制将网络化控制系统建模为具有多个状态的马尔科夫跳变系统; 然后, 基于动态规划和马尔科夫跳变线性系统理论设计满足二次型性能指标的最优控制序列, 通过求解耦合黎卡提方程的镇定解, 给出最优控制律的计算方法, 使得网络化控制系统均方指数稳定; 最后, 通过仿真实验表明了所提出方法的有效性.

     

Maximal and Stabilizing Hermitian Solutions for Discrete-Time Coupled Algebraic Riccati Equations

Discrete-time coupled algebraic Riccati equations that arise in quadratic optimal control and H _∞-control of Markovian jump linear systems are considered. First, the equations that arise from the quadratic optimal control problem are studied. The matrix cost is only assumed to be hermitian. Conditions for the existence of the maximal hermitian solution are derived in terms of the concept of mean square stabilizability and a convex set not being empty. A connection with convex optimization is established, leading to a numerical algorithm. A necessary and sufficient condition for the existence of a stabilizing solution (in the mean square sense) is derived. Sufficient conditions in terms of the usual observability and detectability tests for linear systems are also obtained. Finally, the coupled algebraic Riccati equations that arise from the H _∞-control of discrete-time Markovian jump linear systems are analyzed. An algorithm for deriving a stabilizing solution, if it exists, is obtained. These results generalize and unify several previous ones presented in the literature of discrete-time coupled Riccati equations of Markovian jump linear systems. Date received: November 14, 1996. Date revised: January 12, 1999.     

自由终端随机最优调节器的注记

本文讨论了无限时间自由终端随机最优调节器问题和其相应的广义代数R iccati方程解之间的关系.具体而言,本文证明了无限时间自由终端随机最优调节器对应着广义代数R iccati方程的最小非负解,该最小解的核空间等于随机系统的精确不能观子空间.另外本文指出了以往文献中关于广义代数R iccati方程最大解存在性的一个证明错误,并对错误进行了分析.     

基于策略迭代算法的连续时间线性Markov跳变系统非零和微分反馈Nash控制

针对一类连续时间线性Markov跳变系统,本文提出了一种新的策略迭代算法用于求解系统的非零和微分反馈Nash控制问题.通过求解耦合的数值迭代解,以获得具有线性动力学特性和无限时域二次成本的双层非零和微分策略的Nash均衡解.在每一个策略层,采用策略迭代算法来计算与每一组给定的反馈控制策略相关联的最小无限时域值函数.然后,通过子系统分解将Markov跳变系统分解为N个并行的子系统,并将该算法应用于跳变系统.本文提出的策略迭代算法可以很容易求解非零和微分策略所对应的耦合代数Riccati方程,且对高维系统有效.最后通过仿真示例证明了本文设计方法的有效性和可行性.     

On the detectability and observability of discrete-time Markov jump linear systems

This paper presents a new detectability concept for discrete-time Markov jump linear systems with finite Markov state, which generalizes the MS-detectability concept found in the literature. The new sense of detectability can similarly assure that the solution of the coupled algebraic Riccati equation associated to the quadratic control problem is a stabilizing solution. In addition, the paper introduces a related observability concept that also generalizes previous concepts. A test for detectability based on a coupled matrix equation is derived from the definition, and a test for observability is presented, which can be performed in a finite number of steps. The results are illustrated by examples, including one that shows that a system may be detectable in the new sense but not in the MS sense.     

Optimal Estimation Equations for the State Vector of a Stochastic Bilinear System: Its Bilinear Approximation

A finite-dimensional approximation for the optimal filtrating equations of the class of Markov diffusion processes described by a bilinear stochastic system is derived. The solution of the stochastic system is expressed in terms of the Peano series and its formal algebraic representation. A finite system of equations for the approximate filter is derived as the optimal Stratonovich–Kushner filter for a system with finite Peano series.     

Robust stability,stabilization and ℋ∞ control of time‐delay systems with Markovian jump parameters

In this paper, the problems of stochastic stability and stabilization for a class of uncertain time‐delay systems with Markovian jump parameters are investigated. The jumping parameters are modelled as a continuous‐time, discrete‐state Markov process. The parametric uncertainties are assumed to be real, time‐varying and norm‐bounded that appear in the state, input and delayed‐state matrices. The time‐delay factor is constant and unknown with a known bound. Complete results for both delay‐independent and delay‐dependent stochastic stability criteria for the nominal and uncertain time‐delay jumping systems are developed. The control objective is to design a state feedback controller such that stochastic stability and a prescribed ?_∞‐performance are guaranteed. We establish that the control problem for the time‐delay Markovian jump systems with and without uncertain parameters can be essentially solved in terms of the solutions of a finite set of coupled algebraic Riccati inequalities or linear matrix inequalities. Extension of the developed results to the case of uncertain jumping rates is also provided. Copyright © 2003 John Wiley & Sons, Ltd.     

Continuity of the solution of the Riccati equations for continuoustime JLQP

In this paper the continuity of the solution of the differential and algebraic Riccati equations for a continuous-time, Markovian, jump linear quadratic control problem as a function of coefficients is verified. The assumptions for this are stochastic stabilizability and observability     

On the existence of maximal solution for generalized algebraic Riccati equations arising in stochastic control

Existence of maximal solution is proved for a generalized version of the well-known standard algebraic Riccati equations which arise in certain stochastic optimal control problems.     

Weak detectability and the linear-quadratic control problem of discrete-time Markov jump linear systems

The paper deals with the stochastic concepts of weak detectability and weak observability for Markov jump linear systems, which is an special class of composite linear systems. The concepts are explored here to strengthen the similarities with the corresponding concepts of deterministic detectability and observability. We introduce a collection of matrices, referred to as the observability matrices. We show that weak observability is equivalent to full rank of each matrix in the set of observability matrices. In addition, we present a stochastic counterpart of the well known result on the invariance of trajectories within non-observable subspaces. These characterizations allow us to clarify the relationship between weak detectability and mean square detectability and to provide a testable condition for weak detectability. Relying on the assumption of weak detectability, we develop a method for solving the linear quadratic problem that is based on iterations of uncoupled algebraic Riccati equations, which converges to the solution of the coupled algebraic Riccati equation if and only if the system is mean-square stabilizable. Numerical examples are included.     

带马尔科夫跳和乘积噪声的随机系统的最优控制

讨论了N个选手随机系统的最优控制问题. 设计了无限时间的带有马尔科夫跳和乘积噪声的随机系统的Pareto最优控制器. 应用推广的Lyapunov方法和解随机Riccati代数方程得到了系统的Pareto最优解, 证明了最优控制器是稳定的反馈控制器, 以及对应于最优控制器的反馈增益中的随机Riccati代数方程的解是最小解.     

Stabilizability and positiveness of solutions of the jump linear quadratic problem and the coupled algebraic Riccati equation

This note addresses the jump linear quadratic problem of Markov jump linear systems and the associated algebraic Riccati equation. Necessary and sufficient conditions for stability of the optimal control and positiveness of Riccati solutions are developed. We show that the concept of weak detectability is not only a sufficient condition for the finiteness of cost functional to imply stability of the associated trajectory, but also a necessary one. This, together with a characterization developed here for the kernel of the Riccati solution, allows us to show that the control solution stabilizes the system if and only if the system is weakly detectable, and that the Riccati solution is positive-definite if and only if the system is weakly observable. The connection between the algebraic Riccati equation and the control problem is made, as far as the minimal positive-semidefinite solution for the algebraic Riccati equation is identified with the optimal solution of the linear quadratic problem. Illustrative numerical examples and comparisons are included.     

CONTROL OF INTERCONNECTED JUMPING SYSTEMS: AN H∞ APPROACH

This paper investigates, by using an approach, the problems of stochastic stability and control for a class of interconnected systems with Markovian jumping parameters. Both cases of finite‐ and infinite‐horizon are studied. It is shown that the problems under consideration can be solved if a set of coupled differential or algebraic Riccati equations are solvable.     

Further Study on Networked Control Systems with Unreliable Communication Channels

This paper focuses on the fundamental problems of linear quadratic gaussian (LQG) control and stabilization problems for networked control systems (NCSs) with unreliable communication channels (UCCs) where packet dropout, input delay and observation delay occur. These basic issues have attracted extensive attentions due to broad applications. Our contributions are as follows. For the finite horizon case, without time-stamping technique, the optimal estimator is derived by using the novelty method of innovation sequences based on the delayed intermittent observations; A necessary and sufficient condition for the optimal control problem is presented on the basis of the solution to the forward and backward difference equations (FBDEs) and two coupled Riccati equations. For the infinite horizon case, it is shown that under certain assumption, the system can stay bounded in the mean square sense if and only if the algebraic Riccati equation admits the unique positive solution.

     

Optimum H designs under sampled state measurements

We present the complete solution to the H^∞-optimal control problem when only sampled values of the state are available. For linear time-varying systems the optimum controller is characterized in terms of the solution of a particular generalized Riccati-differential equation, with the optimum performance determined by the conjugate point conditions associated with a family of generalized Riccati differential equations. For the infinite-horizon time-invariant problem, however, the optimum controller is characterized in terms of the solution of a particular generalized algebraic Riccati equation, and the performance is determined in terms of the conjugate-point conditions of a single generalized Riccati equation, defined on the longest sampling interval. If the distribution of the sampling times is also taken as part of the general design, uniform sampling turns out to be optimal for the infinite horizon case, while for the finite horizon problem a nonuniform sampling generally leads to a better performance.