Characterizing all optimal controls for an indefinite stochasticlinear quadratic control problem

This paper is concerned with a stochastic linear quadratic (LQ) control problem in the infinite-time horizon, with indefinite state and control weighting matrices in the cost function. It is shown that the solvability of this problem is equivalent to the existence of a so-called static stabilizing solution to a generalized algebraic Riccati equation. Moreover, another algebraic Riccati equation is introduced and all the possible optimal controls, including the ones in state feedback form, of the underlying LQ problem are explicitly obtained in terms of the two Riccati equations     

Robust stabilization of stochastic systems based on the LQ controller

The robust exponential stability in mean square for a class of linear stochastic uncertain control systems is dealt with. For the uncertain stochastic systems, we have designed an optimal controller which guarantees the exponential stability of the system. Actually, we employed Lyapunov fimction approach and the stochastic algebraic Riccati equation (SARE) to have shown the robusmess of the linear quadratic(LQ) optimal control law.And the algebraic criteria for the exponential stability on the linear stochastic uncertain closed-loop systems are given.     

Infinite horizon indefinite stochastic linear quadratic control for discrete-time systems

This paper discusses discrete-time stochastic linear quadratic (LQ) problem in the infinite horizon with state and control dependent noise, where the weighting matrices in the cost function are assumed to be indefinite. The problem gives rise to a generalized algebraic Riccati equation (GARE) that involves equality and inequality constraints. The well-posedness of the indefinite LQ problem is shown to be equivalent to the feasibility of a linear matrix inequality (LMI). Moreover, the existence of a stabilizing solution to the GARE is equivalent to the attainability of the LQ problem. All the optimal controls are obtained in terms of the solution to the GARE. Finally, we give an LMI -based approach to solve the GARE via a semidefinite programming.     

Robust stabilization of stochastic systems based on the LQ cont roller

The robust exponential stability in mean square for a class of linear stochastic uncertain control systems is dealt with. For the uncertain stochastic systems ,we have designed an optimal controller which guarantees the exponential stability of the system. Actually ,we employed Lyapunov function approach and the stochastic algebraic Riccati equation (SARE) to have shown the robustness of the linear quadratic (LQ) optimal control law. And the algebraic criteria for the exponential stability on the linear stochastic uncertain closed- loop systems are given.     

Well-posedness and attainability of indefinite stochastic linear quadratic control in infinite time horizon

This paper is concerned with a stochastic linear-quadratic (LQ) problem in an infinite time horizon with multiplicative noises both in the state and the control. A distinctive feature of the problem under consideration is that the cost weighting matrices for the state and the control are allowed to be indefinite. A new type of algebraic Riccati equation – called a generalized algebraic Riccati equation (GARE) – is introduced which involves a matrix pseudo-inverse and two additional algebraic equality/inequality constraints. It is then shown that the well-posedness of the indefinite LQ problem is equivalent to a linear matrix inequality (LMI) condition, whereas the attainability of the LQ problem is equivalent to the existence of a “stabilizing solution” to the GARE. Moreover, all possible optimal controls are identified via the solution to the GARE. Finally, it is proved that the solution to the GARE can be obtained via solving a convex optimization problem called semidefinite programming.     

基于策略迭代的连续时间系统的随机线性二次最优控制

针对模型参数部分未知的随机线性连续时间系统, 通过策略迭代算法求解无限时间随机线性二次(LQ) 最优控制问题. 求解随机LQ最优控制问题等价于求随机代数Riccati 方程(SARE) 的解. 首先利用伊藤公式将随机微分方程转化为确定性方程, 通过策略迭代算法给出SARE 的解序列; 然后证明SARE 的解序列收敛到SARE 的解, 而且在迭代过程中系统是均方可镇定的; 最后通过仿真例子表明策略迭代算法的可行性.

     

Solvability of indefinite stochastic Riccati equations and linear quadratic optimal control problems

A new approach to study the indefinite stochastic linear quadratic (LQ) optimal control problems, which we called the “equivalent cost functional method”, is introduced by Yu (2013) in the setup of Hamiltonian system. On the other hand, another important issue along this research direction, is the possible state feedback representation of optimal control and the solvability of associated indefinite stochastic Riccati equations. As the response, this paper continues to develop the equivalent cost functional method by extending it to the Riccati equation setup. Our analysis is featured by its introduction of some equivalent cost functionals which enable us to have the bridge between the indefinite and positive-definite stochastic LQ problems. With such bridge, some solvability relation between the indefinite and positive-definite Riccati equations is further characterized. It is remarkable the solvability of the former is rather complicated than the latter, hence our relation provides some alternative but useful viewpoint. Consequently, the corresponding indefinite linear quadratic problem is discussed for which the unique optimal control is derived in terms of state feedback via the solution of the Riccati equation. In addition, some example is studied using our theoretical results.     

具有加性和乘性噪声的线性离散时间随机系统的无模型最优跟踪控制

本文研究一类同时受加性和乘性噪声影响的离散时间随机系统的最优跟踪控制问题.通过构造由原始系统和参考轨迹组成的增广系统,将随机线性二次跟踪控制(SLQT)的成本函数转化为与增广状态相关的二次型函数,由此推导出用于求解SLQT的贝尔曼方程和增广随机代数黎卡提方程(SARE),而后进一步针对系统和参考轨迹动力学信息完全未知的情形,提出一种Q-学习算法来在线求解增广SARE,证明了该算法的收敛性,并采用批处理最小二乘法(BLS)解决该在线无模型控制算法的实现问题.通过对单相电压源UPS逆变器的仿真,验证了所提出控制方案的有效性.     

Indefinite LQ optimal control for stochastic Takagi–Sugeno fuzzy system under sensor data scheduling: Finite-horizon case

The present paper considers the finite-horizon indefinite linear quadratic (LQ) control problem for stochastic Takagi–Sugeno (T-S) fuzzy systems with input delay. In this paper, we consider the presence of sensor data scheduling, which imposes a communication energy constraint and necessitates optimal state estimation for measurements. Then, by utilizing dynamic programming principles, the stochastic LQ problem under consideration can be solved, while the optimal control policy is developed in terms of the unique solutions to a set of coupled difference Riccati equations (CDREs). Specifically, for simple delay-free case, the linear matrix inequalities based conditions are also proposed, whose feasibility is shown to be equivalent to the well-posedness of the indefinite LQ control under consideration. As an application, our theoretic analysis is extended to study the intermittent observation model caused by random denial-of-service attack.     

具有参数不确定性和外干扰系统的鲁棒H-状态反馈控制

本文考虑具有参数不确定性和外干扰系统的鲁棒Ｈ∞状态反馈控制问题。这个问题的解只需求解一个代数Ｒｉｃｃａｔｉ方程就可得到其状态反馈阵，且这个状态反馈阵仍保持和ＬＱ最优设计相同的形式。运用这样的状态反馈控制。即能保证具有参数不确定性系统是稳定的，又能达到Ｈ∞最优干扰抑制效果。     

线性离散奇异系统的不定二次最优控制问题: Krein空间方法

The finite time horizon indefinite linear quadratic(LQ) optimal control problem for singular linear discrete time-varying systems is discussed. Indefinite LQ optimal control problem for singular systems can be transformed to that for standard state-space systems under a reasonable assumption. It is shown that the indefinite LQ optimal control problem is dual to that of projection for backward stochastic systems. Thus, the optimal LQ controller can be obtained by computing the gain matrices of Kalman filter. Necessary and sufficient conditions guaranteeing a unique solution for the indefinite LQ problem are given. An explicit solution for the problem is obtained in terms of the solution of Riccati difference equations.     

一类双线性随机系统M量测不定LQ控制

研究了带有乘性噪声和受扰动观测的离散时间随机系统不定线性二次(Linear quadratic, LQ) 最优输出反馈控制问题. 对此类问题而言,二次成本函数的加权矩阵不定号,并且最优控制具有对偶效果.为在最优性和计算复杂度间 进行折衷,本文采用了一种M量测反馈控制设计方法.基于动态规划方法,将未来的测量结合到当前控制 计算当中的M量测反馈控制可以通过倒向求解一类与原系统维数相同的广义差分Riccati方程(Generalized difference Riccati equation, GDRE)得到.仿真结果 表明本文提出的算法与目前普遍采用的确定等价性方法相比具有优越性.     

Continuous-time singular linear–quadratic control: Necessary and sufficient conditions for the existence of regular solutions

The purpose of this paper is to provide a full understanding of the role that the constrained generalized continuous algebraic Riccati equation plays in singular linear–quadratic (LQ) optimal control. Indeed, in spite of the vast literature on LQ problems, only recently a sufficient condition for the existence of a non-impulsive optimal control has for the first time connected this equation with the singular LQ optimal control problem. In this paper, we establish four equivalent conditions providing a complete picture that connects the singular LQ problem with the constrained generalized continuous algebraic Riccati equation and with the geometric properties of the underlying system.     

Infinite time horizon nonzero-sum linear quadratic stochastic differential games with state and control-dependent noise

This paper discusses the infinite time horizon nonzero-sum linear quadratic （LQ） differential games of stochastic systems governed by Itoe＇s equation with state and control-dependent noise. First, the nonzero-sum LQ differential games are formulated by applying the results of stochastic LQ problems. Second, under the assumption of mean-square stabilizability of stochastic systems, necessary and sufficient conditions for the existence of the Nash strategy are presented by means of four coupled stochastic algebraic Riccati equations. Moreover, in order to demonstrate the usefulness of the obtained results, the stochastic H-two/H-infinity control with state, control and external disturbance-dependent noise is discussed as an immediate application.     

An equivalence result in linear-quadratic theory

We consider the zero-endpoint infinite-horizon LQ problem. We show that the existence of an optimal policy in the class of feedback controls is a sufficient condition for the existence of a stabilizing solution to the algebraic Riccati equation. This result is shown without assuming positive definiteness of the state weighting matrix. The feedback formulation of the optimization problem is natural in the context of differential games and we provide a characterization of feedback Nash equilibria both in a deterministic and stochastic context.     

Data-driven policy iteration algorithm for continuous-time stochastic linear-quadratic optimal control problems

This paper studies a continuous-time stochastic linear-quadratic (SLQ) optimal control problem on infinite-horizon. Combining the Kronecker product theory with an existing policy iteration algorithm, a data-driven policy iteration algorithm is proposed to solve the problem. In contrast to most existing methods that need all information of system coefficients, the proposed algorithm eliminates the requirement of three system matrices by utilizing data of a stochastic system. More specifically, this algorithm uses the collected data to iteratively approximate the optimal control and a solution of the stochastic algebraic Riccati equation (SARE) corresponding to the SLQ optimal control problem. The convergence analysis of the obtained algorithm is given rigorously, and a simulation example is provided to illustrate the effectiveness and applicability of the algorithm.     

Solvability and asymptotic behavior of generalized Riccatiequations arising in indefinite stochastic LQ controls

The optimal control problem in a finite time horizon with an indefinite quadratic cost function for a linear system subject to multiplicative noise on both the state and control can be solved via a constrained matrix differential Riccati equation. In this paper, we provide general necessary and sufficient conditions for the solvability of this generalized differential Riccati equation. Furthermore, its asymptotic behavior is investigated along with its connection to the generalized algebraic Riccati equation associated with the linear quadratic control problem in finite time horizon. Examples are presented to illustrate the results established     

离散时间不定号随机线性二次型最优控制:无限时区情形

应用渐近分析方法讨论了无限时区离散时间不定号随机线性二次型最优控制问题. 所进行的研究是建立在这一问题有限时区情形结果和系统均方能镇定假设基础之上的. 广义代数Riccati方程(GARE)解的一些性质也得到了考虑. 最后提供了两个例子来说明所推出的结果是有效的.     

Employing the algebraic Riccati equation for a parametrization of the solutions of the finite-horizon LQ problem: the discrete-time case

In this paper, a new methodology is developed for the closed-form solution of a generalized version of the finite-horizon linear-quadratic regulator problem for LTI discrete-time systems. The problem considered herein encompasses the classical version of the LQ problem with assigned initial state and weighted terminal state, as well as the so-called fixed-end point version, in which both the initial and the terminal states are sharply assigned. The present approach is based on a parametrization of all the solutions of the extended symplectic system. In this way, closed-form expressions for the optimal state trajectory and control law may be determined in terms of the boundary conditions. By taking advantage of standard software routines for the solution of the algebraic Riccati and Stein equations, our results lead to a simple and computationally attractive approach for the solution of the considered optimal control problem without the need of iterating the Riccati difference equation.