Infinite horizon linear quadratic differential games for discrete-time stochastic systems

This paper deals with the infinite horizon linear quadratic(LQ)differential games for discrete-time stochastic systems with both state and control dependent noise.The Popov-Belevitch-Hautus(PBH)criteria for exact observability and exact detectability of discrete-time stochastic systems are presented.By means of them,we give the optimal strategies (Nash equilibrium strategies)and the optimal cost values for infinite horizon stochastic differential games.It indicates that the infinite horizon LQ stochastic differential games are associated with four coupled matrix-valued equations.Furthermore, an iterative algorithm is proposed to solve the four coupled equations.Finally,an example is given to demonstrate our results.     

Multi-player non-zero-sum games: Online adaptive learning solution of coupled Hamilton–Jacobi equations

In this paper we present an online adaptive control algorithm based on policy iteration reinforcement learning techniques to solve the continuous-time (CT) multi player non-zero-sum (NZS) game with infinite horizon for linear and nonlinear systems. NZS games allow for players to have a cooperative team component and an individual selfish component of strategy. The adaptive algorithm learns online the solution of coupled Riccati equations and coupled Hamilton–Jacobi equations for linear and nonlinear systems respectively. This adaptive control method finds in real-time approximations of the optimal value and the NZS Nash-equilibrium, while also guaranteeing closed-loop stability. The optimal-adaptive algorithm is implemented as a separate actor/critic parametric network approximator structure for every player, and involves simultaneous continuous-time adaptation of the actor/critic networks. A persistence of excitation condition is shown to guarantee convergence of every critic to the actual optimal value function for that player. A detailed mathematical analysis is done for 2-player NZS games. Novel tuning algorithms are given for the actor/critic networks. The convergence to the Nash equilibrium is proven and stability of the system is also guaranteed. This provides optimal adaptive control solutions for both non-zero-sum games and their special case, the zero-sum games. Simulation examples show the effectiveness of the new algorithm.     

Matrix games with missing,interval, and ambiguous lottery payoffs of pure strategy profiles and compound strategy profiles

In a matrix game, the interactions among players are based on the assumption that each player has accurate information about the payoffs of their interactions and the other players are rationally self‐interested. As a result, the players should definitely take Nash equilibrium strategies. However, in real‐life, when choosing their optimal strategies, sometimes the players have to face missing, imprecise (i.e., interval), ambiguous lottery payoffs of pure strategy profiles and even compound strategy profile, which means that it is hard to determine a Nash equilibrium. To address this issue, in this paper we introduce a new solution concept, called ambiguous Nash equilibrium, which extends the concept of Nash equilibrium to the one that can handle these types of ambiguous payoff. Moreover, we will reveal some properties of matrix games of this kind. In particular, we show that a Nash equilibrium is a special case of ambiguous Nash equilibrium if the players have accurate information of each player's payoff sets. Finally, we give an example to illustrate how our approach deals with real‐life game theory problems.     

Solutions for the Lurie-Postnikov and Aizerman problems

This paper deals with a class of many-person non-zero-sum differential games in which one player has the role of ‘ leader ’ while the others ‘ follow ’. Necessary conditions are obtained for the existence of open-loop Stackelberg solutions under the assumption that the followers respond to the leader by selecting Nash equilibrium controls. Some simple investment problems are described which give rise to discontinuous optimal controls for both leader and follower(s).     

Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation

In this paper we introduce a model of multigenerational stochastic games of capital accumulation where each generation consists of m different players. The main objective is to prove the existence of a perfect stationary equilibrium in an infinite horizon game. A suitable change in the terminology used in this paper provides (in the case of perfect altruism between generations) a new Nash equilibrium theorem for standard stochastic games with uncountable state space.     

Properties of Nash solutions of a two-stage nonzero-sum game

This paper contains exact expressions for the complete class of uncountably many globally optimal affine Nasb equilibrium strategies for a two-stage two-person nonzero-sum game problem with quadratic objective functionals and with dynamic information for beth players. Existence conditions for each of these Nash equilibrium solutions are derived and it is shown that a recursive Nash solution is not necessarily globally optimal. Cost-uniqueness property of the derived Nash strategies is investigated and it is proven that the game problem under consideration admits a unique Nash cost pair if and only if it can be made equivalent to either a team problem or a zero-sum game. It is also shown that existence conditions of a globally optimal Nash solution will be independent of the parameters characterizing the nonuniques of the Nash strategies only if the game problem can be made equivalent to a team problem.     

Iterative algorithms for computing the feedback Nash equilibrium point for positive systems

The paper studies N-player linear quadratic differential games on an infinite time horizon with deterministic feedback information structure. It introduces two iterative methods (the Newton method as well as its accelerated modification) in order to compute the stabilising solution of a set of generalised algebraic Riccati equations. The latter is related to the Nash equilibrium point of the considered game model. Moreover, we derive the sufficient conditions for convergence of the proposed methods. Finally, we discuss two numerical examples so as to illustrate the performance of both of the algorithms.     

Strategic stability in linear-quadratic differential games with nontransferable payoffs

We address the problem of strategically supported cooperation for linear-quadratic differential games with nontransferable payoffs. As an optimality principle, we study Pareto-optimal solutions. It is assumed that players use a payoff distribution procedure guaranteeing individual rationality of a cooperative solution over the entire game horizon. We prove that under these conditions a Pareto-optimal solution can be strategically supported by an ε-Nash equilibrium. An example is considered.     

An evolutionary algorithmic approach to determine the Nash equilibrium in a duopoly with nonlinearities and constraints

This paper presents an algorithmic approach to obtain the Nash equilibrium in a duopoly. Analytical solutions to duopolistic competition draw on principles of game theory and require simplifying assumptions such as symmetrical payoff functions, linear demand and linear cost. Such assumptions can reduce the practical use of duopolistic models. In contrast, we use an evolutionary algorithmic approach (EAA) to determine the Nash equilibrium values. This approach has the advantage that it can deal with and find optimum values for duopolistic competition modelled using non-linear functions. In the paper we gradually build up the competitive situation by considering non-linear demand functions, non-linear cost functions, production and environmental constraints, and production in discrete bands. We employ particle swarm optimization with composite particles (PSOCP), a variant of particle swarm optimization, as the evolutionary algorithm. Through the paper we explicitly demonstrate how EAA can solve games with constrained payoff functions that cannot be dealt with by traditional analytical methods. We solve several benchmark problems from the literature and compare the results obtained from EAA with those obtained analytically, demonstrating the resilience and rigor of our EAA solution approach.     

线性Markov 切换系统的随机Nash 微分博弈及混合H2/H∞ 控制

研究线性Markov切换系统的随机Nash微分博弈问题。首先借助线性Markov切换系统随机最优控制的相关结果,得到了有限时域和无线时域Nash均衡解的存在条件等价于其相应微分(代数) Riccati方程存在解,并给出了最优解的显式形式;然后应用相应的微分博弈结果分析线性Markov切换系统的混合H2/H∞控制问题;最后通过数值算例验证了所提出方法的可行性。     

A discrete-time bioresource management problem with asymmetric players

This paper explores the discrete-time game-theoretic model associated with a bioresource management problem (fish catching). The game engages players (countries or fishing firms) that harvest fish stocks on the infinite horizon. The paper aims at defining the cooperative payoff under different discount factors of the players. We propose applying a Nash bargaining solution for constructing the cooperative strategies of the players. The analysis covers two bargaining schemes, namely, the one for the whole duration of the game and the recursive bargaining procedure.     

Pareto-optimal Nash equilibrium: Sufficient conditions and existence in mixed strategies

This paper considers the Nash equilibrium strategy profiles that are Pareto optimal with respect to the rest Nash equilibrium strategy profiles. The sufficient conditions for the existence of such pure strategy profiles are established. These conditions employ the Germeier convolutions of the payoff functions. For the non-cooperative games with compact strategy sets and continuous payoff functions, the existence of the Pareto optimal Nash equilibria in mixed strategies is proved.     

带Poisson跳的线性二次随机微分博弈及其在鲁棒控制中的应用

研究了一类带Poisson跳扩散过程的线性二次随机微分博弈,包括非零和博弈的Nash均衡策略与零和博弈的鞍点均衡策略问题．利用微分博弈的最大值原理,得到Nash均衡策略的存在条件等价于两个交叉耦合的矩阵Riccati方程存在解,鞍点均衡策略的存在条件等价于一个矩阵Riccati方程存在解的结论,并给出了均衡策略的显式表达及最优性能泛函值．最后,将所得结果应用于现代鲁棒控制中的随机H₂/H_∞控制与随机H_∞控制问题,得到了鲁棒控制策略的存在条件及显式表达,并验证所得结果在金融市场投资组合优化问题中的应用．     

The Hamilton-Jacobi-Bellman equation for a class of differential games with random duration

We consider the class of differential games with random duration. We show that a problem with random game duration can be reduced to a standard problem with an infinite time horizon. A Hamilton-Jacobi-Bellman-type equation is derived for finding optimal solutions in differential games with random duration. Results are illustrated by an example of a game-theoretic model of nonrenewable resource extraction. The problem is analyzed under the assumption of Weibull-distributed random terminal time of the game.     

Evolutionarily stable sets in quantum penny flip games

In game theory, an Evolutionarily Stable Set (ES set) is a set of Nash Equilibrium (NE) strategies that give the same payoffs. Similar to an Evolutionarily Stable Strategy (ES strategy), an ES set is also a strict NE. This work investigates the evolutionary stability of classical and quantum strategies in the quantum penny flip games. In particular, we developed an evolutionary game theory model to conduct a series of simulations where a population of mixed classical strategies from the ES set of the game were invaded by quantum strategies. We found that when only one of the two players’ mixed classical strategies were invaded, the results were different. In one case, due to the interference phenomenon of superposition, quantum strategies provided more payoff, hence successfully replaced the mixed classical strategies in the ES set. In the other case, the mixed classical strategies were able to sustain the invasion of quantum strategies and remained in the ES set. Moreover, when both players’ mixed classical strategies were invaded by quantum strategies, a new quantum ES set was emerged. The strategies in the quantum ES set give both players payoff 0, which is the same as the payoff of the strategies in the mixed classical ES set of this game.     

Two-person zero-sum Markov games: receding horizon approach

We consider a receding horizon approach as an approximate solution to two-person zero-sum Markov games with infinite horizon discounted cost and average cost criteria. We first present error bounds from the optimal equilibrium value of the game when both players take "correlated" receding horizon policies that are based on exact or approximate solutions of receding finite horizon subgames. Motivated by the worst-case optimal control of queueing systems by Altman, we then analyze error bounds when the minimizer plays the (approximate) receding horizon control and the maximizer plays the worst case policy. We finally discuss some state-space size independent methods to compute the value of the subgame approximately for the approximate receding horizon control, along with heuristic receding horizon policies for the minimizer.     

The Nash Equilibrium Point of Dynamic Games Using Evolutionary Algorithms in Linear Dynamics and Quadratic System

In this paper, examining some games, we show that classical techniques are not always effective for games with not many stages and players and it can’t be claimed that these techniques of solution always obtain the optimal and actual Nash equilibrium point. For solving these problems, two evolutionary algorithms are then presented based on the population to solve general dynamic games. The first algorithm is based on the genetic algorithm and we use genetic algorithms to model the players' learning process in several models and evaluate them in terms of their convergence to the Nash Equilibrium. in the second algorithm, a Particle Swarm Intelligence Optimization (PSO) technique is presented to accelerate solutions’ convergence. It is claimed that both techniques can find the actual Nash equilibrium point of the game keeping the problem’s generality and without imposing any limitation on it and without being caught by the local Nash equilibrium point. The results clearly show the benefits of the proposed approach in terms of both the quality of solutions and efficiency.     

Equilibria problems on games: Complexity versus succinctness

We study the computational complexity of problems involving equilibria in strategic games and in perfect information extensive games when the number of players is large. We consider, among others, the problems of deciding the existence of a pure Nash equilibrium in strategic games or deciding the existence of a pure Nash or a subgame perfect Nash equilibrium with a given payoff in finite perfect information extensive games. We address the fundamental question of how can we represent a game with a large number of players? We propose three ways of representing a game with different degrees of succinctness for the components of the game. For perfect information extensive games we show that when the number of moves of each player is large and the input game is represented succinctly these problems are PSPACE-complete. In contraposition, when the game is described explicitly by means of its associated tree all these problems are decidable in polynomial time. For strategic games we show that the complexity of deciding the existence of a pure Nash equilibrium depends on the succinctness of the game representation and then on the size of the action sets. In particular we show that it is NP-complete, when the number of players is large and the number of actions for each player is constant, and that the problem is -complete when the number of players is a constant and the size of the action sets is exponential in the size of the game representation. Again when the game is described explicitly the problem is decidable in polynomial time.     

Finite-dimensional nonlinear output feedback dynamic games andbounds for sector nonlinearities

In general, nonlinear output feedback dynamic games are infinite dimensional. The paper treats a class of minimax games when the nonlinearities enter the dynamics of the unobservable states. An information state approach is introduced to recast these games as one of full information in infinite dimensions. Explicit solutions of the first-order partial differential information state equation are derived in terms of a finite-number of sufficient statistics. When the nonlinearities are sector bounded, suboptimal finite-dimensional strategies are derived