基于Monte Carlo 粒子滤波的POMDPs 在线算法

针对部分可观察马尔可夫决策过程(POMDPs)的信念状态空间是一个双指数规模问题,提出一种基于 Monte Carlo 粒子滤波的 POMDPs 在线算法.首先,分别采用粒子滤波和粒子映射更新和扩展信念状态,建立可达信念状态与或树;然后,采用分支界限裁剪方法对信念状态与或树进行裁剪,降低求解规模.实验结果表明,所提出算法具有较低的误差率和较快的收敛性,能够满足系统实时性的要求.

Online algorithm based on Monte Carlo particle filtering in POMDPs

In order to solve the double exponential size problem of belief states space in partially observable Markov decision
processes(POMDPs), an online algorithm based on Monte Carlo particle filtering(MCPF) is proposed. Firstly, the methods
of particle filtering and particle projection are used to update and expand the belief states respectively, and the and-or tree of
reachable belief states is built. Then, a branch-and-bound pruning method is proposed to prune the tree to reduce computation.
Finally, the experiment and simulation results show that the proposed algorithm has the effectiveness in retaining the quality
of the policies and reducing the cost of computing policies, so it can meet the requirement of a real-time system.