A two-level hierarchical Markov decision model with considering interaction between levels

Decision in reality often have the characteristic of hierarchy because of the hierarchy of an organization’s structure. In this paper, we propose a two-level hierarchic Markov decision model that considers the interactions of agents in different levels and different time scales of levels. A backward induction algorithm is given for the model to solve the optimal policy of finite stage hierarchic decision problem. The proposed model and its algorithm are illustrated with an example about two-level hierarchical decision problem of infrastructure maintenance. The optimal policy of the example is solved and the impacts of interactions between levels on decision making are analyzed.