This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Guo, X. Search for Related Content

Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces

The School of Mathematics and Computational Science, Zhongshan University, Guangzhou 510275, People’s Republic of China
mcsgxp{at}mail.sysu.edu.cn

This paper deals with continuous-time Markov decision processes in Polish spaces, under an expected discounted reward criterion. The transition rates of underlying continuous-time jump Markov processes are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. We first give conditions on the controlled system’s primitive data. Under these conditions we prove that the transition functions of possibly nonhomogeneous continuous-time Markov processes are regular by using Feller’s construction approach to such transition functions. Then, under additional continuity and compactness conditions, we ensure the existence of optimal stationary policies by using the technique of extended infinitesimal operators associated with the transition functions, and also provide a recursive way to compute (or at least to approximate) the optimal reward values. Finally, we use examples to illustrate our results and the gap between our conditions and those in the previous literature.

Key Words: Q-process; general state space; discounted reward; optimal stationary policy
History: Received: March 24, 2005; revision received: February 12, 2006;