This Article Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Federgruen, A. Articles by Schweitzer, P. J. Search for Related Content

Successive Approximation Methods for Solving Nested Functional Equations in Markov Decision Problems

Graduate School of Business, Columbia University, New York, New York 10027
Graduate School of Business, University of Rochester, Rochester, New York 14627

This paper presents a successive approximation method for solving systems of nested functional equations which arise, e.g., when considering Markov renewal programs in which policies that are maximal gain or optimal under more selective discount—and average overtaking optimality criteria are to be found. In particular, a successive approximation method is given to find the optimal bias vector and bias-optimal policies. Applications with respect to a number of additional stochastic control models are pointed out.

Our method is based on systems of simultaneously generated (single-equation) value-iteration schemes.

Key Words: nested functional equations; Markov renewal programs; undiscounted value-iteration; geometric convergence rates; sensitive discounted and average overtaking optimality criteria