This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Feinberg, E. A. Articles by Lewis, M. E. Search for Related Content

Optimality Inequalities for Average Cost Markov Decision Processes and the Stochastic Cash Balance Problem

Department of Applied Mathematics & Statistics, State University of New York at Stony Brook, Stony Brook, New York 11794
School of Operations Research & Industrial Engineering, Cornell University, 226 Rhodes Hall, Ithaca, New York 14853
efeinberg{at}notes.cc.sunysb.edu, http://www.ams.sunysb.edu/feinberg/
mark.lewis{at}cornell.edu, http://www.orie.cornell.edu/orie/people/faculty/profile.cfm?netidd=mark.lewis

For general state and action space Markov decision processes, we present sufficient conditions for the existence of solutions of the average cost optimality inequalities. These conditions also imply the convergence of both the optimal discounted cost value function and policies to the corresponding objects for the average costs per unit time case. Inventory models are natural applications of our results. We describe structural properties of average cost optimal policies for the cash balance problem; an inventory control problem where the demand may be negative and the decision-maker can produce or scrap inventory. We also show the convergence of optimal thresholds in the finite horizon case to those under the expected discounted cost criterion and those under the expected discounted costs to those under the average costs per unit time criterion.

Key Words: Markov decision process; average cost per unit time; optimality inequality; optimal policy; inventory control
History: Received: April 27, 2006; revision received: September 7, 2006;

This article has been cited by other articles:

				H.-S. Ahn, M. Gumus, and P. Kaminsky Inventory, Discounts, and the Timing Effect MSOM, October 1, 2009; 11(4): 613 - 629. [Abstract] [PDF]