Proper Policies in InfiniteState Stochastic Shortest Path Problems
Abstract
We consider stochastic shortest path problems with infinite state and control spaces, a nonnegative cost per stage, and a termination state. We extend the notion of a proper policy, a policy that terminates within a finite expected number of steps, from the context of finite state space to the context of infinite state space. We consider the optimal cost function $J^*$, and the optimal cost function $\hat J$ over just the proper policies. We show that $J^*$ and $\hat J$ are the smallest and largest solutions of Bellman's equation, respectively, within a suitable class of Lyapounovlike functions. If the cost per stage is bounded, these functions are those that are bounded over the effective domain of $\hat J$. The standard value iteration algorithm may be attracted to either $J^*$ or $\hat J$, depending on the initial condition.
 Publication:

arXiv eprints
 Pub Date:
 November 2017
 arXiv:
 arXiv:1711.10129
 Bibcode:
 2017arXiv171110129B
 Keywords:

 Mathematics  Optimization and Control