Abstract
We consider discrete-time Markov decision processes in which the decision maker is interested in long but finite horizons. First we consider reachability objective: the decision maker's goal is to reach a specific target state with the highest possible probability. A strategy is said to overtake another strategy, if it gives a strictly higher probability of reaching the target state on all sufficiently large but finite horizons. We prove that there exists a pure stationary strategy that is not overtaken by any pure strategy nor by any stationary strategy, under some condition on the transition structure and respectively under genericity. A strategy that is not overtaken by any other strategy, called an overtaking optimal strategy, does not always exist. We provide sufficient conditions for its existence. Next we consider safety objective: the decision maker's goal is to avoid a specific state with the highest possible probability. We argue that the results proven for reachability objective extend to this model.
Original language | English |
---|---|
Pages (from-to) | 945-965 |
Number of pages | 21 |
Journal | Journal of Optimization Theory and Applications |
Volume | 185 |
Issue number | 3 |
Early online date | 18 May 2020 |
DOIs | |
Publication status | Published - Jun 2020 |
Keywords
- Markov decision process
- Reachability objective
- Safety objective
- Overtaking optimality
- Perron-Frobenius eigenvalue
- OPTIMALITY
- OVERTAKING