Discrete stop-or-go games

Dubins and Savage (How to gamble if you must: inequalities for stochastic processes, McGraw-Hill, New York, 1965) found an optimal strategy for limsup gambling problems in which a player has at most two choices at every state x at most one of which could differ from the point mass δ(x). Their result is extended here to a family of two-person, zero-sum stochastic games in which each player is similarly restricted. For these games we show that player 1 always has a pure optimal stationary strategy and that player 2 has a pure ϵ-optimal stationary strategy for every ϵ>0. However, player 2 has no optimal strategy in general. A generalization to n-person games is formulated and ϵ-equilibria are constructed.