Automatic Goal Discovery in Subgoal Monte Carlo Tree Search

Dominik Jeurissen*, Mark H.M. Winands, Chiara F. Sironi, Diego Perez-Liebana

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

16 Downloads (Pure)


Monte Carlo Tree Search (MCTS) is a heuristic search algorithm that can play a wide range of games without requiring any domain-specific knowledge. However, MCTS tends to struggle in very complicated games due to an exponentially increasing branching factor. A promising solution for this problem is to focus the search only on a small fraction of states. Subgoal Monte Carlo Tree Search (S-MCTS) achieves this by using a predefined subgoal-predicate that detects promising states called subgoals. However, not only does this make S-MCTS domaindependent, but also it is often difficult to define a good predicate. In this paper, we propose using quality diversity (QD) algorithms to detect subgoals in real-time. Furthermore, we show how integrating QD-algorithms into S-MCTS significantly improves its performance in the Physical Travelling Salesmen Problem without requiring any domain-specific knowledge.
Original languageEnglish
Title of host publication2021 IEEE Conference on Games (CoG)
Number of pages4
ISBN (Electronic)978-1-6654-3886-5
ISBN (Print)978-1-6654-4608-2
Publication statusPublished - 17 Aug 2021
Event2021 IEEE Conference on Games - Online, IT University of Copenhagen, Copenhagen, Denmark
Duration: 17 Aug 202120 Aug 2021


Conference2021 IEEE Conference on Games
Internet address

Cite this