Monte-Carlo tree search enhancements for Havannah

Jan A. Stankiewicz, Mark H M Winands, Jos W H M Uiterwijk

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review

Abstract

This article shows how the performance of a Monte-Carlo Tree Search (MCTS) player for Havannah can be improved by guiding the search in the playout and selection steps of MCTS. To improve the playout step of the MCTS algorithm, we used two techniques to direct the simulations, Last-Good-Reply (LGR) and N-grams. Experiments reveal that LGR gives a significant improvement, although it depends on which LGR variant is used. Using N-grams to guide the playouts also achieves a significant increase in the winning percentage. Combining N-grams with LGR leads to a small additional improvement. To enhance the selection step of the MCTS algorithm, we initialize the visit and win counts of the new nodes based on pattern knowledge. By biasing the selection towards joint/neighbor moves, local connections, and edge/corner connections, a significant improvement in the performance is obtained. Experiments show that the best overall performance is obtained when combining the visit-and-win-count initialization with LGR and N-grams. In the best case, a winning percentage of 77.5% can be achieved against the default MCTS program. © 2012 Springer-Verlag.
Original languageEnglish
Title of host publicationAdvances in Computer Games
PublisherSpringer
Pages60-71
Number of pages12
ISBN (Print)9783642318658
DOIs
Publication statusPublished - 2012

Publication series

SeriesLecture Notes in Computer Science
Volume7168

Cite this