RSPSA: Enhanced Parameter Optimisation in Games

L. Kocsis, C.S. Szepesvari, M.H.M. Winands

Research output: Chapter in Book/Report/Conference proceedingChapterAcademic

Abstract

Most game programs have a large number of parameters that are crucial for their performance. Tuning these parameters by hand is rather difficult. Therefore automatic optimization algorithms in game programs are interesting research domains. However, successful applications are only known for parameters that belong to certain components (e.g., evaluation-function parameters). The spsa (simultaneous perturbation stochastic approximation) algorithm is an attractive choice for optimizing any kind of parameters of a game program, both for its generality and its simplicity. Its disadvantage is that it can be very slow.in this article we propose several methods to speed up spsa, in particular, the combination with rprop, using common random numbers, antithetic variables, and averaging. We test the resulting algorithm for tuning various types of parameters in two domains, poker and loa. From the experimental study, we may conclude that using spsa is a viable approach for optimization in game programs, in particular if no good alternative exists for the types of parameters considered.
Original languageEnglish
Title of host publicationAdvances in Computer Games. ACG 2005.
EditorsH.J. van den Herik, S.C. Hsu, H.H.L.M. Donkers
Place of PublicationHeidelberg
PublisherSpringer
Pages39-56
Volume4250
ISBN (Electronic)978-3-540-48889-7
ISBN (Print)978-3-540-48887-3
DOIs
Publication statusPublished - 1 Jan 2006

Publication series

SeriesLecture Notes in Computer Science
ISSN0302-9743
SeriesLecture Notes in Computer Science
Volume4250

Cite this

Kocsis, L., Szepesvari, C. S., & Winands, M. H. M. (2006). RSPSA: Enhanced Parameter Optimisation in Games. In H. J. van den Herik, S. C. Hsu, & H. H. L. M. Donkers (Eds.), Advances in Computer Games. ACG 2005. (Vol. 4250, pp. 39-56). Springer. Lecture Notes in Computer Science, Lecture Notes in Computer Science, Vol.. 4250 https://doi.org/10.1007/11922155_4