"Superstition" in the Network: Deep Reinforcement Learning Plays Deceptive Games

Matthew Stephenson, Philip Bontrager*, Ahmed Khalifa, Damien Anderson, Christoph Salge, Julian Togelius

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review


Deep reinforcement learning has learned to play many games well, but failed on others. To better characterize the modes and reasons of failure of deep reinforcement learners, we test the widely used Asynchronous Actor-Critic (A2C) algorithm on four deceptive games, which are specially designed to provide challenges to game-playing agents. These games are implemented in the General Video Game AI framework, which allows us to compare the behavior of reinforcement learningbased agents with planning agents based on tree search. We find that several of these games reliably deceive deep reinforcement learners, and that the resulting behavior highlights the shortcomings of the learning algorithm. The particular ways in which agents fail differ from how planning-based agents fail, further illuminating the character of these algorithms. We propose an initial typology of deceptions which could help us better understand pitfalls and failure modes of (deep) reinforcement learning.

Original languageEnglish
Title of host publicationFifteenth Annual AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
EditorsGillian Smith, Levi Lelis
Publication statusPublished - Oct 2019
EventFifteenth AAAI Conference on ArtificialIntelligence and Interactive Digital Entertainment - Georgia Institute of Technology in Atlanta, Georgia, United States
Duration: 8 Oct 201912 Oct 2019
Conference number: 15


ConferenceFifteenth AAAI Conference on ArtificialIntelligence and Interactive Digital Entertainment
Abbreviated titleAIIDE-19
Country/TerritoryUnited States
Internet address

Cite this