“Did You Hear That?” Learning to Play Video Games from Audio Cues

Raluca D. Gaina*, Matthew Stephenson

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingAcademicpeer-review


Game-playing AI research has focused for a long time on learning to play video games from visual input or symbolic information. However, humans benefit from a wider array of sensors which we utilise in order to navigate the world around us. In particular, sounds and music are key to how many of us perceive the world and influence the decisions we make. In this paper, we present initial experiments on game-playing agents learning to play video games solely from audio cues. We expand the Video Game Description Language to allow for audio specification, and the General Video Game AI framework to provide new audio games and an API for learning agents to make use of audio observations. We analyse the games and the audio game design process, include initial results with simple QLearning agents, and encourage further research in this area.
Original languageEnglish
Title of host publicationIEEE Conference on Games 2019
Number of pages4
ISBN (Print)9781728118840
Publication statusPublished - 2019
EventIEEE Conference on Games (IEEE COG) - London, United Kingdom
Duration: 20 Aug 201923 Aug 2019

Publication series

SeriesIEEE Conference on Computational Intelligence and Games


ConferenceIEEE Conference on Games (IEEE COG)
Country/TerritoryUnited Kingdom
Internet address


Dive into the research topics of '“Did You Hear That?” Learning to Play Video Games from Audio Cues'. Together they form a unique fingerprint.

Cite this