One of the key problems in AI is how an agent can best learn in a largely unsupervised manner via interactions with its environment. Games provide an excellent way to test approaches to this problem. They provide ready made environments of variable complexity, offering dynamic and unpredictable challenges. They enable the emergence of open-ended intelligent behaviour and provide natural metrics to measure the success of that behaviour.

Two main ways to train agents given no prior export knowledge are temporal difference learning, and evolution (or co-evolution). We’ll study ways in which these methods can train agents for games such as Othello and Ms Pac-Man. The results show that each method has important strengths and weaknesses, and understanding these leads to the development of new hybrid algorithms such as EvoTDL, where evolution is used to evolve a population of TD learners. Examples will also be given of where seemingly innocuous changes to the learning environment have profound effects on the performance of each algorithm. Choice of architecture (e.g. type of neural network) is also critical.

The main conclusion is that these are powerful methods capable of learning interesting agent behaviours, but there is still something of a black art in how best to apply them, and there is a great deal of scope for designing new learning algorithms. The talk will also include live demonstrations.

Brief biography: Dr. Simon M. Lucas (SMIEEE) received the BSc degree in computer systems engineering from the University of Kent, UK, in 1986 and the PhD degree from the University of Southampton, UK, in 1991, having worked a year in between as a research engineer for GEC Avionics. After a one-year post doctoral research fellowship (funded by British Telecom) he was appointed to a lectureship at the University of Essex in 1992 and is currently a reader in computer science there.

His main research interests are evolutionary computation, games, and pattern recognition, and he has published widely in these fields with over 120 refereed papers, mostly in leading international conferences and journals. He was chair of IAPR Technical Committee 5 on Benchmarking and Software (2002 – 2006) and is the inventor of the scanning n-tuple classifier, a fast and accurate OCR method. He was appointed inaugural chair of the IEEE CIS Games Technical Committee in July 2006, has been competitions chair for many international conferences, and co-chaired the first IEEE Symposium on Computational Intelligence and Games in 2005. He was program chair for IEEE CEC 2006, and program co-chair for IEEE CIG 2007, and will be program co-chair for PPSN 2008. He is an associated editor of IEEE Transactions on Evolutionary Computation, and the Journal of Memetic Computing. He was invited keynote speaker at IEEE CEC 2007.