DiplomacyTournamentsJohn Newbury 17 Jul 2012 |
Home > Diplomacy > Tournaments
Tournaments for AI in the Game of Diplomacy
DeepLoamSea contains (ad hoc) tools for testing itself and other bots that I call DeepLoamSea Tournament Director Tools (DTDT). In particular, it can run tournaments between any suitable bots and record results in an Access database, for which I have written a few analysis queries. Tournaments are used for debugging and evaluating my bot, and also for it to gain experience in play. Generally these tournaments run without undue delays, by waiting for appropriate state changes, rather than for arbitrary times (unlike DTD), but DTDT has to be prepared to restart a game that hangs (especially due to occasional crashes of the Server (at least in version 0.24) or DiploBot), reducing CPU priority of Project20M (see Bots), and other ad hoc tricks, if it is to complete a lengthy tournament.
For my latest results and comments from an ongoing comparison of all released bots, and more, see the Arena, which will, in future, normally be used in place of discrete tournaments.
Tournament #1 : Description and results – finished 26 April 2005
Project20M was not included, as its games failed too often to make satisfactory progress through the tournament (see note in Bots).
All deadlines set to 2 seconds. (No bots attempt to use all time available, except due to occasional infinite loops.)
Tournament #2 : Description and results – finished 17 November 2005
As Tournament #1, except:
DeepLoamSea now exists and works reliably, but not yet sea-worthy – informally, it would come last!
Added Project20M (see note in Bots), and HoldBot and RandBot.
Tournament #3 : Description and results – finished 18 November 2005
As Tournament #2, except:
Uses a Slow Knockout method for gradually eliminating bots, based on Strength (ability to win)
Tournament #4 : Description and results – finished 28 November 2005
As Tournament #3, except:
Uses a Slow Knockout method for gradually eliminating bots, based on Tenacity (ability to survive).
Server kill value raise from 1 to 4.
Tournament #5 : Description and results – finished 8 February 2006
As Tournament #3, except:
Added KissMyBot 1.0.
Different type of analysis.
Tournament #6 : Description and results – 2 March 2006
As Tournament #5, except:
Server kill value raised from 4 to100.
Tournament #7 : Description and results – finished 3 March 2006
As Tournament #6, except:
Added KissMyBot 2.0.
Uses Slow Knockout method for gradually eliminating bots, based on Fitness (ability to gain high DAIDE-standard scores)
Different type of analysis.
Tournament #8 : Description and results – finished 3 May 2006
As Tournament #7, except:
Upgraded to Server version 0.27, which no longer uses WRT and message_id in press messages.
Added BlabBot 1, which was designed use simple press to form, and take advantage of, PCE agreements.
Press level set to 8000 (no change, but irrelevant previously, as no bots used press).
The original analysis was withdrawn when found to be flawed due to various failures that were discovered later. The failures also invalidated its objectives (the first test of press). However, the useful games (without BlabBot) have been retained in the SAGA database for any future analysis.
Tournament #9 : Description and results – finished 13 May 2006
As Tournament #8, except:
Upgraded to BlabBot 2.0, which used press as intended – the first and only bot capable of any press at the time.
Upgraded to KissMyBot 3.01, including ability to handle (albeit not use) the new press syntax.
Updated HoldBot, RandBot and DumbBot to handle (albeit not use) the new press syntax. Version numbers unchanged.
To maximize the effects of press, all games comprised 3 of one bot-type and 4 of another.
Server kill value reduced from 100 to 10, due to games between a few combinations of bot otherwise failing to terminate in a feasible time.
BlabBot 2.0 was later found to have serious bugs, albeit they would not have affected the tournament. The bugs were fixed in 2.1. As 2.1 would have played exactly the same as 2.0 in the tournament, BlabBot's version is renamed as 2.1 in the SAGA database, thereby facilitating future DEMO trials and analyses.