Support different competition formats
This Ludii paper lists a number of possible ways to run accademic competitions they are considering. They also seem quite reasonable and so It'd be good if we could support them server-side for them :). The below descriptions are from their paper so might need to be generallised for the server.
Elimination
Agents play against each other in a series of knockout style rounds. Each round of the competition selects a new set of games that all remaining agents are then evaluated using. The best performing agents from each round then proceed to the next, until only one agent remains. If the initial number of agents is very high, then a benchmark set of games may be used to select a smaller number of best performing agents that then proceed to the main tournament rounds.
Round-Robin
Each agent plays against all other agentson a single set of games. Points are awarded for each match-up based on the “Three points for a win” system (win = 3 points, draw = 1 point, loss = 0 points). The final point ranking may be used to calculate the winner of the competition, or to determine which agents qualify for a separate elimination format tournament.
Leaderboard
Agents can be submitted to a centralised server that keeps a continuously updated ranking of each agent’s average performance across all games in the public Ludii repository. Individual performances are recorded for each game, as well as for each game category and across the entire game set. New agents or versions can be submitted at any time, to compare their performance against other existing agents. Humans are also able to be ranked on this leaderboard alongside these agents, as a benchmark measure of human vs AI ability. Performance values for each agent will likely be based on their percentage number of wins. Several rating systems for this are possible, such as Glicko-2 which is used by the GVGAI two-player track or AGON which is used for GGP.
League
Agents can be entered to compete in a league that runs over a certain period of time, likely a few months. This league operates similar to those of most sports, where all entrants play against each other over a series of matches. This would be conducted in a manner very similar to that of the described Round-Robin format, but where a greater number of games are played less often and over a longer period of time. Development teams might also be able to adjust their agent between games, to fix issues or tailor strategies towards an upcoming opponent. New games are expected to be played each week and will be livestreamed online for people to watch. Some of the best performing human players may also be invited to take part in these competitions, to provide a comparison with human capabilities.