When AI solves a game, focus on the game’s mechanics, not its theme.
A game design consists of two things: mechanics and theme.
The game mechanics is the abstract protocol governing how the players interact with each other and their shared environment.
The game theme is a fictional interpretation of the elements of the game.
Consider Battleship — the theme is a naval battle, and the mechanics are a particular 2-player sequential discovery game.
There is a correspondence between the ontology of the mechanics and the ontology of the theme, but this correspondence is mostly arbitrary. For example, the Knight in chess has almost nothing to do with actual knights.
When an AI solves a game, people often overfocus on the theme of the game relative to the mechanics of the game.
Maybe this is for psychological reasons:
The theme is more interesting than the mechanics.
The theme is in our pre-cached ontology — my brain already has a pre-cached concept of “naval battle” but it doesn’t have a pre-cached concept corresponding to the particular mechanics of Battleship. In fact, this is why games have themes in the first place — they serve partly as mnemonics for the rules.
Did you know that people cooperate more in the Prisoner’s Dilemma if the game is called “The Community Game” than “The Wall Street Game”?!
Or maybe this is for rational reasons:
Other people might think that there is a deeper connection between the theme and the mechanics of the particular game than I do. For example, they might think there is some genuine non-arbitrary connection between the mechanics of monopoly and the real estate market.
See the existing debate about ludonarrative dissonance.
If people overfocus on the theme, then they will make incorrect predictions about AI.
For example, they’ll hear that AI has solved Full-Press Diplomacy and extrapolate that AI will soon be able to solve other games of a similar theme (i.e. international military negotiations).
However, they should instead extrapolate that AI will soon be able to solve other games with similar mechanics.
Here’s some practical advice: imagine the game had the same mechanics but a different theme. Sure, AI has solved Full-Press Diplomacy, which is scary because the theme is militaries negotiating about nations to invade. But what if the theme was about gardeners negotiating which flowers to buy? Okay, still a bit scary, but you get my point.
Appendix
So anyway, what are game mechanics?
Is time discrete (i.e. turn-based) or continuous?
How many turns are there? Boundedly-many? Finite-but-unboundedly many? Infinitely many?
Do the players move sequentially or simultaneously?
How many possible moves are there each turn?
How many players are there?
Are players ever eliminated?
What are the winning conditions?
How many players can win?
Do the players accumulate points?
How sparsely are points allocated?
Do players know how many points the other players have?
Is it harder to accumulate points the more you have, or easier?
Is there an upper bound on the number of points? A lower bound?
Is the game purely cooperative, purely adversarial, or somewhere inbetween?
Can the players communicate publicly?
Can the players communicate privately?
Are the mechanics symmetric with respect to each player?
Are the mechanics symmetric with respect to each pair of players? E.t.c.
How unequal is the advantage between players at the start?
Is the environment deterministic? Or stochastic? What’s the bit-rate from the Random Number Generator?
How Kolmogorov-complex is the environment?
How computationally-complex is the environment?
Is there little interference between players (like the 400m dash), or much interference (like a football game)?
Does one strategy dominate all others, or is the game nontransitive?
Is the state space continuous or discrete?
Is the action space continuous or discrete?
If discrete, how many bits specify the game state?
If continuous, what’s the topology?
Are the actions reversible?
And so on, and so on.
Kill all humans is an actual game so, maybe!
AI has solved DEFCON! Oh no!
Random, possibly stupid thought from my associations: what if we could create an AI capable of finding exploits in the rules of the games? Not just Goodhart the rules, but explicitly output “hey, game designers, I think this is an exploit, it’s against the spirit of the game”. It might have something to do with the alignment.
This happened with EURISKO.
As i understand the linked text, EURISKO just played a game, not compared the spirit of the game with the rules as written. The latter would require general knowledge about the world at the level of current language models.
Even if an AI wouldn’t explicitly search for exploits, if you just had it search for the best winning solution it’s quite likely that it’d hit on something that the people making the game would consider an exploit. EURISKO did it, evolutionary algorithms often do it, and communities dedicated to specific games also often find effective strategies that are considered “exploity”. So if you just had an AI optimize for winning, you could probably find lots of exploits just by looking to see what its best strategies are based on.
Yes, I understand. My whole idea is that this AI should explicitly output something like “I found this strategy and I think this is an exploit and it should be fixed” in some cases (for example, if it found dominant strategy in a game that is primarily about trade negotiations and this strategy allows you to not use trade at all. Or if it found that in a game about air combat you can fly into terrain because of a bug in game engine) and just be good at playing in other cases (for example, in chess or go).