In board games, the endgame is a period generally characterized by strategic clarity, more direct calculation of the consequences of actions rather than evaluating possible actions with heuristics, and maybe a narrower space of actions that players consider.
Relevant actors, particularly AI labs, are likely to experience increased strategic clarity before the end, including AI risk awareness, awareness of who the leading labs are, roughly how much lead time they have, and how threshold-y being slightly ahead is.
There may be opportunities for coordination in the endgame that were much less incentivized earlier, and pausing progress may be incentivized in the endgame despite being disincentivized earlier (from an actor’s imperfect perspective and/or from a maximally informed/wise/etc. perspective).
The downside of “AI endgame” as a conceptual handle is that it suggests thinking of actors as opponents/adversaries. Probably “crunch time” is better, but people often use that to gesture at hinge-y-ness rather than strategic clarity.
That could be, but also maybe there won’t be a period of increased strategic clarity. Especially if the emergence of new capabilities with scale remains unpredictable, or if progress depends on finding new insights.
I can’t think of many games that don’t have an endgame. These examples don’t seem that fun:
A single round of musical chairs.
A tabletop game that follows an unpredictable, structureless storyline.
Agree. I merely assert that we should be aware of and plan for the possibility of increased strategic clarity, risk awareness, etc. (and planning includes unpacking “etc.”).
Probably taking the analogy too far, but: most games-that-can-have-endgames also have instances that don’t have endgames; e.g. games of chess often end in the midgame.
For example, if a lab is considering deploying a powerful model, it can prosocially show its hand—i.e., demonstrate that it has a powerful model—and ask others to make themselves partially transparent too. This affordance doesn’t appear until the endgame. I think a refined version of it could be a big deal.
AI endgame
In board games, the endgame is a period generally characterized by strategic clarity, more direct calculation of the consequences of actions rather than evaluating possible actions with heuristics, and maybe a narrower space of actions that players consider.
Relevant actors, particularly AI labs, are likely to experience increased strategic clarity before the end, including AI risk awareness, awareness of who the leading labs are, roughly how much lead time they have, and how threshold-y being slightly ahead is.
There may be opportunities for coordination in the endgame that were much less incentivized earlier, and pausing progress may be incentivized in the endgame despite being disincentivized earlier (from an actor’s imperfect perspective and/or from a maximally informed/wise/etc. perspective).
The downside of “AI endgame” as a conceptual handle is that it suggests thinking of actors as opponents/adversaries. Probably “crunch time” is better, but people often use that to gesture at hinge-y-ness rather than strategic clarity.
That could be, but also maybe there won’t be a period of increased strategic clarity. Especially if the emergence of new capabilities with scale remains unpredictable, or if progress depends on finding new insights.
I can’t think of many games that don’t have an endgame. These examples don’t seem that fun:
A single round of musical chairs.
A tabletop game that follows an unpredictable, structureless storyline.
Agree. I merely assert that we should be aware of and plan for the possibility of increased strategic clarity, risk awareness, etc. (and planning includes unpacking “etc.”).
Probably taking the analogy too far, but: most games-that-can-have-endgames also have instances that don’t have endgames; e.g. games of chess often end in the midgame.
I wouldn’t take board games as a reference class but rather war or maybe elections. I’m not sure in these cases you have more clarity towards the end.
For example, if a lab is considering deploying a powerful model, it can prosocially show its hand—i.e., demonstrate that it has a powerful model—and ask others to make themselves partially transparent too. This affordance doesn’t appear until the endgame. I think a refined version of it could be a big deal.