The early game, during which interest in AI is not mainstream. I think this ended within the last year 😢 [ETA: early game ended 2022]
The midgame, during which interest in AI is mainstream but before AGI is imminent. During the midgame:
The AI companies are building AIs that they don’t expect will be transformative.
The alignment work we do is largely practice for alignment work later, rather than an attempt to build AIs that we can get useful cognitive labor from without them staging coups.
For the purpose of planning my life, I’m going to imagine this as lasting four more years. This is shorter than my median estimate of how long this phase will actually last.
The endgame, during which AI companies conceive of themselves as actively building models that will imminently be transformative, and that pose existential takeover risk.
During the endgame, I think that we shouldn’t count on having time to develop fundamentally new alignment insights or techniques (except maybe if AIs do most of the work? Idt we should count on this); we should be planning to mostly just execute on alignment techniques that involve ingredients that seem immediately applicable.
For the purpose of planning my life, I’m going to imagine this as lasting three years. This is about as long as I expect this phase to actually take.
I like Buck’s A freshman year during the AI midgame: my approach to the next year post from 2023.
Here’s his taxonomy: