My Eliezer-model thinks pivotal acts are genuinely, for-real, actually important. Like, he’s not being metaphorical or making a pedagogical point when he says (paraphrasing) ‘we need to use the first AGI systems to execute a huge, disruptive, game-board-flipping action, or we’re all dead’.
When my Eliezer-model says that the most plausible pivotal acts he’s aware of involve capabilities roughly at the level of ‘develop nanotech’ or ‘put two cellular-identical strawberries on a plate’, he’s being completely literal. If some significantly weaker capability level realistically suffices for a pivotal act, then my Eliezer-model wants us to switch to focusing on that (far safer) capability level instead.
If we can save the world before we get anywhere near AGI, then we don’t necessarily have to sort out how consequentialist, dangerous, hardware-overhang-y, etc. the first AGI systems will be. We can just push the ‘End The Acute Existential Risk Period’ button, and punt most other questions to the non-time-pressured Reflection that follows.
My Eliezer-model thinks pivotal acts are genuinely, for-real, actually important. Like, he’s not being metaphorical or making a pedagogical point when he says (paraphrasing) ‘we need to use the first AGI systems to execute a huge, disruptive, game-board-flipping action, or we’re all dead’.
When my Eliezer-model says that the most plausible pivotal acts he’s aware of involve capabilities roughly at the level of ‘develop nanotech’ or ‘put two cellular-identical strawberries on a plate’, he’s being completely literal. If some significantly weaker capability level realistically suffices for a pivotal act, then my Eliezer-model wants us to switch to focusing on that (far safer) capability level instead.
If we can save the world before we get anywhere near AGI, then we don’t necessarily have to sort out how consequentialist, dangerous, hardware-overhang-y, etc. the first AGI systems will be. We can just push the ‘End The Acute Existential Risk Period’ button, and punt most other questions to the non-time-pressured Reflection that follows.