The post says “On our current view of the technological landscape, there are a number of plausible future technologies that could be leveraged to end the acute risk period.” I’m wondering what these other plausible future technologies are. (I’m guessing things like whole brain emulation and intelligence enhancement count, but are there any others?)
One of the footnotes says “There are other paths to good outcomes that we view as lower-probability, but still sufficiently high-probability that the global community should allocate marginal resources to their pursuit.” What do some of these other paths look like?
I’m confused about the differences between “minimal aligned AGI” and “task AGI”. (As far as I know, this post is the only place MIRI has used the term “minimal aligned AGI”, so I have very little to go on.) Is “minimal aligned AGI” the larger class, and “task AGI” the specific kind of minimal aligned AGI that MIRI has decided is most promising? Or is the plan to first build a minimal aligned AGI, which then builds a task AGI, which then performs a pivotal task/helps build a Sovereign?
If the latter, then it seems like MIRI has gone from a one-step view (“build a Sovereign”), to a two-step view (“build a task-directed AGI first, then go for Sovereign”), to a three-step view (“build a minimal aligned AGI, then task AGI, then Sovereign”). I’m not sure why “three” is the right number of stages (why not two or four?), and I don’t think MIRI has explained this. In fact, I don’t think MIRI has even explained why it switched to the two-step view in the first place. (Wei Dai made this point here.)
Thanks! I have some remaining questions:
The post says “On our current view of the technological landscape, there are a number of plausible future technologies that could be leveraged to end the acute risk period.” I’m wondering what these other plausible future technologies are. (I’m guessing things like whole brain emulation and intelligence enhancement count, but are there any others?)
One of the footnotes says “There are other paths to good outcomes that we view as lower-probability, but still sufficiently high-probability that the global community should allocate marginal resources to their pursuit.” What do some of these other paths look like?
I’m confused about the differences between “minimal aligned AGI” and “task AGI”. (As far as I know, this post is the only place MIRI has used the term “minimal aligned AGI”, so I have very little to go on.) Is “minimal aligned AGI” the larger class, and “task AGI” the specific kind of minimal aligned AGI that MIRI has decided is most promising? Or is the plan to first build a minimal aligned AGI, which then builds a task AGI, which then performs a pivotal task/helps build a Sovereign?
If the latter, then it seems like MIRI has gone from a one-step view (“build a Sovereign”), to a two-step view (“build a task-directed AGI first, then go for Sovereign”), to a three-step view (“build a minimal aligned AGI, then task AGI, then Sovereign”). I’m not sure why “three” is the right number of stages (why not two or four?), and I don’t think MIRI has explained this. In fact, I don’t think MIRI has even explained why it switched to the two-step view in the first place. (Wei Dai made this point here.)