I’m fond of “x percent of sun’s energy used”-syle stuff because I would expect runaway superintelligence to probably go ahead and use that energy, and it has a decent shot at being resolvable.
But I think we need to be careful about assuming all the crazy-ambitious milestones end up only a few months from each other. You could have a situation where cosmic industrialization is explosively fast heading away from Earth, with every incentive to send out seed ships for a land-grab. But it’s plausible that could be going on despite here on Earth it’s much slower, if there are some big incumbents that maintain control and develop more slowly. I’m not super sure that’ll happen but it’s not obvious that all the big milestones happen within a few months of each other, if we assume local control is maintained and the runaway Foom goes elsewhere.
This is an example of why I think it does matter what milestone people pick, but it will often be for reasons that are very hard to foresee.
Yeah, I somewhat prefer “could capture >1% of the energy within a year if we wanted.” But if being more serious I think it’s better to just directly try to get at the spirit of “basically everything happens all at once,” or else do something more mundane like massive growth in energy capture (like 100%/year rather than “all the sun’s energy”).
I’m fond of “x percent of sun’s energy used”-syle stuff because I would expect runaway superintelligence to probably go ahead and use that energy, and it has a decent shot at being resolvable.
But I think we need to be careful about assuming all the crazy-ambitious milestones end up only a few months from each other. You could have a situation where cosmic industrialization is explosively fast heading away from Earth, with every incentive to send out seed ships for a land-grab. But it’s plausible that could be going on despite here on Earth it’s much slower, if there are some big incumbents that maintain control and develop more slowly. I’m not super sure that’ll happen but it’s not obvious that all the big milestones happen within a few months of each other, if we assume local control is maintained and the runaway Foom goes elsewhere.
This is an example of why I think it does matter what milestone people pick, but it will often be for reasons that are very hard to foresee.
Yeah, I somewhat prefer “could capture >1% of the energy within a year if we wanted.” But if being more serious I think it’s better to just directly try to get at the spirit of “basically everything happens all at once,” or else do something more mundane like massive growth in energy capture (like 100%/year rather than “all the sun’s energy”).