Consider that “if AGI is very near” probably means that it’s already happened (or, equivalently, that we are past the point of no return) on Copernican grounds, since the odds of living in a very special moment where the timelines are short but it’s not too late yet are very low. Not seeing an obvious AGI around likely means that either it’s not very near, or that the take-off is slow, not fast.
Ironically, it’s not Roko’s basilisk that is an infohazard, it’s the “AGI go foom!” idea that is.
I want to clarify that “AGI go foom!” is not really concerned with the nearness of the advent of AGI, but with whether AGIs have a discontinuity that results in an acceleration of the development of their intelligence over time.
I don’t understand how the Copernican argument works. Being alive in the moment before the first AGI exists is very unlikely, but surely it is very unlikely to be alive in any moment around the development of AGI in general. If anything, you could possibly argue that it’s more likely we are in some kind of simulation than in base reality right before AGI takeoff. If that’s not the point you’re making, could you restate the argument?
Consider that “if AGI is very near” probably means that it’s already happened (or, equivalently, that we are past the point of no return) on Copernican grounds, since the odds of living in a very special moment where the timelines are short but it’s not too late yet are very low. Not seeing an obvious AGI around likely means that either it’s not very near, or that the take-off is slow, not fast.
Ironically, it’s not Roko’s basilisk that is an infohazard, it’s the “AGI go foom!” idea that is.
I want to clarify that “AGI go foom!” is not really concerned with the nearness of the advent of AGI, but with whether AGIs have a discontinuity that results in an acceleration of the development of their intelligence over time.
I don’t understand how the Copernican argument works. Being alive in the moment before the first AGI exists is very unlikely, but surely it is very unlikely to be alive in any moment around the development of AGI in general. If anything, you could possibly argue that it’s more likely we are in some kind of simulation than in base reality right before AGI takeoff. If that’s not the point you’re making, could you restate the argument?
Yes, it is a reversed doomsday argument: it is unlikely that the end is nigh.