Interesting stuff. And I agree. Once you have a nanosystem or something of equivalent power, humans are no longer any threat. But we’re yet to be sure if such thing is physically possible. I know many here think so, but I still have my doubts.
Maybe it’s even more likely that some random narrow AI failure will start big wars before anything more fancy. Although with the scaling hypothesis on sight, AGI could come suddenly indeed.
“This is basically as bad as also killing everyone, because we’d still be imprisoned away from our largest possible impact.”
Although I quite disagree with this. I’m not a huge supporter of our largest possible impact. I guess it’s naive to attribute any net positive expectation to that when you look at history or at the present. In fact, such outcome (things staying exactly the same forever) would probably be among the most positive ones in the advent of non aligned AI. As long as we could still take care of Earth, like ending factory farming and dictatorships, it really wouldn’t be that bad...
Interesting stuff. And I agree. Once you have a nanosystem or something of equivalent power, humans are no longer any threat. But we’re yet to be sure if such thing is physically possible. I know many here think so, but I still have my doubts.
Maybe it’s even more likely that some random narrow AI failure will start big wars before anything more fancy. Although with the scaling hypothesis on sight, AGI could come suddenly indeed.
“This is basically as bad as also killing everyone, because we’d still be imprisoned away from our largest possible impact.”
Although I quite disagree with this. I’m not a huge supporter of our largest possible impact. I guess it’s naive to attribute any net positive expectation to that when you look at history or at the present. In fact, such outcome (things staying exactly the same forever) would probably be among the most positive ones in the advent of non aligned AI. As long as we could still take care of Earth, like ending factory farming and dictatorships, it really wouldn’t be that bad...