Bugs could potentially result in a new sentient species many millions of years down the line. With super-AI that happens to be non-sentient, there is no such hope.
If it’s possible for super-intelligent AI to be non-sentient, wouldn’t it be possible for insects to evolve non-sentient intelligence as well? I guess I didn’t assume “non-sentient” in the definition of “unaligned”.
In principle, I prefer sentient AI over non-sentient bugs. But the concern that is if non-sentient superintelligent AI is developed, it’s an attractor state, that is hard or impossible to get out of. Bugs certainly aren’t bound to evolve into sentient species, but at least there’s a chance.
Bugs could potentially result in a new sentient species many millions of years down the line. With super-AI that happens to be non-sentient, there is no such hope.
If it’s possible for super-intelligent AI to be non-sentient, wouldn’t it be possible for insects to evolve non-sentient intelligence as well? I guess I didn’t assume “non-sentient” in the definition of “unaligned”.
In principle, I prefer sentient AI over non-sentient bugs. But the concern that is if non-sentient superintelligent AI is developed, it’s an attractor state, that is hard or impossible to get out of. Bugs certainly aren’t bound to evolve into sentient species, but at least there’s a chance.