If it’s possible for super-intelligent AI to be non-sentient, wouldn’t it be possible for insects to evolve non-sentient intelligence as well? I guess I didn’t assume “non-sentient” in the definition of “unaligned”.
In principle, I prefer sentient AI over non-sentient bugs. But the concern that is if non-sentient superintelligent AI is developed, it’s an attractor state, that is hard or impossible to get out of. Bugs certainly aren’t bound to evolve into sentient species, but at least there’s a chance.
If it’s possible for super-intelligent AI to be non-sentient, wouldn’t it be possible for insects to evolve non-sentient intelligence as well? I guess I didn’t assume “non-sentient” in the definition of “unaligned”.
In principle, I prefer sentient AI over non-sentient bugs. But the concern that is if non-sentient superintelligent AI is developed, it’s an attractor state, that is hard or impossible to get out of. Bugs certainly aren’t bound to evolve into sentient species, but at least there’s a chance.