Dumb question: In the omnicide outcome agi continues its existence, but whether it’s sentient or not no one else from earth does. In the undignified survival outcome agi continues its existence and potentially so do many humans for functionally ever and those human may have lives better than death (even barely). Why is parochial value-alignment necessarily worse than no alignment at all?
Jay95
Karma: −13
A poor but certain attempt to philosophically undermine the orthogonality of intelligence and aims
[Question] Orthogonality and the increased overlap of actions with the set of all events that comes with increased intelligence
An “ontological crisis” is treated well enough when you note your perception as a belief. If it’s literally impossible to not have a best working model of reality in its entirety that undermines profound doubt.
I don’t think there’s a physical difference between morality being arbitrary because it is the evolution of blind selfish memes, or morality being limited by practical considerations. If someone had a hypothetical objective morality they could still be a slave-owner and war-monger in one age and a civil-rights activist calling for world peace in a different age.