you think there is no simple procedure that would find roughly the same “should function” hidden somewhere in the brain of a brain-washed blood-thirsty religious zealot?
Sure, the kolmogorov complexity of a set of edits to change the moral reflective equilibrium of a human is probably pretty low compared to the complexity of the overall human preference set. But that works the other way around too. Somewhere hidden in the brain of a a liberal western person is a murderer/terrorist/child abuser/fundamentalist if you just perform the right set of edits.
But that works the other way around too. Somewhere hidden in the brain of a a liberal western person is a murderer/terrorist/child abuser/fundamentalist if you just perform the right set of edits.
Again, not all beliefs are equal. You don’t want to use the procedure that’ll find a murderer in yourself, you want to use the procedure that’ll find a nice fellow in a murderer. And given such a procedure, you won’t need to exclude murderers from extrapolated volition.
Sure, the kolmogorov complexity of a set of edits to change the moral reflective equilibrium of a human is probably pretty low compared to the complexity of the overall human preference set. But that works the other way around too. Somewhere hidden in the brain of a a liberal western person is a murderer/terrorist/child abuser/fundamentalist if you just perform the right set of edits.
Again, not all beliefs are equal. You don’t want to use the procedure that’ll find a murderer in yourself, you want to use the procedure that’ll find a nice fellow in a murderer. And given such a procedure, you won’t need to exclude murderers from extrapolated volition.