I’m not sure I’d agree with that framing. If an ethical feature changes with education, that’s good evidence that it’s not a terminal value, to whatever extent that it makes sense to talk about terminal values in humans. Which may very well be “not very much”; our value structure is a lot messier than that of the theoretical entities for which the terminal/instrumental dichotomy works well, and if we had a good way of cleaning it up we wouldn’t need proposals like CEV.
People can change between egalitarian and hierarchical ethics without neurological insults or biochemical tinkering, so human “terminal” values clearly don’t necessitate one or the other. More importantly, though, CEV is not magic; it can resolve contradictions between the ethics you feed into it, and it might be able to find refinements of those ethics that our biases blind us to or that we’re just not smart enough to figure out, but it’s only as good as its inputs. In particular, it’s not guaranteed to find universal human values when evaluated over a subset of humanity.
If you took a collection of 19th-century slave owners and extrapolated their ethical preferences according to CEV-like rules, I wouldn’t expect that to spit out an ethic that allowed slavery—the historical arguments I’ve read for the practice didn’t seem very good—but I wouldn’t be hugely surprised if it did, either. Either way it wouldn’t imply that the resulting ethic applies to all humans or that it derives from immutable laws of rationality; it’d just tell us whether it’s possible to reconcile slavery with middle-and-upper-class 19th-century ethics without downstream contradictions.
I’m not sure I’d agree with that framing. If an ethical feature changes with education, that’s good evidence that it’s not a terminal value, to whatever extent that it makes sense to talk about terminal values in humans. Which may very well be “not very much”; our value structure is a lot messier than that of the theoretical entities for which the terminal/instrumental dichotomy works well, and if we had a good way of cleaning it up we wouldn’t need proposals like CEV.
People can change between egalitarian and hierarchical ethics without neurological insults or biochemical tinkering, so human “terminal” values clearly don’t necessitate one or the other. More importantly, though, CEV is not magic; it can resolve contradictions between the ethics you feed into it, and it might be able to find refinements of those ethics that our biases blind us to or that we’re just not smart enough to figure out, but it’s only as good as its inputs. In particular, it’s not guaranteed to find universal human values when evaluated over a subset of humanity.
If you took a collection of 19th-century slave owners and extrapolated their ethical preferences according to CEV-like rules, I wouldn’t expect that to spit out an ethic that allowed slavery—the historical arguments I’ve read for the practice didn’t seem very good—but I wouldn’t be hugely surprised if it did, either. Either way it wouldn’t imply that the resulting ethic applies to all humans or that it derives from immutable laws of rationality; it’d just tell us whether it’s possible to reconcile slavery with middle-and-upper-class 19th-century ethics without downstream contradictions.