jsalvati—“I think the difference is that in a world where one of them is miscalculating, that person can be shown that they are miscalculating and will then calculate correctly.”
This still won’t do, due to path-dependence and such. Suppose Bob could be corrected in any number of ways, and each will cause him to adopt a different conclusion—and one that he will then persist in holding no matter what other arguments you give him. Which conclusion is the true value for our original morality_Bob? There can presumably be no fact of the matter, on Eliezer’s account. And if this sort of underdetermination is very common (which I imagine it is), then there’s probably no facts at all about what any of our “moralities” are. There may always be some schedule of information that would bring us to make radically different moral judgments.
Also worrying is the implication that it’s impossible to be stubbornly wrong. Once you become impervious to argument in your adoption of inconsistent moral beliefs, well, those contradictions are now apparently part of your true morality, which you’re computing just fine.(?)
jsalvati—“I think the difference is that in a world where one of them is miscalculating, that person can be shown that they are miscalculating and will then calculate correctly.”
This still won’t do, due to path-dependence and such. Suppose Bob could be corrected in any number of ways, and each will cause him to adopt a different conclusion—and one that he will then persist in holding no matter what other arguments you give him. Which conclusion is the true value for our original morality_Bob? There can presumably be no fact of the matter, on Eliezer’s account. And if this sort of underdetermination is very common (which I imagine it is), then there’s probably no facts at all about what any of our “moralities” are. There may always be some schedule of information that would bring us to make radically different moral judgments.
Also worrying is the implication that it’s impossible to be stubbornly wrong. Once you become impervious to argument in your adoption of inconsistent moral beliefs, well, those contradictions are now apparently part of your true morality, which you’re computing just fine.(?)