Suppose we were to write down all (input, output) pairs for the ideal “one-place function” described by Eliezer on a oblong stone tablet somewhere. This stone tablet would then contain perfect moral wisdom. It would tell us the right course of action in any possible situation.
This tablet would be the result of computation, but it’s computation that nobody can actually do, as we currently only have access to approximations to the ideal Morality(X) function. Thus, as far as we’re concerned, this tablet is just a giant look-up table. Its contents are a brute fact about the universe, like the ratio of the masses of the proton and the electron. If we are confronted with a moral dilemna, and our personal ideas of right and wrong contradict the tablet, this will always be a result of our own morality functions poorly approximating the ideal. In such a situation, we should override our instincts and go with the tablet every time.
In other words, according to Eliezer’s model, in a universe where this tablet exists morality is given.
This is also true of a universe where the tablet does not exist (such as ours—it wouldn’t fit!).
So Eliezer has just rediscovered “morality is the will of God”, except he’s replacing “God” with a giant block of stone somewhere in a hypothetical universe. It’s not clear to me that this is an impovement.
It seems to me that the functional difference is that Eliezer believes he can successfully approximate the will of the Giant Hypothetical Oblong Stone Tablet out of his own head. If George Washington says “Slavery is sometimes just,” Eliezer does not take this assertion seriously; he does not start trying to re-work his personal GHOST-approximator to take Washington’s views into account. Rather he says, “I know that slavery is wrong, and I approximate the GHOST, so slavery is wrong,” ignoring the fact that all men—including Washington—approximate the GHOST as best they can. Worse, by emphasizing the process of making, weighing and pondering moral “arguments”, he privileges the verbally and quantitatively quick over the less intelligent, even though the correlation between being good with words and having a good GHOST-approximator is nowhere shown.
Everyone’s GHOST-approximator is shaped by his environment. If the modern world encourages people to deny the GHOST in particular ways, and Eliezer indeed does so, then he would not be able to tell. His tool for measuring, his personal GHOST-finder, would have been twisted. His friends’ and respected peers’ GHOST-approximators might all be twisted in the same way, so nobody would point out his error and he would have no opportunity to correct it. He would use his great skill with words to try to convince everyone that his personal morality was correct. Him and people like him might well succeed. His assertion of moral progress would then merely be the statement that the modern world reflects his personal biases—or perhaps that he reflects the biases of the modern world.
I’m concerned that the metamorality described by Eliezer will encourage self-named rationalists to worship their own egos, placing their personal imperfect GHOST-approximators—all shaped by the moral environment of the modern world—at the same level as those in past ages placed the will of God. Perhaps this is not Eliezer’s intention. But to do otherwise, to look beyond the biases of the present day, one would have to acknowledge that the GHOST-readers of our ancestors may have in some ways have been better than ours. This would require humility; and pride cures humility.
Suppose we were to write down all (input, output) pairs for the ideal “one-place function” described by Eliezer on a oblong stone tablet somewhere. This stone tablet would then contain perfect moral wisdom. It would tell us the right course of action in any possible situation.
This tablet would be the result of computation, but it’s computation that nobody can actually do, as we currently only have access to approximations to the ideal Morality(X) function. Thus, as far as we’re concerned, this tablet is just a giant look-up table. Its contents are a brute fact about the universe, like the ratio of the masses of the proton and the electron. If we are confronted with a moral dilemna, and our personal ideas of right and wrong contradict the tablet, this will always be a result of our own morality functions poorly approximating the ideal. In such a situation, we should override our instincts and go with the tablet every time.
In other words, according to Eliezer’s model, in a universe where this tablet exists morality is given.
This is also true of a universe where the tablet does not exist (such as ours—it wouldn’t fit!).
So Eliezer has just rediscovered “morality is the will of God”, except he’s replacing “God” with a giant block of stone somewhere in a hypothetical universe. It’s not clear to me that this is an impovement.
It seems to me that the functional difference is that Eliezer believes he can successfully approximate the will of the Giant Hypothetical Oblong Stone Tablet out of his own head. If George Washington says “Slavery is sometimes just,” Eliezer does not take this assertion seriously; he does not start trying to re-work his personal GHOST-approximator to take Washington’s views into account. Rather he says, “I know that slavery is wrong, and I approximate the GHOST, so slavery is wrong,” ignoring the fact that all men—including Washington—approximate the GHOST as best they can. Worse, by emphasizing the process of making, weighing and pondering moral “arguments”, he privileges the verbally and quantitatively quick over the less intelligent, even though the correlation between being good with words and having a good GHOST-approximator is nowhere shown.
Everyone’s GHOST-approximator is shaped by his environment. If the modern world encourages people to deny the GHOST in particular ways, and Eliezer indeed does so, then he would not be able to tell. His tool for measuring, his personal GHOST-finder, would have been twisted. His friends’ and respected peers’ GHOST-approximators might all be twisted in the same way, so nobody would point out his error and he would have no opportunity to correct it. He would use his great skill with words to try to convince everyone that his personal morality was correct. Him and people like him might well succeed. His assertion of moral progress would then merely be the statement that the modern world reflects his personal biases—or perhaps that he reflects the biases of the modern world.
I’m concerned that the metamorality described by Eliezer will encourage self-named rationalists to worship their own egos, placing their personal imperfect GHOST-approximators—all shaped by the moral environment of the modern world—at the same level as those in past ages placed the will of God. Perhaps this is not Eliezer’s intention. But to do otherwise, to look beyond the biases of the present day, one would have to acknowledge that the GHOST-readers of our ancestors may have in some ways have been better than ours. This would require humility; and pride cures humility.