I still haven’t gotten through the metaethics sequence yet, so I can’t answer that exactly, but if he believed in an “objective” morality (i.e. some definition of “should” that is meaningful from the perspective of fundamental reality, not based on any facts about minds, or an internally-consistent set of universally compelling moral arguments), then he would probably expect a superintelligence to be smart enough (many times over) to discover it and follow it, and that is quite the opposite of his current position. If I recall correctly, that was his pre-2002 position, and he now considers it a huge mistake.
“Fundamental reality” doesn’t have a perspective, so it seems weird to draw the lines there. Rather, there’s a fact about what’s prime, and the pebblesorters care about that, and there’s a fact about what’s right, and humans care about that. We can be mistaken about what’s right, and we can have disagreements about what’s right, and we can change our minds. And given time and progress, we will hopefully get closer to understanding what’s right. And if the pebblesorters claim that they care about what’s right rather than what’s prime, they’re factually incorrect.
“Fundamental reality” doesn’t have a perspective, so it seems weird to draw the lines there.
Of course — I was just doing my best to imagine the mindset of a non-religious person who believes in an objectively objective morality (i.e. that even in the absence of a deity, the universe still somehow imposes moral laws). Admittedly, I don’t encounter too many of those (people who think they’ve devised universally compelling moral arguments are more common; even big-O Objectivists seem to just be an overconfident version of that), but I do still meet them from time to time, e.g. people who manage to believe in things like “natural law” or “natural rights” (as facts about the universe rather than facts about human minds) without theistic belief.
All I was saying was that things like that are what the phrase “objective morality” make me think of, and that Eliezer’s conclusions are different enough that I’m not sure they quite fit in the same category. His may be an “objective morality” by our best definitions of “objective” and “morality”, but it could make people (especially new people) imagine all the wrong things.
I still haven’t gotten through the metaethics sequence yet, so I can’t answer that exactly, but if he believed in an “objective” morality (i.e. some definition of “should” that is meaningful from the perspective of fundamental reality, not based on any facts about minds, or an internally-consistent set of universally compelling moral arguments), then he would probably expect a superintelligence to be smart enough (many times over) to discover it and follow it, and that is quite the opposite of his current position. If I recall correctly, that was his pre-2002 position, and he now considers it a huge mistake.
“Fundamental reality” doesn’t have a perspective, so it seems weird to draw the lines there. Rather, there’s a fact about what’s prime, and the pebblesorters care about that, and there’s a fact about what’s right, and humans care about that. We can be mistaken about what’s right, and we can have disagreements about what’s right, and we can change our minds. And given time and progress, we will hopefully get closer to understanding what’s right. And if the pebblesorters claim that they care about what’s right rather than what’s prime, they’re factually incorrect.
Of course — I was just doing my best to imagine the mindset of a non-religious person who believes in an objectively objective morality (i.e. that even in the absence of a deity, the universe still somehow imposes moral laws). Admittedly, I don’t encounter too many of those (people who think they’ve devised universally compelling moral arguments are more common; even big-O Objectivists seem to just be an overconfident version of that), but I do still meet them from time to time, e.g. people who manage to believe in things like “natural law” or “natural rights” (as facts about the universe rather than facts about human minds) without theistic belief.
All I was saying was that things like that are what the phrase “objective morality” make me think of, and that Eliezer’s conclusions are different enough that I’m not sure they quite fit in the same category. His may be an “objective morality” by our best definitions of “objective” and “morality”, but it could make people (especially new people) imagine all the wrong things.