The “4%” wasn’t addressing the question of “where do humans get their values from?” It was addressing “When trying to make predictions about AGI outcomes, how much weight should we assign these various sources of evidence?”
Correct me if I’m wrong, but this would assume that if you could somehow make a human level intelligence snake and raise it in human society (let’s pretend nobody considers it weird that there’s a snake taking Chemistry class with them), then that snake would be 96% aligned with humanity?
My perspective isn’t blank slatism. The genome has various leavers by which it can influence the sorts of values that a human forms. E.g., the snake wouldn’t have human-like reward circuitry, so it would probably learn to value very different things than a human which went through the same experiences. For more on this, see: “Learning from scratch” in the brain.
E.g., the snake wouldn’t have human-like reward circuitry, so it would probably learn to value very different things than a human which went through the same experiences.
So in this case I think we then agree. But it seems a bit at odds with the 4% weighting of genetic roots. If we agree the snake would exhibit very different values despite experiencing the ‘human learning’ part then shouldn’t this adjust the 60% weight you grant that? Seems the evolutionary roots made all the difference for the snake. Which is the whole point about initial AGI alignment having to be exactly right.
Otherwise I understand your post to be ‘for humans, how much of human value is derived from evolution vs learning’. But that’s using humans as evidence who are human to begin with.
The “4%” wasn’t addressing the question of “where do humans get their values from?” It was addressing “When trying to make predictions about AGI outcomes, how much weight should we assign these various sources of evidence?”
My perspective isn’t blank slatism. The genome has various leavers by which it can influence the sorts of values that a human forms. E.g., the snake wouldn’t have human-like reward circuitry, so it would probably learn to value very different things than a human which went through the same experiences. For more on this, see: “Learning from scratch” in the brain.
So in this case I think we then agree. But it seems a bit at odds with the 4% weighting of genetic roots. If we agree the snake would exhibit very different values despite experiencing the ‘human learning’ part then shouldn’t this adjust the 60% weight you grant that? Seems the evolutionary roots made all the difference for the snake. Which is the whole point about initial AGI alignment having to be exactly right.
Otherwise I understand your post to be ‘for humans, how much of human value is derived from evolution vs learning’. But that’s using humans as evidence who are human to begin with.