Have you read the Blue-Minimizing Robot? Early Homo sapiens was in the simple environment where it seemed like they were “minimizing blue,” i.e. maximizing genetic fitness. Now, you might say, it seems like our behavior indicates preferences for happiness, meaning, validation, etc, but really that’s just an epiphenomenon no more meaningful than our previous apparent preference for genetic fitness.
However, there is an important difference between us and the blue-minimizing robot, which is that we have a much better model of the world, and within that model of the world we do a much better job than the robot at making plans. What kind of plans? The thing that motivates our plans is, from a purely functional perspective, our preferences. And this thing isn’t all that different in modern humans versus hunter-gatherers. We know, we’ve talked to them. There have been some alterations due to biology and culture, but not as much as there could have been. Hunter-gatherers still like happiness, meaning, validation, etc.
What seems to have happened is that evolution stumbled upon a set of instincts that produced human planning, and that in the ancestral environment this correlated well with genetic fitness, but in the modern environment this diverges even though the planning process itself hasn’t changed all that much. There are certain futuristic scenarios that could seriously disrupt the picture of human values I’ve given, but I don’t think it’s the default, particularly if there aren’t any optimization processes much stronger than humans running around.
Yes, and that’s closely related to the point I made about ” we’re adaptation executioners, not fitness maximizers.”
My point is a step further, I think—I’m asking what decides which things we plan to do? It’s obviously our “preferences,” but if we’ve already destroyed everything blue, the next priority is very underspecified.
Have you read the Blue-Minimizing Robot? Early Homo sapiens was in the simple environment where it seemed like they were “minimizing blue,” i.e. maximizing genetic fitness. Now, you might say, it seems like our behavior indicates preferences for happiness, meaning, validation, etc, but really that’s just an epiphenomenon no more meaningful than our previous apparent preference for genetic fitness.
However, there is an important difference between us and the blue-minimizing robot, which is that we have a much better model of the world, and within that model of the world we do a much better job than the robot at making plans. What kind of plans? The thing that motivates our plans is, from a purely functional perspective, our preferences. And this thing isn’t all that different in modern humans versus hunter-gatherers. We know, we’ve talked to them. There have been some alterations due to biology and culture, but not as much as there could have been. Hunter-gatherers still like happiness, meaning, validation, etc.
What seems to have happened is that evolution stumbled upon a set of instincts that produced human planning, and that in the ancestral environment this correlated well with genetic fitness, but in the modern environment this diverges even though the planning process itself hasn’t changed all that much. There are certain futuristic scenarios that could seriously disrupt the picture of human values I’ve given, but I don’t think it’s the default, particularly if there aren’t any optimization processes much stronger than humans running around.
Yes, and that’s closely related to the point I made about ” we’re adaptation executioners, not fitness maximizers.”
My point is a step further, I think—I’m asking what decides which things we plan to do? It’s obviously our “preferences,” but if we’ve already destroyed everything blue, the next priority is very underspecified.