The claim about Stone Age people seems probably false to me—I think if Stone Age people could understand what they were actually doing (not at the level of psychology or morality, but at the purely “physical” level), they would probably do lots of very nice things for their friends and family, in particular give them a lot of resources. However, even if it is true, I don’t think the reason we have gotten better is because of philosophy—I think it’s because we’re smarter in a more general way. Stone Age people were uneducated and had less good nutrition than us; they were literally just stupid.
Education is part of what I’m talking about. Modern humans iterate on the output of thousands of years of cultural evolution, their basic framework of how the world works is drastically different from the ancestral ones. Implicit and explicit lessons of how to care about people without e. g. violating their agency come part and parcel with it.
I think if Stone Age people could understand what they were actually doing (not at the level of psychology or morality, but at the purely “physical” level), they would probably do lots of very nice things for their friends and family
At the basic level, why do you think that their idea of “nice things” would be nuanced enough to understand that, say, non-consensual wireheading is not a nice thing? Some modern people don’t.
Stone Age people didn’t live a very comfortable life by modern standards, the experience of pleasure and escape from physical ailments would be common aspirations, while the high-cognitive-tech ideas of “self-actualization” would make no native sense to them. Why would a newly-ascended Stone Age god not assume that making everyone experience boundless pleasure free of pain forever is not the greatest thing there could possibly be? Would it occur to that god to even think carefully about whether such assumptions are right?
Edit: More than that, ancient people’s world-models are potentially much more alien and primitive than we can easily imagine. I refer you to the speculations in the section 2 here. The whole “voices of the gods” thing in the rest of the post is probably wrong, but I find it easy to believe that the basic principles of theory-of-mind that we take for granted are not something any human would independently invent. And if so, who knows what people without it would consider the maximally best way to be nice to someone?
I think the world modelling improvements from modern science and IQ raising social advances can be analytically separated from changes in our approach to welfare. As for non consensual wireheading, I am uncertain as to the moral status of this, so it seems like partially we just disagree about values. I am also uncertain as to the attitude of Stone Age people towards this—while your argument seems plausible, the fact that early philosophers like the Ancient Greeks were not pure hedonists in the wireheading sense but valued flourishing seems like evidence against this, suggesting that favoring non consensual wireheading is downstream of modern developments in utilitarianism.
favoring non consensual wireheading is downstream of modern developments in utilitarianism
Fair enough, I suppose. My point is more— Okay, let’s put the Stone Age people aside for a moment and think about the medieval people instead. Many of them were religious and superstitious and nationalistic, as the result of being raised on the diet of various unwholesome ideologies. These ideologies often had their own ideas of “the greater good” that they tried to sell people, ideas no less un-nice than non-consensual wireheading. Thus, a large fraction of humanity for the majority of its history endorsed views that would be catastrophic if scaled up.
I just assume this naturally extrapolates backwards to the Stone Age. Stone-age people had their own superstitions and spiritual traditions, and rudimentary proto-ideologies. I assume that these would also be catastrophic if scaled up.
Note that I’m not saying that the people in the past were literally alien, to the extent that they wouldn’t be able to converge towards modern moral views if we e. g. resurrected and educated one of them (and slightly intelligence-amplified them to account for worse nutrition, though I’m not as convinced that it’d be necessary as you), then let them do value reflection. But this process of “education” would need to be set up very carefully, in a way that might need to be “perfect”.
My argument is simply that if we granted godhood to one of these people and let them manage this process themselves, that will doom the light cone.
The claim about Stone Age people seems probably false to me—I think if Stone Age people could understand what they were actually doing (not at the level of psychology or morality, but at the purely “physical” level), they would probably do lots of very nice things for their friends and family, in particular give them a lot of resources. However, even if it is true, I don’t think the reason we have gotten better is because of philosophy—I think it’s because we’re smarter in a more general way. Stone Age people were uneducated and had less good nutrition than us; they were literally just stupid.
Education is part of what I’m talking about. Modern humans iterate on the output of thousands of years of cultural evolution, their basic framework of how the world works is drastically different from the ancestral ones. Implicit and explicit lessons of how to care about people without e. g. violating their agency come part and parcel with it.
At the basic level, why do you think that their idea of “nice things” would be nuanced enough to understand that, say, non-consensual wireheading is not a nice thing? Some modern people don’t.
Stone Age people didn’t live a very comfortable life by modern standards, the experience of pleasure and escape from physical ailments would be common aspirations, while the high-cognitive-tech ideas of “self-actualization” would make no native sense to them. Why would a newly-ascended Stone Age god not assume that making everyone experience boundless pleasure free of pain forever is not the greatest thing there could possibly be? Would it occur to that god to even think carefully about whether such assumptions are right?
Edit: More than that, ancient people’s world-models are potentially much more alien and primitive than we can easily imagine. I refer you to the speculations in the section 2 here. The whole “voices of the gods” thing in the rest of the post is probably wrong, but I find it easy to believe that the basic principles of theory-of-mind that we take for granted are not something any human would independently invent. And if so, who knows what people without it would consider the maximally best way to be nice to someone?
I think the world modelling improvements from modern science and IQ raising social advances can be analytically separated from changes in our approach to welfare. As for non consensual wireheading, I am uncertain as to the moral status of this, so it seems like partially we just disagree about values. I am also uncertain as to the attitude of Stone Age people towards this—while your argument seems plausible, the fact that early philosophers like the Ancient Greeks were not pure hedonists in the wireheading sense but valued flourishing seems like evidence against this, suggesting that favoring non consensual wireheading is downstream of modern developments in utilitarianism.
Fair enough, I suppose. My point is more— Okay, let’s put the Stone Age people aside for a moment and think about the medieval people instead. Many of them were religious and superstitious and nationalistic, as the result of being raised on the diet of various unwholesome ideologies. These ideologies often had their own ideas of “the greater good” that they tried to sell people, ideas no less un-nice than non-consensual wireheading. Thus, a large fraction of humanity for the majority of its history endorsed views that would be catastrophic if scaled up.
I just assume this naturally extrapolates backwards to the Stone Age. Stone-age people had their own superstitions and spiritual traditions, and rudimentary proto-ideologies. I assume that these would also be catastrophic if scaled up.
Note that I’m not saying that the people in the past were literally alien, to the extent that they wouldn’t be able to converge towards modern moral views if we e. g. resurrected and educated one of them (and slightly intelligence-amplified them to account for worse nutrition, though I’m not as convinced that it’d be necessary as you), then let them do value reflection. But this process of “education” would need to be set up very carefully, in a way that might need to be “perfect”.
My argument is simply that if we granted godhood to one of these people and let them manage this process themselves, that will doom the light cone.