I think the world modelling improvements from modern science and IQ raising social advances can be analytically separated from changes in our approach to welfare. As for non consensual wireheading, I am uncertain as to the moral status of this, so it seems like partially we just disagree about values. I am also uncertain as to the attitude of Stone Age people towards this—while your argument seems plausible, the fact that early philosophers like the Ancient Greeks were not pure hedonists in the wireheading sense but valued flourishing seems like evidence against this, suggesting that favoring non consensual wireheading is downstream of modern developments in utilitarianism.
favoring non consensual wireheading is downstream of modern developments in utilitarianism
Fair enough, I suppose. My point is more— Okay, let’s put the Stone Age people aside for a moment and think about the medieval people instead. Many of them were religious and superstitious and nationalistic, as the result of being raised on the diet of various unwholesome ideologies. These ideologies often had their own ideas of “the greater good” that they tried to sell people, ideas no less un-nice than non-consensual wireheading. Thus, a large fraction of humanity for the majority of its history endorsed views that would be catastrophic if scaled up.
I just assume this naturally extrapolates backwards to the Stone Age. Stone-age people had their own superstitions and spiritual traditions, and rudimentary proto-ideologies. I assume that these would also be catastrophic if scaled up.
Note that I’m not saying that the people in the past were literally alien, to the extent that they wouldn’t be able to converge towards modern moral views if we e. g. resurrected and educated one of them (and slightly intelligence-amplified them to account for worse nutrition, though I’m not as convinced that it’d be necessary as you), then let them do value reflection. But this process of “education” would need to be set up very carefully, in a way that might need to be “perfect”.
My argument is simply that if we granted godhood to one of these people and let them manage this process themselves, that will doom the light cone.
I think the world modelling improvements from modern science and IQ raising social advances can be analytically separated from changes in our approach to welfare. As for non consensual wireheading, I am uncertain as to the moral status of this, so it seems like partially we just disagree about values. I am also uncertain as to the attitude of Stone Age people towards this—while your argument seems plausible, the fact that early philosophers like the Ancient Greeks were not pure hedonists in the wireheading sense but valued flourishing seems like evidence against this, suggesting that favoring non consensual wireheading is downstream of modern developments in utilitarianism.
Fair enough, I suppose. My point is more— Okay, let’s put the Stone Age people aside for a moment and think about the medieval people instead. Many of them were religious and superstitious and nationalistic, as the result of being raised on the diet of various unwholesome ideologies. These ideologies often had their own ideas of “the greater good” that they tried to sell people, ideas no less un-nice than non-consensual wireheading. Thus, a large fraction of humanity for the majority of its history endorsed views that would be catastrophic if scaled up.
I just assume this naturally extrapolates backwards to the Stone Age. Stone-age people had their own superstitions and spiritual traditions, and rudimentary proto-ideologies. I assume that these would also be catastrophic if scaled up.
Note that I’m not saying that the people in the past were literally alien, to the extent that they wouldn’t be able to converge towards modern moral views if we e. g. resurrected and educated one of them (and slightly intelligence-amplified them to account for worse nutrition, though I’m not as convinced that it’d be necessary as you), then let them do value reflection. But this process of “education” would need to be set up very carefully, in a way that might need to be “perfect”.
My argument is simply that if we granted godhood to one of these people and let them manage this process themselves, that will doom the light cone.