I think the insights from Selectorate Theory imply that it is impossible to keep power without gradually growing more corrupt, in the sense of appeasing the “Winning Coalition” with private goods. No matter what your terminal goals, more power, and power kept for longer, is a convergent instrumental goal, and one which usually takes so much effort to achieve that you gradually lose sight of your terminal goals too, compromising ethics in the short term in the name of an “ends justify the means” long term (which often never arrives).
So yeah, I think that powerful humans are unaligned by default, as our ancestors who rejected all attempts to form hierarchies for tens of thousands of years before finally succumbing to the first nationstates may attest.
I think the insights from Selectorate Theory imply that it is impossible to keep power without gradually growing more corrupt, in the sense of appeasing the “Winning Coalition” with private goods. No matter what your terminal goals, more power, and power kept for longer, is a convergent instrumental goal, and one which usually takes so much effort to achieve that you gradually lose sight of your terminal goals too, compromising ethics in the short term in the name of an “ends justify the means” long term (which often never arrives).
So yeah, I think that powerful humans are unaligned by default, as our ancestors who rejected all attempts to form hierarchies for tens of thousands of years before finally succumbing to the first nationstates may attest.
Seems like there are two meanings of “power” that get conflated, because in real life it is a combination of both:
to be able to do whatever you want;
to successfully balance the interests of others, so that you can stay nominally on the top.
Good point. Perhaps there’s some people who would be corrupted by the realities of human politics, but not by e.g. ascension to superintelligence.