You didn’t talk about counterexamples: for example, lots of nations like the US have few corrupt politicians. There’s an availability effect here. Philanthropists are boring, bad boys are interesting.
I actually agree this is a big danger, but I’m not sure it’s unsolvable with the right individuals. Does power corrupt women, or lifelong meditators, or hopelessly geeky folks, or people whose entire career has consisted of researching this sort of thing, or people who fall into all four categories?
WRT the Stanford Prison Experiment: I’ve hung out with some elite university undergraduates and mostly they seem full of youthful foolishness and much less reflective than median Less Wrong user. All your other examples feature even worse candidates where power corruption is concerned. (Also, iirc the portion resisting authority in the Milgram experiment was decently large.)
And you haven’t discussed the problem of corruption when an entire group, not self-selected on the basis of power seeking tendencies, receives committee style power. It seems like this could significantly curtail power-seeking tendencies.
My own views on politicians in that while their behavior is genuinely better than that of power holders in many other governments today and throughout history, their behavior is also far from saintly, and the fact that they operate in a system where the incentives favor less-bad behavior.
Also, it seems to me that a group trying to build FOOM-capable AI is by definition self-selected on the basis of power seeking tendencies. But maybe they could hand off final decision making authority to someone else at last stage?
Well, maybe they wouldn’t be. But if you think self-selection based on power-seeking is important, then maybe they would. IDK, I was just responding to John.
You didn’t talk about counterexamples: for example, lots of nations like the US have few corrupt politicians. There’s an availability effect here. Philanthropists are boring, bad boys are interesting.
I actually agree this is a big danger, but I’m not sure it’s unsolvable with the right individuals. Does power corrupt women, or lifelong meditators, or hopelessly geeky folks, or people whose entire career has consisted of researching this sort of thing, or people who fall into all four categories?
WRT the Stanford Prison Experiment: I’ve hung out with some elite university undergraduates and mostly they seem full of youthful foolishness and much less reflective than median Less Wrong user. All your other examples feature even worse candidates where power corruption is concerned. (Also, iirc the portion resisting authority in the Milgram experiment was decently large.)
And you haven’t discussed the problem of corruption when an entire group, not self-selected on the basis of power seeking tendencies, receives committee style power. It seems like this could significantly curtail power-seeking tendencies.
My own views on politicians in that while their behavior is genuinely better than that of power holders in many other governments today and throughout history, their behavior is also far from saintly, and the fact that they operate in a system where the incentives favor less-bad behavior.
Also, it seems to me that a group trying to build FOOM-capable AI is by definition self-selected on the basis of power seeking tendencies. But maybe they could hand off final decision making authority to someone else at last stage?
Why would or should they believe someone else would make better decisions then they would?
Well, maybe they wouldn’t be. But if you think self-selection based on power-seeking is important, then maybe they would. IDK, I was just responding to John.