So dath Ilan I understand is the thought experiment of “every human has about as much intelligence as Eliezer Yudnowsky”.
Starting with that assumption—the flaw is that I think a lot of the issues with current civilization isn’t that people are stupid, it’s Moloch.
The rules of the adversarial game creates situations where every actor is stuck in an inadequate equilibrium. No one has the power to fix anything, because each actor is just doing their own role and acting in their own interests.
Making the actors smarter doesn’t help—they just try hard their jobs even more. This might make the situation worse.
For an example: the FDA doesn’t exist to help human beings live longer, healthier lives. It exists to ensure every drug is “safe and effective”. Making then smarter means they allow even less errors in drug applications. But drug company workers also are smarter and make less obvious errors and cover up any lies in their clinical trial reports better, since their role is to get a drug approved so their parent company doesn’t go bankrupt.
So you are stuck in the same inadequate equilibrium where everyone is doing their role and the actual people humans should care about—human patients—suffers.
So dath Ilan I understand is the thought experiment of “every human has about as much intelligence as Eliezer Yudnowsky”.
Starting with that assumption—the flaw is that I think a lot of the issues with current civilization isn’t that people are stupid, it’s Moloch.
The rules of the adversarial game creates situations where every actor is stuck in an inadequate equilibrium. No one has the power to fix anything, because each actor is just doing their own role and acting in their own interests.
Making the actors smarter doesn’t help—they just try hard their jobs even more. This might make the situation worse.
For an example: the FDA doesn’t exist to help human beings live longer, healthier lives. It exists to ensure every drug is “safe and effective”. Making then smarter means they allow even less errors in drug applications. But drug company workers also are smarter and make less obvious errors and cover up any lies in their clinical trial reports better, since their role is to get a drug approved so their parent company doesn’t go bankrupt.
So you are stuck in the same inadequate equilibrium where everyone is doing their role and the actual people humans should care about—human patients—suffers.