I’m a bit confused about why you think it’s so clearly a bad idea, your points weren’t elaborated at all, so I’d absolutely love some elaboration by you or some of the people that voted up your comment, because clearly I’m missing something.
on the reduction of chance of FAI being developed, sure, some of this of course would happen, but slowing down development of solutions to a problem (alignment problem) whilst slowing down growth of the problem itself even more is surely net good for stopping the problem? Especially if you’re really worried about the problem and worried it’d happen faster than you could think of good solutions for!
waiting for elaboration on the suffering point but let’s assume you’ve got good reasons there
Alright, if we want to estimate the likely effects of allowing government regulation of AI, it’s worth considering the effects of government regulation of everything else. The FDA’s efforts to slow the adoption of new medicines kill far more people than they save (at least according to SlateStarCodex, which has a lot of excellent material on the topic). It is not uncommon for them to take actions that do not even pretend to be about patient safety, such as banning a drug because initial trials make it appear so helpful that it would be “unethical” to have a control group that was denied it in further studies, but apparently not so helpful that it’s worth allowing the general public access. I highly recommend Zvi Moskowitz’ blog posts on the subject; he’s collected more sources and examples on the topic than this margin can contain.
There is a very common reaction I have noticed to these sorts of events, where most people brush them off as “just how the world works”. A patient dying due to having been deliberately refused medicine is treated as a tragedy, but no one is actually at fault. Meanwhile, a patient who is slightly inconvenienced by an officially approved treatment is treated as strong evidence that we need more medical regulation. Certainly this reaction is not universal, but it’s common enough to create an inferential gap between general perceptions of regulation and a claim like “the FDA are mass murderers”. However, whether or not you want to call it murder, the real-world effect of medical regulation is primarily to make more people sick and dead.
This raises two concerns about having an “FDA for AI”, as the original post recommends. First, that the same sorts of errors would occur, even in the absence of malice. And secondly, that malice would in fact be present, and create deliberate problems for the general population. How likely is this?
Enough errors would almost certainly occur in AI regulation to make it net negative. Even leaving aside the alarming example of the FDA, we can consider countless other examples of what regulation looks like in the real world. For instance, ALARA laws demand nuclear energy have radiation emissions As Low As Reasonably Possible, where Reasonably Possible means that if nuclear power becomes cheaper than its competitors, clearly it can afford to spend more money on shielding. There is no cost-benefit analysis here, just the attempt to look good by “standing up to nuclear risks”. Never mind the fact that insisting that nuclear be made always more expensive cripples a likely source of both economic growth and carbon-free energy, and must at some point become unreasonable, as the laws are never lifted despite how low radiation emissions may become. Or actively counterproductive responses to Covid (again, Zvi explains this topic in as much detail as you care to read, so I will not reproduce it here, but you will likely greatly enjoy his posts on the subject). Or laws that demand everything from not using foreign dredging ships to expand our ports, to getting a college degree before braiding hair. And I defy anyone exposed to American public schools to claim that these are in any way a sane use of power.
All of these examples, moreover, are dealing with subjects relatively well understood by regulators. While the precise details of medical safety might be beyond a bureaucrat, they generally have a concept of “safe and effective” vs “unsafe or ineffective”. Likewise with nuclear power and college degrees. But AI risks are complex and counterintuitive-try explaining to a congressman that your system is totally safe and productive unless it develops mesa optimizers, or unless someone builds an agentic version. Insofar as the counterproductive results of regulation are the result of misunderstandings and poor incentives rather than deliberate evil, a field where there is bound to be vastly more misunderstanding should be at least as prone to regulation backfiring.
As for deliberate evil, it’s worth considering the track record of regimes both historically and in the present day. Even leaving aside the horrors of feudalism, Nazism and twentieth century Communism, Putin is currently pursuing a war of aggression complete with war-crimes, Xi is inflicting unspeakable tortures on captive Uighurs, Kim Jong-Un is maintaining a state which keeps its subjects in grinding poverty and demands constant near-worship, and the list continues. It should be quite obvious, I hope, why the idea of such a regime gaining controllable AI would produce an astronomical suffering risk.
The obvious counterargument here is to place liberal democratic regimes in a special category, such that one wouldn’t expect them to engage in the same sorts of abuses. After all, the proposal is to create an FDA for AI, not a North Korean Politburo, and the horrors I just cited are all from non liberal democratic regimes. However, there are multiple problems with that.
The first problem is that while acts like denying people medicine and letting them die quietly are less flashy than locking Uighurs up in camps, they are no less evil. If the actions of the FDA were taken by a political opponent, we would have no hesitation in declaring them atrocities. It seems unwise to conclude that people who take blatant actions to kill innocents for political convenience would be safe custodians of AI, especially if that AI advanced to the point that what accountability to the public that still exists for democratic regulators was lost.
The second problem is that to the extent that liberal democracies behave better than their autocratic counterparts, this is often because of better incentives, and those incentives would not necessarily apply to AI governance. Western nations tend to have both liberal democracy and the rest of the Enlightenment values: individualism, free markets, tolerance and the like. These other values allow them to be rich and stable, removing many incentives for vicious acts of desperation, as well as creating an electorate which may punish certain types of abuses. However, the electorate does not understand AI, removing the possibility for meaningful checks on poor decisions (and given the other regulatory failures it’s worth questioning the extent to which electoral checks on poor decisions actually occur. Does that model match the world we see?) Moreover, the possibility of a rival nation obtaining unchallengeable power might be expected to create the sort of desperation we normally associate with rogue states.
Finally, the third problem with that argument is that it borders on reference class tennis. Citing liberal democracy as a special category doesn’t make sense without a proposed mechanism for such regimes actually making better decisions regarding AI! I am reminded of Eliezer’s story about a speaker at the Singularity Summit calling for “democratic development of artificial intelligence” without having any concept of how to apply democracy to said development. The speaker appeared to simply have positive affect around citing democracy, no more, no less. Unless the voters are specifically concerned about AI, and knowledgeable enough for that concern to do more good than harm, and officials are selected as a result of this who are vastly more moral than almost all other officials, one wouldn’t expect liberal democracy to get better results with AI regulation.
There is an inherent risk in discussing politics here, both in the possibility of getting mind-killed and the chance that differences in our background political assumptions may distract from the main point. If we have different priors on the likelihood of various regime types acting wisely and well, we should try to keep that from interfering with the discussion on the specific question of AI regulation and the more general question of how to obtain positive results from AI. (Not that I’m averse to discussing political questions if you want to, but that should for the most part not be in this thread.) But hopefully this should clarify to a degree why I anticipate both severe X risks and S risks from most attempts at AI regulation without being too inflammatory.
Thank you for your elaboration, I appreciate it a lot, and upvoted for the effort. Here are your clearest points paraphrased as I understand them (sometimes just using your words), and my replies:
The FDA is net negative for health, therefore creating an FDA-for-AI would be likely net negative for the AI challenges.
I don’t think you can come to this conclusion, even if I agree with the premise. The counterfactuals are very different. With drugs the counterfactual of no FDA might be some people get more treatments, and some die but many don’t, and they were sick anyway so need to do something, and maybe fewer die than do with the FDA around, so maybe the existence of the FDA compared to the counterfactual is net bad. I won’t dispute this, I don’t know enough about it. However, the counterfactual in AI is different. If unregulated, AI progress steams on ahead, competition over the high rewards is high, and if we don’t have good safety plan (which we don’t) then maybe we all die at some point, who knows when. However, if an FDA-for-AI creates bad regulation (as long as it’s not bad enough to cause AI regulation winter) then it starts slowing down that progress. Maybe it’s bad for, idk, the diseases that could have been solved during the 10 years slowing down from when AI would have solved cancer vs not, and that kind of thing, but it’s nowhere near as bad as the counterfactual! These scenarios are different and not comparable, because the counterfactual of no FDA is not as bad as the counterfactual of no AI regulator.
Enough errors would almost certainly occur in AI regulation to make it net negative.
You gave a bunch of examples from non-AI regulation of bad regulation (I am not going to bother to think about whether I agree that they are bad regulation as it’s not cruxy) - but you didn’t explain how exactly errors lead to making AI regulation net negative? Again I think similar to the previous claim, the counterfactuals likely make this not hold.
...a field where there is bound to be vastly more misunderstanding should be at least as prone to regulation backfiring
That is an interesting claim, I am not sure what makes you think it’s obviously true, as it depends what your goal is. My understanding of the OP is that the goal of the type of regulation they advocate is simply to slow down AI development, nothing more, nothing less. If the goal is to do good regulation of AI, that’s totally different. Is there a specific way in which you imagine it backfiring for the goal of simply slowing down AI progress?
...an [oppressive] regime gaining controllable AI would produce an astronomical suffering risk.
I am unsure what point you were making in the paragraph about evil. Was it about another regime getting there first that might not do safety? For response, see the OP Objection 4 which I share and added additional reason for that not being a real worry in this world.
...unwise to think that people who take blatant actions to kill innocents for political convenience would be safe custodians of AI..
I don’t think it’s fair to say regulators would be a custodian. They have a special kind of lever called “slow things down”, and that lever does not mean that they can, for example, seize and start operating the AI. It is not in their power to do that, legally, nor do they have the capability to do anything with it. We are talking here about slowing things down before AGI, not post AGI.
the electorate does not understand AI
Answer is same as my answer to 3. and also similar to OP Objection 1.
And finally to reply to this: “hopefully this should clarify to a degree why I anticipate both severe X risks and S risks from most attempts at AI regulation”
Basically, no, it doesn’t really clarify it. You started off with a premise I agreed with or at least do not know enough to refute, that the FDA may be net negative, and then drew a conclusion that I disagree with (see 1. above), and then all your other points were assuming that conclusion, so I couldn’t really follow. I tried to pick out bits that seemed like possible key points and reply, but yeah I think you’re pretty confused.
What do you think of my reply to 1. - the counterfactuals being different. I think that’s the best way to progress the conversation.
(Warning: this response is long and much of it is covered by what Tamgen and others have said. )
The way I understand your fears, they fall into four main categories. In the order you raise them and, I think, in order of importance these concerns are as follows:
1) Regulations tend to cause harm to people, therefore we should not regulate AI.
I completely agree that a Federal AI Regulatory Commission will impose costs in the form of human suffering. This is inevitable, since Policy Debates Should Not Appear One Sided. Maybe in the world without the FAIRC, some AI Startup cures Alzheimer’s or even aging a good decade before AGI. In the world with FAIRC, we risk condemning all those people to dementia and decrepitude. This is quite similar to FDA unintended consequences.
Response:
You suggest that the OP was playing reference class tennis, but to me looking at the problem in terms of “regulators” and “harm” is the wrong reference class. They are categories that do not help us predict the answer to the one question we care about most: what is the impact on timelines to AGI?
If we zoom in closer to the object level, it becomes clear that the mechanism by which regulators harm the public is by impeding production. Using Paul Christiano’s rules of reference class tennis, “regulation impedes production” is a more probable narrative (i.e. supported by greater evidence, albeit not simpler) than simply “regulation always causes harm.” At the object level, we see this directly as when the FDA shoots fines anyone with the temerity to produce cheaper EpiPens, the nuclear regulatory commission doesn’t let anyone build nuclear reactors, etc. Or it can happen indirectly as a drag on innovation. To count the true cost of the FDA, we need to know how many wondrous medical breakthroughs we’ve already made on Earth prime.
But if you truly believe that AGI represents an existential threat, and that at present innovation speeds AGI happens before Alignment, then AI progress (even when it solves Alzheimers) is on net a negative. The lives saved by Alzheimer’s have to be balanced against human extinction—and the balance leaves us way, way in the red. This means that all the regulatory failure modes you cite in your reply become net beneficial. We want to impede production.
By way of analogy, it would be as if Pfizer were nearly guaranteed to be working its way toward making a pill that would instantly wipe out humanity; or if Nuclear power actually was as dangerous as its detractors believe! Under such scenarios, the FDA is your best friend. Unfortunately, that is where we stand with AI.
To return to the key question: once it is clear that, at a mechanical level, the things that regulatory agencies do are to impede production, it also becomes clear that regulation is likely to lengthen AGI timelines.
2) The voting public is insufficiently knowledgeable about AI.
I’m not sure I understand the objection here. The government regulates tons of things that the electorate doesn’t understand. In fact, ideally that is what regulatory agencies do. They say, “hey we are a democracy, but you, the demos, don’t understand how education works so we need a department of education.” This is often self-serving patronage, but the general point stands that the way regulatory agencies come into being in practice is not because the electorate achieves subject-area expertise. I can see a populist appeal for a Manhattan project to speed up AI in order to “beat China” (or whatever enemy du jour), but this is not the sort of thing that regulators in permanent bureaucracies do. (Just look at operation “warp speed”; quite apart from the irony in the name, the FDA and the CDC had to be dragged kicking and screaming to do it.)
3) Governments might use AI to do evil things
In your response you write:
As for deliberate evil, it’s worth considering the track record of regimes both historically and in the present day. Even leaving aside the horrors of feudalism, Nazism and twentieth century Communism, Putin is currently pursuing a war of aggression complete with war-crimes, Xi is inflicting unspeakable tortures on captive Uighurs, Kim Jong-Un is maintaining a state which keeps its subjects in grinding poverty and demands constant near-worship, and the list continues. It should be quite obvious, I hope, why the idea of such a regime gaining controllable AI would produce an astronomical suffering risk.
I agree, of course, that these are all terrible evils wrought by governments. But I’m not sure what it has to do with regulation of AI. The historical incidents you cite would be relevant if the Holocaust were perpetrated by the German Bureau of Chemical Safety or if the Uighurs were imprisoned by the Chinese Ethnic Affairs Commission. Permanent regulatory bureaucracies are not and never have been responsible for (or even capable of) mission-driven atrocities. They do commit atrocities, but only by preventing access to useful goods (i.e. impeding production).
Finally, one sentence in this section sticks out and makes me think we are talking past each other. You write:
the idea of such a regime gaining controllable AI would produce an astronomical suffering risk
By my lights, this would be a WONDERFUL problem to have. An AI that was controllable by anyone (including Kim Jung-Un, Pol Pot, or Hitler) would, in my estimation, be preferable to a completely unaligned paper clip maximizer. Maybe we disagree here?
4) Liberal democracies are not magic, and we can’t expect them to make the right decisions just because of our own political values.
I don’t think my OP mentioned liberal democracy, but if I gave that impression then you are quite right I did so in error. You may be referring to my point about China. I did not mean to imply a normative superiority of American or any other democracy, and I regret the lack of clarity. My intent was to make a positive observation that governments do, in fact, mimic each other’s regulatory growth. Robin Hanson makes a similar point; that governments copy each other largely because of institutional and informal status associations. This observation is neutral with regard to political system. If we announce a FAIRC, I predict that China will follow, and with due haste.
I’m a bit confused about why you think it’s so clearly a bad idea, your points weren’t elaborated at all, so I’d absolutely love some elaboration by you or some of the people that voted up your comment, because clearly I’m missing something.
on the reduction of chance of FAI being developed, sure, some of this of course would happen, but slowing down development of solutions to a problem (alignment problem) whilst slowing down growth of the problem itself even more is surely net good for stopping the problem? Especially if you’re really worried about the problem and worried it’d happen faster than you could think of good solutions for!
waiting for elaboration on the suffering point but let’s assume you’ve got good reasons there
Hey, that’s a great question. When I get a bit more time I’ll write a clarification. Sorry for the delay.
No worries, thank you, I look forward to it
Alright, if we want to estimate the likely effects of allowing government regulation of AI, it’s worth considering the effects of government regulation of everything else. The FDA’s efforts to slow the adoption of new medicines kill far more people than they save (at least according to SlateStarCodex, which has a lot of excellent material on the topic). It is not uncommon for them to take actions that do not even pretend to be about patient safety, such as banning a drug because initial trials make it appear so helpful that it would be “unethical” to have a control group that was denied it in further studies, but apparently not so helpful that it’s worth allowing the general public access. I highly recommend Zvi Moskowitz’ blog posts on the subject; he’s collected more sources and examples on the topic than this margin can contain.
There is a very common reaction I have noticed to these sorts of events, where most people brush them off as “just how the world works”. A patient dying due to having been deliberately refused medicine is treated as a tragedy, but no one is actually at fault. Meanwhile, a patient who is slightly inconvenienced by an officially approved treatment is treated as strong evidence that we need more medical regulation. Certainly this reaction is not universal, but it’s common enough to create an inferential gap between general perceptions of regulation and a claim like “the FDA are mass murderers”. However, whether or not you want to call it murder, the real-world effect of medical regulation is primarily to make more people sick and dead.
This raises two concerns about having an “FDA for AI”, as the original post recommends. First, that the same sorts of errors would occur, even in the absence of malice. And secondly, that malice would in fact be present, and create deliberate problems for the general population. How likely is this?
Enough errors would almost certainly occur in AI regulation to make it net negative. Even leaving aside the alarming example of the FDA, we can consider countless other examples of what regulation looks like in the real world. For instance, ALARA laws demand nuclear energy have radiation emissions As Low As Reasonably Possible, where Reasonably Possible means that if nuclear power becomes cheaper than its competitors, clearly it can afford to spend more money on shielding. There is no cost-benefit analysis here, just the attempt to look good by “standing up to nuclear risks”. Never mind the fact that insisting that nuclear be made always more expensive cripples a likely source of both economic growth and carbon-free energy, and must at some point become unreasonable, as the laws are never lifted despite how low radiation emissions may become. Or actively counterproductive responses to Covid (again, Zvi explains this topic in as much detail as you care to read, so I will not reproduce it here, but you will likely greatly enjoy his posts on the subject). Or laws that demand everything from not using foreign dredging ships to expand our ports, to getting a college degree before braiding hair. And I defy anyone exposed to American public schools to claim that these are in any way a sane use of power.
All of these examples, moreover, are dealing with subjects relatively well understood by regulators. While the precise details of medical safety might be beyond a bureaucrat, they generally have a concept of “safe and effective” vs “unsafe or ineffective”. Likewise with nuclear power and college degrees. But AI risks are complex and counterintuitive-try explaining to a congressman that your system is totally safe and productive unless it develops mesa optimizers, or unless someone builds an agentic version. Insofar as the counterproductive results of regulation are the result of misunderstandings and poor incentives rather than deliberate evil, a field where there is bound to be vastly more misunderstanding should be at least as prone to regulation backfiring.
As for deliberate evil, it’s worth considering the track record of regimes both historically and in the present day. Even leaving aside the horrors of feudalism, Nazism and twentieth century Communism, Putin is currently pursuing a war of aggression complete with war-crimes, Xi is inflicting unspeakable tortures on captive Uighurs, Kim Jong-Un is maintaining a state which keeps its subjects in grinding poverty and demands constant near-worship, and the list continues. It should be quite obvious, I hope, why the idea of such a regime gaining controllable AI would produce an astronomical suffering risk.
The obvious counterargument here is to place liberal democratic regimes in a special category, such that one wouldn’t expect them to engage in the same sorts of abuses. After all, the proposal is to create an FDA for AI, not a North Korean Politburo, and the horrors I just cited are all from non liberal democratic regimes. However, there are multiple problems with that.
The first problem is that while acts like denying people medicine and letting them die quietly are less flashy than locking Uighurs up in camps, they are no less evil. If the actions of the FDA were taken by a political opponent, we would have no hesitation in declaring them atrocities. It seems unwise to conclude that people who take blatant actions to kill innocents for political convenience would be safe custodians of AI, especially if that AI advanced to the point that what accountability to the public that still exists for democratic regulators was lost.
The second problem is that to the extent that liberal democracies behave better than their autocratic counterparts, this is often because of better incentives, and those incentives would not necessarily apply to AI governance. Western nations tend to have both liberal democracy and the rest of the Enlightenment values: individualism, free markets, tolerance and the like. These other values allow them to be rich and stable, removing many incentives for vicious acts of desperation, as well as creating an electorate which may punish certain types of abuses. However, the electorate does not understand AI, removing the possibility for meaningful checks on poor decisions (and given the other regulatory failures it’s worth questioning the extent to which electoral checks on poor decisions actually occur. Does that model match the world we see?) Moreover, the possibility of a rival nation obtaining unchallengeable power might be expected to create the sort of desperation we normally associate with rogue states.
Finally, the third problem with that argument is that it borders on reference class tennis. Citing liberal democracy as a special category doesn’t make sense without a proposed mechanism for such regimes actually making better decisions regarding AI! I am reminded of Eliezer’s story about a speaker at the Singularity Summit calling for “democratic development of artificial intelligence” without having any concept of how to apply democracy to said development. The speaker appeared to simply have positive affect around citing democracy, no more, no less. Unless the voters are specifically concerned about AI, and knowledgeable enough for that concern to do more good than harm, and officials are selected as a result of this who are vastly more moral than almost all other officials, one wouldn’t expect liberal democracy to get better results with AI regulation.
There is an inherent risk in discussing politics here, both in the possibility of getting mind-killed and the chance that differences in our background political assumptions may distract from the main point. If we have different priors on the likelihood of various regime types acting wisely and well, we should try to keep that from interfering with the discussion on the specific question of AI regulation and the more general question of how to obtain positive results from AI. (Not that I’m averse to discussing political questions if you want to, but that should for the most part not be in this thread.) But hopefully this should clarify to a degree why I anticipate both severe X risks and S risks from most attempts at AI regulation without being too inflammatory.
Thank you for your elaboration, I appreciate it a lot, and upvoted for the effort. Here are your clearest points paraphrased as I understand them (sometimes just using your words), and my replies:
The FDA is net negative for health, therefore creating an FDA-for-AI would be likely net negative for the AI challenges.
I don’t think you can come to this conclusion, even if I agree with the premise. The counterfactuals are very different. With drugs the counterfactual of no FDA might be some people get more treatments, and some die but many don’t, and they were sick anyway so need to do something, and maybe fewer die than do with the FDA around, so maybe the existence of the FDA compared to the counterfactual is net bad. I won’t dispute this, I don’t know enough about it. However, the counterfactual in AI is different. If unregulated, AI progress steams on ahead, competition over the high rewards is high, and if we don’t have good safety plan (which we don’t) then maybe we all die at some point, who knows when. However, if an FDA-for-AI creates bad regulation (as long as it’s not bad enough to cause AI regulation winter) then it starts slowing down that progress. Maybe it’s bad for, idk, the diseases that could have been solved during the 10 years slowing down from when AI would have solved cancer vs not, and that kind of thing, but it’s nowhere near as bad as the counterfactual! These scenarios are different and not comparable, because the counterfactual of no FDA is not as bad as the counterfactual of no AI regulator.
Enough errors would almost certainly occur in AI regulation to make it net negative.
You gave a bunch of examples from non-AI regulation of bad regulation (I am not going to bother to think about whether I agree that they are bad regulation as it’s not cruxy) - but you didn’t explain how exactly errors lead to making AI regulation net negative? Again I think similar to the previous claim, the counterfactuals likely make this not hold.
...a field where there is bound to be vastly more misunderstanding should be at least as prone to regulation backfiring
That is an interesting claim, I am not sure what makes you think it’s obviously true, as it depends what your goal is. My understanding of the OP is that the goal of the type of regulation they advocate is simply to slow down AI development, nothing more, nothing less. If the goal is to do good regulation of AI, that’s totally different. Is there a specific way in which you imagine it backfiring for the goal of simply slowing down AI progress?
...an [oppressive] regime gaining controllable AI would produce an astronomical suffering risk.
I am unsure what point you were making in the paragraph about evil. Was it about another regime getting there first that might not do safety? For response, see the OP Objection 4 which I share and added additional reason for that not being a real worry in this world.
...unwise to think that people who take blatant actions to kill innocents for political convenience would be safe custodians of AI..
I don’t think it’s fair to say regulators would be a custodian. They have a special kind of lever called “slow things down”, and that lever does not mean that they can, for example, seize and start operating the AI. It is not in their power to do that, legally, nor do they have the capability to do anything with it. We are talking here about slowing things down before AGI, not post AGI.
the electorate does not understand AI
Answer is same as my answer to 3. and also similar to OP Objection 1.
And finally to reply to this: “hopefully this should clarify to a degree why I anticipate both severe X risks and S risks from most attempts at AI regulation”
Basically, no, it doesn’t really clarify it. You started off with a premise I agreed with or at least do not know enough to refute, that the FDA may be net negative, and then drew a conclusion that I disagree with (see 1. above), and then all your other points were assuming that conclusion, so I couldn’t really follow. I tried to pick out bits that seemed like possible key points and reply, but yeah I think you’re pretty confused.
What do you think of my reply to 1. - the counterfactuals being different. I think that’s the best way to progress the conversation.
Hi Aiyen, thanks for clarification.
(Warning: this response is long and much of it is covered by what Tamgen and others have said. )
The way I understand your fears, they fall into four main categories. In the order you raise them and, I think, in order of importance these concerns are as follows:
1) Regulations tend to cause harm to people, therefore we should not regulate AI.
I completely agree that a Federal AI Regulatory Commission will impose costs in the form of human suffering. This is inevitable, since Policy Debates Should Not Appear One Sided. Maybe in the world without the FAIRC, some AI Startup cures Alzheimer’s or even aging a good decade before AGI. In the world with FAIRC, we risk condemning all those people to dementia and decrepitude. This is quite similar to FDA unintended consequences.
Response:
You suggest that the OP was playing reference class tennis, but to me looking at the problem in terms of “regulators” and “harm” is the wrong reference class. They are categories that do not help us predict the answer to the one question we care about most: what is the impact on timelines to AGI?
If we zoom in closer to the object level, it becomes clear that the mechanism by which regulators harm the public is by impeding production. Using Paul Christiano’s rules of reference class tennis, “regulation impedes production” is a more probable narrative (i.e. supported by greater evidence, albeit not simpler) than simply “regulation always causes harm.” At the object level, we see this directly as when the FDA
shootsfines anyone with the temerity to produce cheaper EpiPens, the nuclear regulatory commission doesn’t let anyone build nuclear reactors, etc. Or it can happen indirectly as a drag on innovation. To count the true cost of the FDA, we need to know how many wondrous medical breakthroughs we’ve already made on Earth prime.But if you truly believe that AGI represents an existential threat, and that at present innovation speeds AGI happens before Alignment, then AI progress (even when it solves Alzheimers) is on net a negative. The lives saved by Alzheimer’s have to be balanced against human extinction—and the balance leaves us way, way in the red. This means that all the regulatory failure modes you cite in your reply become net beneficial. We want to impede production.
By way of analogy, it would be as if Pfizer were nearly guaranteed to be working its way toward making a pill that would instantly wipe out humanity; or if Nuclear power actually was as dangerous as its detractors believe! Under such scenarios, the FDA is your best friend. Unfortunately, that is where we stand with AI.
To return to the key question: once it is clear that, at a mechanical level, the things that regulatory agencies do are to impede production, it also becomes clear that regulation is likely to lengthen AGI timelines.
2) The voting public is insufficiently knowledgeable about AI.
I’m not sure I understand the objection here. The government regulates tons of things that the electorate doesn’t understand. In fact, ideally that is what regulatory agencies do. They say, “hey we are a democracy, but you, the demos, don’t understand how education works so we need a department of education.” This is often self-serving patronage, but the general point stands that the way regulatory agencies come into being in practice is not because the electorate achieves subject-area expertise. I can see a populist appeal for a Manhattan project to speed up AI in order to “beat China” (or whatever enemy du jour), but this is not the sort of thing that regulators in permanent bureaucracies do. (Just look at operation “warp speed”; quite apart from the irony in the name, the FDA and the CDC had to be dragged kicking and screaming to do it.)
3) Governments might use AI to do evil things
In your response you write:
I agree, of course, that these are all terrible evils wrought by governments. But I’m not sure what it has to do with regulation of AI. The historical incidents you cite would be relevant if the Holocaust were perpetrated by the German Bureau of Chemical Safety or if the Uighurs were imprisoned by the Chinese Ethnic Affairs Commission. Permanent regulatory bureaucracies are not and never have been responsible for (or even capable of) mission-driven atrocities. They do commit atrocities, but only by preventing access to useful goods (i.e. impeding production).
Finally, one sentence in this section sticks out and makes me think we are talking past each other. You write:
By my lights, this would be a WONDERFUL problem to have. An AI that was controllable by anyone (including Kim Jung-Un, Pol Pot, or Hitler) would, in my estimation, be preferable to a completely unaligned paper clip maximizer. Maybe we disagree here?
4) Liberal democracies are not magic, and we can’t expect them to make the right decisions just because of our own political values.
I don’t think my OP mentioned liberal democracy, but if I gave that impression then you are quite right I did so in error. You may be referring to my point about China. I did not mean to imply a normative superiority of American or any other democracy, and I regret the lack of clarity. My intent was to make a positive observation that governments do, in fact, mimic each other’s regulatory growth. Robin Hanson makes a similar point; that governments copy each other largely because of institutional and informal status associations. This observation is neutral with regard to political system. If we announce a FAIRC, I predict that China will follow, and with due haste.