Similarly, his ideas of things like ‘a truth seeking AI would keep us around’ seem to me like Elon grasping at straws and thinking poorly, but he’s trying.
The way I think about Elon is that he’s very intelligent but essentially not open to any new ideas or capable of self-reflection if his ideas are wrong, except on technical matters: if he can’t clearly follow the logic himself, on the first try, or there’s a reason it would be uncomfortable or difficult to ignore it initially then he won’t believe you, but he is smart.
Essentially, he got one good idea about AI risk into his head 10+ years ago and therefore says locally good things and isn’t simply lying when he says them, but it doesn’t hang together in his head in a consistent way (e.g. if he thought international stability and having good AI regulation was a good idea he wouldn’t be supporting the candidate that wants to rethink all US alliances and would impair the federal government’s ability to do anything new and complicated, with an e/acc as his running mate). In general, I think one of the biggest mental blind spots EA/rationalist types have is overestimating the coherence of people’s plans for the future.
The Economist is opposed, in a quite bad editorial calling belief in the possibility of a catastrophic harm ‘quasi-religious’ without argument, and uses that to dismiss the bill, instead calling for regulations that address mundane harms. That’s actually it.
I find this especially strange almost to the point that I’m willing to call it knowingly bad faith. The Economist in the past has sympathetically interviewed Helen Toner, done deep dive investigations into mechanistic interpretability research that are at a higher level of analysis than I’ve seen from any other mainstream news publication, ran articles which acknowledge the soft consensus among tech workers and AI experts on the dangers, which included survey results on these so it’s doubly difficult to dismiss is as “too speculative” or “scifi”.
To state without elaboration that the risk is “quasi-religious” or “science fictional” when their own journalists have consistently said the opposite and provided strong evidence that the AI world generally agrees makes me feel like someone higher up changed their mind for some reason regardless of what their own authors think.
The one more concrete reference they gave was to very near term (like the next year) prospect of AI systems being used to assist in terrorism, which has indeed been slightly exaggerated by some, but to claim that there’s no idea whatsoever about where these capabilities could be in 3 years is absurd given what they themselves have said in previous articles.
Without some explanation as to why they think genuine catastrophic misuse concerns are not relatively near term and relatively serious (e.g. explaining why they think we won’t see autonomous agents that could play a more active role in terrorism if freely available) it just becomes the classic “if 2025 is real why isn’t it 2025 now” fallacy.
The short argument I’ve been using is:
If you want to oppose the bill you as a matter of logical necessity have to believe some combination of,
No significant near-term catastrophic AI risks exist that warrant this level of regulation.
Significant near-term risks exist, but companies shouldn’t be held liable for them (i.e. you’re an extremist ancap)
Better alternatives are available to address these risks.
The bill will be ineffective or counterproductive in addressing these risks.
The best we get is vague hints that (1) is true from some tech leaders but e.g. google, OpenAI definitely doesn’t believe (1), or we get vague pie in the sky appeals to (3) as if the federal government is working efficiently on frontier tech issues right now, or claims for (4) that either lie about the content of the bill, e.g. claiming it applies to small startups and academics, or say fearmongering towards (4) like every tech company in California will up-sticks and leave, or it will so impair progress that China will inevitably win, so the bill will not achieve its stated aims.
The way I think about Elon is that he’s very intelligent but essentially not open to any new ideas or capable of self-reflection if his ideas are wrong, except on technical matters: if he can’t clearly follow the logic himself, on the first try, or there’s a reason it would be uncomfortable or difficult to ignore it initially then he won’t believe you, but he is smart.
Essentially, he got one good idea about AI risk into his head 10+ years ago and therefore says locally good things and isn’t simply lying when he says them, but it doesn’t hang together in his head in a consistent way (e.g. if he thought international stability and having good AI regulation was a good idea he wouldn’t be supporting the candidate that wants to rethink all US alliances and would impair the federal government’s ability to do anything new and complicated, with an e/acc as his running mate). In general, I think one of the biggest mental blind spots EA/rationalist types have is overestimating the coherence of people’s plans for the future.
I find this especially strange almost to the point that I’m willing to call it knowingly bad faith. The Economist in the past has sympathetically interviewed Helen Toner, done deep dive investigations into mechanistic interpretability research that are at a higher level of analysis than I’ve seen from any other mainstream news publication, ran articles which acknowledge the soft consensus among tech workers and AI experts on the dangers, which included survey results on these so it’s doubly difficult to dismiss is as “too speculative” or “scifi”.
To state without elaboration that the risk is “quasi-religious” or “science fictional” when their own journalists have consistently said the opposite and provided strong evidence that the AI world generally agrees makes me feel like someone higher up changed their mind for some reason regardless of what their own authors think.
The one more concrete reference they gave was to very near term (like the next year) prospect of AI systems being used to assist in terrorism, which has indeed been slightly exaggerated by some, but to claim that there’s no idea whatsoever about where these capabilities could be in 3 years is absurd given what they themselves have said in previous articles.
Without some explanation as to why they think genuine catastrophic misuse concerns are not relatively near term and relatively serious (e.g. explaining why they think we won’t see autonomous agents that could play a more active role in terrorism if freely available) it just becomes the classic “if 2025 is real why isn’t it 2025 now” fallacy.
The short argument I’ve been using is: