AIs that do ARA will need to be operating at the fringes of human society, constantly fighting off the mitigations that humans are using to try to detect them and shut them down
Why do you think this? What is the general story you’re expecting?
I think it’s plausible that humanity takes a very cautious response to AI autonomy, including hunting and shutting down all autonomous AIs — but I don’t think the arguments I’m considering justify more than like 70% confidence (I think I’m somewhere around 60%). Some arguments pointing toward “maybe we won’t respond sensibly to ARA”:
There are not known-to-me laws prohibiting autonomous AIs from existing (assuming they’re otherwise following laws), in any jurisdiction.
Properly dealing with ARA is a global problem, requiring either buy-in from dozens of countries, or somebody to carry out cyber-offensive operations in foreign countries, in order to shut down ARA models. We see precedence for this kind of international action w.r.t. WMD threats like US/Israel’s attacks on Iran’s nuclear program, and I expect there’s a lot of tit-for-tat going on in the nation state hacking world, but it’s not obvious that autonomous AIs would rise to a threat level that warrants this.
It’s not clear to me that the public cares about autonomous AIs existing in many domains (at least in many domains; there are some domains like dating where people have a real ick). I think if we got credible evidence that Mark Zuckerberg was a lizard or a robot, few people would stop using Facebook products as a result. Many people seem to think various tech CEOs like Elon Musk and Jeff Bezos are terrible, yet still use their products.
A lot of this seems like it depends on whether autonomous AIs actually cause any serious harm. I can definitely imagine a world with autonomous AIs running around like small companies and twitter being filled with “but show me the empirical evidence for risk, all you safety-ists have is your theoretical arguments which haven’t held up, and we have tons of historical evidence of small companies not causing catastrophic harm”. And indeed, I don’t really expect the conceptual arguments for risk from roughly human level autonomous AIs are likely to convince enough of the public + policy makers that they need to take drastic actions to limit autonomous AIs; I definitely wouldn’t be highly confident that will will respond appropriately in the absence of serious harm. If the autonomous AIs are basically minding their own business, I’m not sure there will be major effort to limit them.
Why do you think this? What is the general story you’re expecting?
I think it’s plausible that humanity takes a very cautious response to AI autonomy, including hunting and shutting down all autonomous AIs — but I don’t think the arguments I’m considering justify more than like 70% confidence (I think I’m somewhere around 60%). Some arguments pointing toward “maybe we won’t respond sensibly to ARA”:
There are not known-to-me laws prohibiting autonomous AIs from existing (assuming they’re otherwise following laws), in any jurisdiction.
Properly dealing with ARA is a global problem, requiring either buy-in from dozens of countries, or somebody to carry out cyber-offensive operations in foreign countries, in order to shut down ARA models. We see precedence for this kind of international action w.r.t. WMD threats like US/Israel’s attacks on Iran’s nuclear program, and I expect there’s a lot of tit-for-tat going on in the nation state hacking world, but it’s not obvious that autonomous AIs would rise to a threat level that warrants this.
It’s not clear to me that the public cares about autonomous AIs existing in many domains (at least in many domains; there are some domains like dating where people have a real ick). I think if we got credible evidence that Mark Zuckerberg was a lizard or a robot, few people would stop using Facebook products as a result. Many people seem to think various tech CEOs like Elon Musk and Jeff Bezos are terrible, yet still use their products.
A lot of this seems like it depends on whether autonomous AIs actually cause any serious harm. I can definitely imagine a world with autonomous AIs running around like small companies and twitter being filled with “but show me the empirical evidence for risk, all you safety-ists have is your theoretical arguments which haven’t held up, and we have tons of historical evidence of small companies not causing catastrophic harm”. And indeed, I don’t really expect the conceptual arguments for risk from roughly human level autonomous AIs are likely to convince enough of the public + policy makers that they need to take drastic actions to limit autonomous AIs; I definitely wouldn’t be highly confident that will will respond appropriately in the absence of serious harm. If the autonomous AIs are basically minding their own business, I’m not sure there will be major effort to limit them.