Worst case scenario, can’t humans just abandon the internet altogether once they realize this is happening? Declare that only physical currency is valid, cut off all internet communications and only communicate by means that the AI can’t access?
Of course it should be easy for the AI to avoid notice for a long while, but once we get to “turn the universe into computronium to make paperclips” (or any other scheme that diverges from business-as-usual drastically) people will eventually catch on. There is an upper bound to the level of havoc the AI can wreak without people eventually noticing and resisting in the manner described above.
How exactly would the order to abandon the internet get out to everyone? There are almost no means of global communications that aren’t linked to the internet in some way.
Government orders the major internet service providers to shut down their services, presumably :) Not saying that that would necessarily be easily to coordinate, nor that the loss of internet wouldn’t cripple the global economy. Just that it seems to be a different order of risk than an extinction event.
My intuition on the matter was that an AI would be limited in its scope of influence to digital networks, and its access to physical resources, e.g. labs, factories and the like would be contingent on persuading people to do things for it. But everyone here is so confident that FAI --> doom that I was wondering if there was some obvious and likely successful method of seizing control of physical resources that everyone else already knew and I had missed.
Worst case scenario, can’t humans just abandon the internet altogether once they realize this is happening? Declare that only physical currency is valid, cut off all internet communications and only communicate by means that the AI can’t access?
Of course it should be easy for the AI to avoid notice for a long while, but once we get to “turn the universe into computronium to make paperclips” (or any other scheme that diverges from business-as-usual drastically) people will eventually catch on. There is an upper bound to the level of havoc the AI can wreak without people eventually noticing and resisting in the manner described above.
How exactly would the order to abandon the internet get out to everyone? There are almost no means of global communications that aren’t linked to the internet in some way.
Government orders the major internet service providers to shut down their services, presumably :) Not saying that that would necessarily be easily to coordinate, nor that the loss of internet wouldn’t cripple the global economy. Just that it seems to be a different order of risk than an extinction event.
My intuition on the matter was that an AI would be limited in its scope of influence to digital networks, and its access to physical resources, e.g. labs, factories and the like would be contingent on persuading people to do things for it. But everyone here is so confident that FAI --> doom that I was wondering if there was some obvious and likely successful method of seizing control of physical resources that everyone else already knew and I had missed.