When a person says “we don’t trade with ants”, I think the implicit explanation is that humans are so big, powerful and smart compared to ants that we don’t need to trade with them because they have nothing of value and if they did we could just take it; anything they can do we can do better, and we can just walk all over them. Why negotiate when you can steal?
I think this is an overly narrow definition of trading for this context. If an AGI wants something from humans it needs to leave us alive and happy enough to produce it. It might be nonconsensual, but dead people can’t produce anything to steal, so the problem has become “avoid being tortured” rather than “avoid being killed altogether.”
A lot of people do trade spiders warmth and shelter in exchange for keeping the insect population low. The spider does not know that this is happening, and it can’t negotiate, but my impression is this is working out pretty well for them. Locally at least- maybe sufficiently intelligent house spiders would regret evolving to only live in houses and die outside, or would resent our reworking of the ecostructure- but they are alive, and selection to live in houses probably doesn’t make them miserable in houses, even if it does limit their options.
If an AGI wants something from humans it needs to leave us alive and happy enough to produce it
No, I imagine an AGI would have many creative ways to force humans to do what it wants—directly pumping nutrients into your blood, removing neurotransmitters from your brain, overwriting your personality with an incorrigible desire to Do The Thing...
I think this is an overly narrow definition of trading for this context. If an AGI wants something from humans it needs to leave us alive and happy enough to produce it. It might be nonconsensual, but dead people can’t produce anything to steal, so the problem has become “avoid being tortured” rather than “avoid being killed altogether.”
A lot of people do trade spiders warmth and shelter in exchange for keeping the insect population low. The spider does not know that this is happening, and it can’t negotiate, but my impression is this is working out pretty well for them. Locally at least- maybe sufficiently intelligent house spiders would regret evolving to only live in houses and die outside, or would resent our reworking of the ecostructure- but they are alive, and selection to live in houses probably doesn’t make them miserable in houses, even if it does limit their options.
No, I imagine an AGI would have many creative ways to force humans to do what it wants—directly pumping nutrients into your blood, removing neurotransmitters from your brain, overwriting your personality with an incorrigible desire to Do The Thing...