AGI & War

Recently Garett Jones made a few tweets implying that the existence of entities with a very wide power differential today is evidence that super intelligence won’t lead to Doom.

It is a reasonable argument. If the powerful today think it’s better to trade with the powerless than to destroy them and take what they have, and trade is so beneficial in general, and war so costly, why would it be different for the even more powerful?

Unfortunately I think the position is based on an incomplete understanding of what AGI changes about the nature of trade and war.

Right now, due to lots of reasons, humans are the most valuable things. Most of the value of production goes to human factors, with a smaller value going to natural resources and physical capital. So dooming the humans is destroying exactly the thing you’d want to steal. Stealing humans themselves is extremely costly, because humans are pretty hard to control. As an alternative, you can access lots of benefits from other humans through trade, so the decision to not doom the powerless makes perfect sense.

Maybe the US could easily take on Mexico, and either occupy it indefinitely, or genocide the whole population, but the non human things in Mexico are not worth the trouble, and occupying it to extract value from the population would be extremely hard, and destroy a lot of the value the Mexicans produce, and the US would lose an important trade partner.

AGI changes this. After AGI, the value of human factors will fall to the subsistence level of the AGI. This means humans become cheap, and resources become relatively expensive. Humans will be deriving all their income from their property, and unlike humans, property is easy to steal.

The decision between going to war or trading changes completely. Now if you win the war, you can take all the most valuable things the losers had, the things you were trading for in the first place.

The US could annex Mexico, genocide their population, meat and silicon alike, and just make more copies of their own AGIs to take their place, and end up with all the extremely valuable (in the new economy) natural resources. Given that population in Mexico stays the same (enough for the marginal cost to run an AGI there to be the same as the marginal product of the AGI) they’re not making trade with Mexico substantially lower, and can keep enjoying it’s benefits.

Do note that even thou the initial debate was about super intelligence and Doom, this has far greater implications. Once AGI makes humans cheap we can expect agents to act like they are.