A coalition strong enough to prevent the world’s leading project from maintaining and lengthening its lead would need to have some way of preventing the leading project from accessing the innovations of the coalition. Otherwise the leading project will free-ride off the research done by the coalition. For this reason I think that a coalition would look very different from the world economy; in order to prevent the leading project from accessing innovations deployed in the world economy you would need to have an enforced universal embargo on them pretty much, and if you have that much political power, why stop there? Why not just annex them or shut them down?
Are you saying that the leading project can easily spy on other projects, but other projects can’t spy on it? Is this because the rest of the world is trading with each other, and trading opens up opportunities for spying? Some other reason I missed? I don’t think it’s usually the case that gains from rabbit-holing, in terms of protection from spying, are large enough to outweigh the costs from not trading. It seems weird to expect AI to change that, since you are arguing that the proportional importance of spying will go down, not up, because it won’t be accelerated as much.
If the leading project can’t spy on everyone else, then how does it differ from all of the other companies who are developing technology, keeping it private, and charging other people to use it? The leading project can use others’ technology when it pays them, just like they use each other’s technology when they pay each other. The leading project can choose not to sell its technology, but then it just has less money and so falls further and further behind in terms of compute etc. (and at any rate, it needs to be selling something to the other people in order to even be able to afford to use their technology).
(I may just be missing something about your model.)
Yes, if the coalition is a large fraction of the world then I am saying there is an asymmetry in that the leading project can more easily spy on that large fraction of the world than the other way round. This is because large fractions of the world contain many different people and groups, some of which will leak secrets (or sell secrets) to the leading project, unless extremely and unprecedentedly effective anti-leaking-and-spying measures are implemented across a large fraction of the world. It’s hard but doable for one corporation to keep trade secrets from the rest of the economy; how on earth can the rest of the economy keep trade secrets from a powerful corporation?
I don’t see how I’m arguing that the proportional importance of spying will go down. The proportional importance of spying will go up precisely because it won’t be accelerated as much as AI technology in general will be. (Why don’t I think spying will be accelerated as much as AI technology in general? I certainly agree that spying technology will be accelerated as much or more as AI technology. However I think that spying is a function of several things, only one of which is spying technology, the others being non-technology things like having literal human spies climb through ranks of enemy orgs and also having anti-spying technology.) I envision a future where spying is way more rewarding than any time in history, and yet nevertheless the actual amount of successful spying is less than 10x-100x more than in the past, due to the factors mentioned in the parenthesis.
“The leading project can choose not to sell its technology, but then it just has less money and so falls further and further behind in terms of compute etc. (and at any rate, it needs to be selling something to the other people in order to even be able to afford to use their technology).”
Again, my whole point is that this is only true in the long run. Yes, in the long run a project which relies on other sources of income to buy the things it needs to buy will lose money to projects which sell their innovations. But in the short run empirically it seems that projects can go for years on funding raised from investors and wealthy parent companies. I guess your point is that in a world where the economy is growing super fast due to AI, this won’t be true: any parent company or group of investors capable of funding the leading project at year X will be relative paupers by year X+3 unless their project has been selling its tech. Am I right in understanding you here?
(Miscellanous: I don’t think the leading project differs from all the other projects that develop tech and keep it private. Like Wei Dai said, insofar as a company can charge people to use tech without letting the secrets of how to build that tech escape, they will obviously do so. I think our disagreement is about your last two sentences, which I quoted above.)
Are you saying that the leading project can easily spy on other projects, but other projects can’t spy on it? Is this because the rest of the world is trading with each other, and trading opens up opportunities for spying? Some other reason I missed? I don’t think it’s usually the case that gains from rabbit-holing, in terms of protection from spying, are large enough to outweigh the costs from not trading. It seems weird to expect AI to change that, since you are arguing that the proportional importance of spying will go down, not up, because it won’t be accelerated as much.
If the leading project can’t spy on everyone else, then how does it differ from all of the other companies who are developing technology, keeping it private, and charging other people to use it? The leading project can use others’ technology when it pays them, just like they use each other’s technology when they pay each other. The leading project can choose not to sell its technology, but then it just has less money and so falls further and further behind in terms of compute etc. (and at any rate, it needs to be selling something to the other people in order to even be able to afford to use their technology).
(I may just be missing something about your model.)
Yes, if the coalition is a large fraction of the world then I am saying there is an asymmetry in that the leading project can more easily spy on that large fraction of the world than the other way round. This is because large fractions of the world contain many different people and groups, some of which will leak secrets (or sell secrets) to the leading project, unless extremely and unprecedentedly effective anti-leaking-and-spying measures are implemented across a large fraction of the world. It’s hard but doable for one corporation to keep trade secrets from the rest of the economy; how on earth can the rest of the economy keep trade secrets from a powerful corporation?
I don’t see how I’m arguing that the proportional importance of spying will go down. The proportional importance of spying will go up precisely because it won’t be accelerated as much as AI technology in general will be. (Why don’t I think spying will be accelerated as much as AI technology in general? I certainly agree that spying technology will be accelerated as much or more as AI technology. However I think that spying is a function of several things, only one of which is spying technology, the others being non-technology things like having literal human spies climb through ranks of enemy orgs and also having anti-spying technology.) I envision a future where spying is way more rewarding than any time in history, and yet nevertheless the actual amount of successful spying is less than 10x-100x more than in the past, due to the factors mentioned in the parenthesis.
“The leading project can choose not to sell its technology, but then it just has less money and so falls further and further behind in terms of compute etc. (and at any rate, it needs to be selling something to the other people in order to even be able to afford to use their technology).”
Again, my whole point is that this is only true in the long run. Yes, in the long run a project which relies on other sources of income to buy the things it needs to buy will lose money to projects which sell their innovations. But in the short run empirically it seems that projects can go for years on funding raised from investors and wealthy parent companies. I guess your point is that in a world where the economy is growing super fast due to AI, this won’t be true: any parent company or group of investors capable of funding the leading project at year X will be relative paupers by year X+3 unless their project has been selling its tech. Am I right in understanding you here?
(Miscellanous: I don’t think the leading project differs from all the other projects that develop tech and keep it private. Like Wei Dai said, insofar as a company can charge people to use tech without letting the secrets of how to build that tech escape, they will obviously do so. I think our disagreement is about your last two sentences, which I quoted above.)