I had this thought yesterday: “If someone believes in the ‘AGI lab nationalization by default’ story, then what would it look like to build an organization or startup in preparation for this scenario?”
For example, you try to develop projects that would work exceptionally well in a ‘nationalization by default’ world while not getting as much payoff if you are in a non-nationalization world. The goal here is to do the normal startup thing: risky bets with a potentially huge upside.
I don’t necessarily support nationalization and am still trying to think through the upsides/downsides, but I was wondering if there are worlds where some safety projects become much more viable in such a world, the kind of things we used to brush off because we assumed we weren’t in such a world.
Some vague directions to perhaps consider: AI security, Control agenda-type stuff, fully centralized monitoring to detect early signs of model drift, inaccessible compute unless you are a government-verified user, etc. By building such tech or proposals, you may be much more likely to end up with a seat at the big boy table, whereas you wouldn’t have in a non-nationalization world. I could be wrong about the specific examples above, but just want to provide some quick examples.
If I expected that an AI above a certain line will be nationalized, I would try to get just below than line and stay there for as long as possible, to maximize my profits. Alternatively, I may choose to cross the line at the moment my competitors get close to it, if I want to be remembered by the history as the one who reached it first.
If I expected that an AI above a certain line will be nationalized, but I believed that the government will give a lot of money in return (for example, to convince other entrepreneurs that the country is not turning into a communist dystopia), I might decide to try to get that money as soon as possible (less work, guaranteed profit, a place in history), so I would actually exaggerate how dangerous my AI is, to make the government take it away sooner.
But if my goal is to avoid nationalization...
One option is to make my work distributed across countries, so whenever one of them starts talking about nationalizing it, I will make it clear that they can only take a small part of it, and I will simply continue developing it in the remaining countries. I would move the key members of my company to countries with the lowest probability of nationalization. Actually, they could live in a country where I have no servers, and they would connect to them remotely. So that when the government takes the servers, it cannot compel the people to explain how the entire thing works, or prevent them from remotely destroying the part that got nationalized. Also, every part would have a backup in another country.
Another option is to make the AI pretend that it is less smart than it is, to officially stay below the line. It would mean I cannot directly sell its abilities to customers, but maybe I could use it myself, e.g. to let it manage my finances, or I could use it in a plausibly deniable way, e.g. you can hire my smaller company that hires 100 experts who also use my AI, and everything they do is officially attributed to the genius of the human experts.
If anyone would like to discuss this privately, please message me. I’m considering whether to build a startup that tackles the kinds of things I describe above (e.g., monitoring), so I would love to get feedback.
If you think nationalization is near and the default, you shouldn’t try to build projects and hope they get scooped into the nationalized thing. You should try to directly influence the policy apparatus through writing, speaking on podcasts, and getting to know officials in the agencies most likely to be in charge of that.
(Note: not a huge fan of nationalization myself due to red-queen’s-race concerns)
You can do the writing, but if you have a useful product and connect with those who are within the agencies, you are in a position where you have built a team and infrastructure for several years with the purpose of getting pulled into the nationalization project. You likely get most of the value by just keeping close ties with others within government while also have built a ready-to-use solution that can prevent the government from rushing out a worse version of what you’ve built.
I think it’s important to see AI Safety as a collective effort rather than one person’s decision (of working inside or out of government).
I think I am very doubtful of the ability of outsiders to correctly predict—especially outsiders new to government contracting—what the government might pull in. I’d love to be wrong, though! Someone should try it, and I think I was probably too definitive in my comment above.
Yes, but this is similar to usual startups, it’s a calculated bet you are making. So you expect some of the people to try this will fail, but investors hope one of them will be a unicorn.
This might look like building influence / a career in the federal orgs that would be involved in nationalization, rather than a startup. Seems like positioning yourself to be in charge of nationalized projects would be the highest impact?
I agree that this would be impactful! I’m mostly thinking about a more holistic approach that assumes you’d have reasonable to ‘the right people’ in those government positions. Similar to the current status quo where you have governance people and technical people filling in the different gaps.
I had this thought yesterday: “If someone believes in the ‘AGI lab nationalization by default’ story, then what would it look like to build an organization or startup in preparation for this scenario?”
For example, you try to develop projects that would work exceptionally well in a ‘nationalization by default’ world while not getting as much payoff if you are in a non-nationalization world. The goal here is to do the normal startup thing: risky bets with a potentially huge upside.
I don’t necessarily support nationalization and am still trying to think through the upsides/downsides, but I was wondering if there are worlds where some safety projects become much more viable in such a world, the kind of things we used to brush off because we assumed we weren’t in such a world.
Some vague directions to perhaps consider: AI security, Control agenda-type stuff, fully centralized monitoring to detect early signs of model drift, inaccessible compute unless you are a government-verified user, etc. By building such tech or proposals, you may be much more likely to end up with a seat at the big boy table, whereas you wouldn’t have in a non-nationalization world. I could be wrong about the specific examples above, but just want to provide some quick examples.
If I expected that an AI above a certain line will be nationalized, I would try to get just below than line and stay there for as long as possible, to maximize my profits. Alternatively, I may choose to cross the line at the moment my competitors get close to it, if I want to be remembered by the history as the one who reached it first.
If I expected that an AI above a certain line will be nationalized, but I believed that the government will give a lot of money in return (for example, to convince other entrepreneurs that the country is not turning into a communist dystopia), I might decide to try to get that money as soon as possible (less work, guaranteed profit, a place in history), so I would actually exaggerate how dangerous my AI is, to make the government take it away sooner.
But if my goal is to avoid nationalization...
One option is to make my work distributed across countries, so whenever one of them starts talking about nationalizing it, I will make it clear that they can only take a small part of it, and I will simply continue developing it in the remaining countries. I would move the key members of my company to countries with the lowest probability of nationalization. Actually, they could live in a country where I have no servers, and they would connect to them remotely. So that when the government takes the servers, it cannot compel the people to explain how the entire thing works, or prevent them from remotely destroying the part that got nationalized. Also, every part would have a backup in another country.
Another option is to make the AI pretend that it is less smart than it is, to officially stay below the line. It would mean I cannot directly sell its abilities to customers, but maybe I could use it myself, e.g. to let it manage my finances, or I could use it in a plausibly deniable way, e.g. you can hire my smaller company that hires 100 experts who also use my AI, and everything they do is officially attributed to the genius of the human experts.
We had a similar thought:
But yeah, my initial comment was about how to take advantage of nationalization if it does happen in the way Leopold described/implied.
If anyone would like to discuss this privately, please message me. I’m considering whether to build a startup that tackles the kinds of things I describe above (e.g., monitoring), so I would love to get feedback.
If you think nationalization is near and the default, you shouldn’t try to build projects and hope they get scooped into the nationalized thing. You should try to directly influence the policy apparatus through writing, speaking on podcasts, and getting to know officials in the agencies most likely to be in charge of that.
(Note: not a huge fan of nationalization myself due to red-queen’s-race concerns)
You can do the writing, but if you have a useful product and connect with those who are within the agencies, you are in a position where you have built a team and infrastructure for several years with the purpose of getting pulled into the nationalization project. You likely get most of the value by just keeping close ties with others within government while also have built a ready-to-use solution that can prevent the government from rushing out a worse version of what you’ve built.
I think it’s important to see AI Safety as a collective effort rather than one person’s decision (of working inside or out of government).
I think I am very doubtful of the ability of outsiders to correctly predict—especially outsiders new to government contracting—what the government might pull in. I’d love to be wrong, though! Someone should try it, and I think I was probably too definitive in my comment above.
Yes, but this is similar to usual startups, it’s a calculated bet you are making. So you expect some of the people to try this will fail, but investors hope one of them will be a unicorn.
This might look like building influence / a career in the federal orgs that would be involved in nationalization, rather than a startup. Seems like positioning yourself to be in charge of nationalized projects would be the highest impact?
I agree that this would be impactful! I’m mostly thinking about a more holistic approach that assumes you’d have reasonable to ‘the right people’ in those government positions. Similar to the current status quo where you have governance people and technical people filling in the different gaps.