if your company is default-dead, you should pursue high-variance strategies
There are rumors OpenAI (which has no moat) is spending much more than it’s making this year despite good revenue, another datapoint on there being $1 billion training runs currently in progress.
I’m curious what sort of policies you’re thinking of which would allow for a pause which plausibly buys us decades, rather than high-months-to-low-years. My imagination is filling in “totalitarian surveillance state which is effective at banning general-purpose computing worldwide, and which prioritizes the maintenance of its own control over all other concerns”. But I’m guessing that’s not what you have in mind.
No more totalitarian than control over manufacturing of nuclear weapons. The issue is that currently there is no buy-in on a similar level, and any effective policy is too costly to accept for people who don’t expect existential risk. This might change once there are long-horizon task capable AIs that can do many jobs, if they are reined in before there is runaway AGI that can do research on its own. And establishing control over compute is more feasible if it turns out that taking anything approaching even a tiny further step in the direction of AGI takes 1e27 FLOPs.
Generally available computing hardware doesn’t need to keep getting better over time, for many years now PCs have been beyond what is sufficient for most mundane purposes. What remains is keeping an eye on GPUs for the remaining highly restricted AI research and specialized applications like medical research. To prevent their hidden stockpiling, all GPUs could be required to need regular unlocking OTPs issued with asymmetric encryption using multiple secret keys kept separately, so that all of the keys would need to be stolen simultaneously to keep the GPUs working (if the GPUs go missing or a country that hosts the datacenter goes rogue, and official unlocking OTPs wouldn’t keep being issued). Hidden manufacturing of GPUs seems much less feasible than hidden or systematically subverted datacenters.
a totalitarian surveillance state which is effective at banning general-purpose computing worldwide, and which prioritizes the maintenance of its own control over all other concerns
I much prefer that to everyone’s being killed by AI. Don’t you?
Almost certainly alive for several more decades if we are talking literal extinction rather than civilization-wreaking catastrophe. Therefore it makes sense to work towards global coordination to pause AI for at least this long.
There are rumors OpenAI (which has no moat) is spending much more than it’s making this year despite good revenue, another datapoint on there being $1 billion training runs currently in progress.
I’m curious what sort of policies you’re thinking of which would allow for a pause which plausibly buys us decades, rather than high-months-to-low-years. My imagination is filling in “totalitarian surveillance state which is effective at banning general-purpose computing worldwide, and which prioritizes the maintenance of its own control over all other concerns”. But I’m guessing that’s not what you have in mind.
No more totalitarian than control over manufacturing of nuclear weapons. The issue is that currently there is no buy-in on a similar level, and any effective policy is too costly to accept for people who don’t expect existential risk. This might change once there are long-horizon task capable AIs that can do many jobs, if they are reined in before there is runaway AGI that can do research on its own. And establishing control over compute is more feasible if it turns out that taking anything approaching even a tiny further step in the direction of AGI takes 1e27 FLOPs.
Generally available computing hardware doesn’t need to keep getting better over time, for many years now PCs have been beyond what is sufficient for most mundane purposes. What remains is keeping an eye on GPUs for the remaining highly restricted AI research and specialized applications like medical research. To prevent their hidden stockpiling, all GPUs could be required to need regular unlocking OTPs issued with asymmetric encryption using multiple secret keys kept separately, so that all of the keys would need to be stolen simultaneously to keep the GPUs working (if the GPUs go missing or a country that hosts the datacenter goes rogue, and official unlocking OTPs wouldn’t keep being issued). Hidden manufacturing of GPUs seems much less feasible than hidden or systematically subverted datacenters.
I much prefer that to everyone’s being killed by AI. Don’t you?