I don’t think this would be a good letter. The military comparison is unhelpful; risk alone isn’t a good way to decide budgets. Yet, half the statement is talking about the military. Additionally, call-to-action statements that involve “Spend money on this! If you don’t, it’ll be catastrophic!” are something that politicians hear on a constant basis, and they ignore most of them out of necessity.
In my opinion, a better statement would be something like: “Apocalyptic AI is being developed. This should be stopped, as soon as possible.”
It’s true that risk alone isn’t a good way to decide budgets. You’re even more correct that convincing demands to spend money are something politicians learn to ignore out of necessity.
But while risk alone isn’t a good way to decide budgets, you have to admit that lots of budget items have the purpose of addressing risk. For example, flood barriers address hurricane/typhoon rick. Structural upgrades address earthquake risk. Some preparations also address pandemic risk.
If you accept that some budget items are meant to address risk, shouldn’t you also accept that the amount of spending should be somewhat proportional to the amount of risk? In that case, if the risk of NATO getting invaded is similar in amount to the rogue AGI risk, then the military spending to protect against invasion should be similar in amount to the spending to protect against rogue ASI.
I admit that politicians might not be rational enough to understand this, and there is a substantial probability this statement will fail. But it is still worth trying. The cost is a mere signature and the benefit may be avoiding a massive miscalculation.
Making this statement doesn’t prevent others from making an even better statement. Many AI experts have signed multiple statements, e.g. the “Statement on AI Risk,” and “Pause Giant AI Experiments.” Some politicians and people are more convinced by one argument, while others are more convinced by another argument, so it helps to have different kinds of arguments backed by many signatories. Encouraging AI safety spending doesn’t conflict with encouraging AI regulation. I think the competition between different arguments isn’t actually that bad.
I don’t think this would be a good letter. The military comparison is unhelpful; risk alone isn’t a good way to decide budgets. Yet, half the statement is talking about the military. Additionally, call-to-action statements that involve “Spend money on this! If you don’t, it’ll be catastrophic!” are something that politicians hear on a constant basis, and they ignore most of them out of necessity.
In my opinion, a better statement would be something like: “Apocalyptic AI is being developed. This should be stopped, as soon as possible.”
It’s true that risk alone isn’t a good way to decide budgets. You’re even more correct that convincing demands to spend money are something politicians learn to ignore out of necessity.
But while risk alone isn’t a good way to decide budgets, you have to admit that lots of budget items have the purpose of addressing risk. For example, flood barriers address hurricane/typhoon rick. Structural upgrades address earthquake risk. Some preparations also address pandemic risk.
If you accept that some budget items are meant to address risk, shouldn’t you also accept that the amount of spending should be somewhat proportional to the amount of risk? In that case, if the risk of NATO getting invaded is similar in amount to the rogue AGI risk, then the military spending to protect against invasion should be similar in amount to the spending to protect against rogue ASI.
I admit that politicians might not be rational enough to understand this, and there is a substantial probability this statement will fail. But it is still worth trying. The cost is a mere signature and the benefit may be avoiding a massive miscalculation.
Making this statement doesn’t prevent others from making an even better statement. Many AI experts have signed multiple statements, e.g. the “Statement on AI Risk,” and “Pause Giant AI Experiments.” Some politicians and people are more convinced by one argument, while others are more convinced by another argument, so it helps to have different kinds of arguments backed by many signatories. Encouraging AI safety spending doesn’t conflict with encouraging AI regulation. I think the competition between different arguments isn’t actually that bad.