Then there is the big change that severely weakens SB 1047. [...] AND the cost of that quantity of computing power would exceed one hundred million dollars
I think this change isn’t that bad. (And I proposed changing to a fixed 10^26 flop threshold in my earlier post.)
It seems unlikely to me that this is crucial. If there is some moderately efficient way to convert money into capabilities, then you should expect the first possibly hazardous model to be quite expensive (1), quite soon (e.g. GPT-5) (2), or after a sudden large jump in capabilities due to algorithmic progress (3).
In scenario (1), plausibly the world is already radically transformed by the time algorithmic advances make $100 million enough for truely dangerous models. If the world isn’t radically transformed, by the point when $100 million is dangerous, it has probably been a while for human algorithmic insights to advance sufficiently. And, in the delay, the government has enough time to respond in principle.
Scenarios (2) and (3) are most concerning, but they are also somewhat unlikely. In these scenarios, various plans look less good anyway and a bunch of the action will be in trying to mobilize a rapid government response.
I think this change isn’t that bad. (And I proposed changing to a fixed 10^26 flop threshold in my earlier post.)
It seems unlikely to me that this is crucial. If there is some moderately efficient way to convert money into capabilities, then you should expect the first possibly hazardous model to be quite expensive (1), quite soon (e.g. GPT-5) (2), or after a sudden large jump in capabilities due to algorithmic progress (3).
In scenario (1), plausibly the world is already radically transformed by the time algorithmic advances make $100 million enough for truely dangerous models. If the world isn’t radically transformed, by the point when $100 million is dangerous, it has probably been a while for human algorithmic insights to advance sufficiently. And, in the delay, the government has enough time to respond in principle.
Scenarios (2) and (3) are most concerning, but they are also somewhat unlikely. In these scenarios, various plans look less good anyway and a bunch of the action will be in trying to mobilize a rapid government response.