Generally. I endorse the comparison of AI with nuclear weapons (especially because AI is currently being mounted on literal nuclear weapons).
But in this case, there’s a really big distinction that should be made between mass-media and specialized institutions. Intelligence/military agencies, specialized Wall-street analyst firms, and bureaucracy leadership all probably know things like exactly how frequently Covid causes brain damage and have the best forecasters predicting the next outbreak. For them, it’s less about spinning stories, and more about figuring out what type of professional employees tend to write accurate/predictive reports and forecasts. Spun stories are certainly more influential then they were 10 years ago, and vastly more influential than they appear to the uninitiated, but I don’t know if we’ve gotten to the point where they can fool the professionals at not getting fooled.
Arms control has happened in the past even though it was difficult to verify, and nuclear weapons were centralized by default so it’s hard to know anything about how hard it is to centralize that sort of thing.
With forecasters from both sides given equal amounts of information, these institutions might not even reliably beat the Metaculus community. If one is such a great forecaster then they can forecast that jobs like this might not be, among other things, that fulfilling.
I don’t know if we’ve gotten to the point where they can fool the professionals at not getting fooled
Quite a few professionals (not at not getting fooled) still believe in a roughly 0% probability of a certain bio-related accident a couple three years ago thanks in large part to a spun story. Maybe the forecasters at the above places know better but none of the entities who might act on that information are necessarily incentivized to push for regulation as a result. So it’s not clear it would matter if most forecasters know AI is probably responsible for some murky disaster while the public believes humans are responsible.
Generally. I endorse the comparison of AI with nuclear weapons (especially because AI is currently being mounted on literal nuclear weapons).
But in this case, there’s a really big distinction that should be made between mass-media and specialized institutions. Intelligence/military agencies, specialized Wall-street analyst firms, and bureaucracy leadership all probably know things like exactly how frequently Covid causes brain damage and have the best forecasters predicting the next outbreak. For them, it’s less about spinning stories, and more about figuring out what type of professional employees tend to write accurate/predictive reports and forecasts. Spun stories are certainly more influential then they were 10 years ago, and vastly more influential than they appear to the uninitiated, but I don’t know if we’ve gotten to the point where they can fool the professionals at not getting fooled.
Arms control has happened in the past even though it was difficult to verify, and nuclear weapons were centralized by default so it’s hard to know anything about how hard it is to centralize that sort of thing.
With forecasters from both sides given equal amounts of information, these institutions might not even reliably beat the Metaculus community. If one is such a great forecaster then they can forecast that jobs like this might not be, among other things, that fulfilling.
Quite a few professionals (not at not getting fooled) still believe in a roughly 0% probability of a certain bio-related accident a couple three years ago thanks in large part to a spun story. Maybe the forecasters at the above places know better but none of the entities who might act on that information are necessarily incentivized to push for regulation as a result. So it’s not clear it would matter if most forecasters know AI is probably responsible for some murky disaster while the public believes humans are responsible.