We would not be destroyed in the first large nuclear war (while effects of radioactivity in the food chains in my view are under researched). But not a single open society would survive.
A world of malthusian masses, and a military aristocracy desperately trying to keep as much firepower as possible is the natural post nuclear war outcome. Then, history happens again, and in 2000 years luckily we are back into deciding if we allow AGI to be developed. What is the point?
The baseline is that our governance systems are completely unaccurate for nuclear weapons. Even in this fortunate age of hegemonic republics.
We need to solve the human alignement problem. Do you have any better suggestion than AGI?
I find it frustrating to correspond with you. You have become attached to an argument for what we should do. To support this argument, you send out many “soldiers”: nuclear war is a potent existential risk; electromagnetic pulse would destroy most electronic devices; not a single open society would survive a nuclear war; nuclear war inevitably leads to more nuclear war; nuclear war will cause widespread societal collapse. And now we have a new soldier, namely, the effect of radiation on the food chain.
Each of these soldiers seems plausible if one’s epistemology consists mostly in noticing how often something is repeated in the press and online. But I haven’t seen a single attempt by you to support any of these assertions / soldiers. When I say that Wikipedia says that most vehicles and cellphones would continue to operate after the electromagnetic pulses of a nuclear war, you ignore that. I offer you an opening to explain why you believe that nuclear war will lead to societal collapse whereas WWI, WWII and the Chinese civil war did not; you decline to engage on that. I still do not know whether you accept the conventional definition of “existential risk” (even after I asked you a direct question): when you wrote that nuclear war is a potent existential risk, maybe you thought that the possibility that half of the human population might die constitutes an existential risk. I.e., maybe you have been using an unconventional definition of existential risk. Your readers (including me) still do not know.
If I continue corresponding with you, I expect you would send out a few more soldiers, but it take me a lot more work to explain why a soldier does not in fact support your argument than it takes you to find the next soldier and to send it out.
Have you ever tried to learn about the effects of radiation on the food chain, e.g., by typing the phrase into a search engine and spending 5 minutes (as measured by an actual clock or timer) looking at the results? Science knows much about the subject. The radiation from an accident at a nuclear power plant is very different from the radiation from a nuclear weapon, so you’d have to be careful not to generalize from the first case to the second. (A much higher fraction of the radiation in the first case comes from long-half-life isotopes.)
“Commercial computer equipment is particularly vulnerable to EMP effects. Computers used in data processing systems, communications systems, displays, industrial control applications, including road and rail signaling, and those embedded in military equipment, such as signal processors, electronic flight controls and digital engine control systems, are all potentially vulnerable to the EMP effect. Other electronic devices and electrical equipment may also be destroyed by the EMP effect. Telecommunications equipment can be highly vulnerable and receivers of all varieties are particularly sensitive to EMP. Therefore radar and electronic warfare equipment, satellite, microwave, UHF, VHF, HF and low band communications equipment and television equipment are all potentially vulnerable to the EMP effect. Cars with electronic ignition systems/ and ignition chips are also vulnerable.”
not a single open society would survive a nuclear war; nuclear war inevitably leads to more nuclear war; nuclear war will cause widespread societal collapse
:-) After total economic disruption, no electricity nor electronics, and a few billion deaths… its like the parachute randomized trial. Too obvious to be argued.
Now let´s compare with peer reviewed literature on AGI:
We would not be destroyed in the first large nuclear war (while effects of radioactivity in the food chains in my view are under researched). But not a single open society would survive.
A world of malthusian masses, and a military aristocracy desperately trying to keep as much firepower as possible is the natural post nuclear war outcome. Then, history happens again, and in 2000 years luckily we are back into deciding if we allow AGI to be developed. What is the point?
The baseline is that our governance systems are completely unaccurate for nuclear weapons. Even in this fortunate age of hegemonic republics.
We need to solve the human alignement problem. Do you have any better suggestion than AGI?
I find it frustrating to correspond with you. You have become attached to an argument for what we should do. To support this argument, you send out many “soldiers”: nuclear war is a potent existential risk; electromagnetic pulse would destroy most electronic devices; not a single open society would survive a nuclear war; nuclear war inevitably leads to more nuclear war; nuclear war will cause widespread societal collapse. And now we have a new soldier, namely, the effect of radiation on the food chain.
Each of these soldiers seems plausible if one’s epistemology consists mostly in noticing how often something is repeated in the press and online. But I haven’t seen a single attempt by you to support any of these assertions / soldiers. When I say that Wikipedia says that most vehicles and cellphones would continue to operate after the electromagnetic pulses of a nuclear war, you ignore that. I offer you an opening to explain why you believe that nuclear war will lead to societal collapse whereas WWI, WWII and the Chinese civil war did not; you decline to engage on that. I still do not know whether you accept the conventional definition of “existential risk” (even after I asked you a direct question): when you wrote that nuclear war is a potent existential risk, maybe you thought that the possibility that half of the human population might die constitutes an existential risk. I.e., maybe you have been using an unconventional definition of existential risk. Your readers (including me) still do not know.
If I continue corresponding with you, I expect you would send out a few more soldiers, but it take me a lot more work to explain why a soldier does not in fact support your argument than it takes you to find the next soldier and to send it out.
Have you ever tried to learn about the effects of radiation on the food chain, e.g., by typing the phrase into a search engine and spending 5 minutes (as measured by an actual clock or timer) looking at the results? Science knows much about the subject. The radiation from an accident at a nuclear power plant is very different from the radiation from a nuclear weapon, so you’d have to be careful not to generalize from the first case to the second. (A much higher fraction of the radiation in the first case comes from long-half-life isotopes.)
Nuclear winter:
https://climate.envsci.rutgers.edu/pdf/WiresClimateChangeNW.pdf
Electromagnetic pulse would destroy most electronic devices:
https://doh.wa.gov/sites/default/files/legacy/Documents/Pubs/320-090_elecpuls_fs.pdf
“Commercial computer equipment is particularly vulnerable to EMP effects. Computers used in data processing systems, communications systems, displays, industrial control applications, including road and rail signaling, and those embedded in military equipment, such as signal processors, electronic flight controls and digital engine control systems, are all potentially vulnerable to the EMP effect. Other electronic devices and electrical equipment may also be destroyed by the EMP effect. Telecommunications equipment can be highly vulnerable and receivers of all varieties are particularly sensitive to EMP. Therefore radar and electronic warfare equipment, satellite, microwave, UHF, VHF, HF and low band communications equipment and television equipment are all potentially vulnerable to the EMP effect. Cars with electronic ignition systems/ and ignition chips are also vulnerable.”
not a single open society would survive a nuclear war; nuclear war inevitably leads to more nuclear war; nuclear war will cause widespread societal collapse
Do you expect a paper? I have this one:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/
:-) After total economic disruption, no electricity nor electronics, and a few billion deaths… its like the parachute randomized trial. Too obvious to be argued.
Now let´s compare with peer reviewed literature on AGI:
https://marginalrevolution.com/marginalrevolution/2023/04/from-the-comments-on-ai-safety.html
“The only peer-reviewed paper making the case for AI risk that I know of is: https://onlinelibrary.wiley.com/doi/10.1002/aaai.12064. Though note that my paper (the second you linked) is currently under review at a top ML conference.”