Basic question because I haven’t thought about this deeply: in national security stuff, we often intentionally elide the difference between capabilities and intentions. The logic is: you can’t assume a capability won’t be used, so you should plan as-if it is intended to be used.
Should we adopt such a rule for AGI with regards to policy decision-making? (My guess is...probably not for threat assessment but probably yes for contingency planning?)
I think it depends on who you can reasonably expect to get control of such a system.
We didn’t assume that by building a nuclear arsenal that it would be used, because only one state actor was likely to have access to the option to use it.
If it’s likely that an AI/AGI will be stolen and copied many times, then we should assume that anything it can do, it will be told to do.
If we assume that there’s only a good chance one other state actor will get its hands on it, then we might assume the worst capabilities are unlikely to be used.
If it’s three or more state actors, it depends what states and exactly how well the diplomacy goes...
Basic question because I haven’t thought about this deeply: in national security stuff, we often intentionally elide the difference between capabilities and intentions. The logic is: you can’t assume a capability won’t be used, so you should plan as-if it is intended to be used.
Should we adopt such a rule for AGI with regards to policy decision-making? (My guess is...probably not for threat assessment but probably yes for contingency planning?)
I think it depends on who you can reasonably expect to get control of such a system.
We didn’t assume that by building a nuclear arsenal that it would be used, because only one state actor was likely to have access to the option to use it.
If it’s likely that an AI/AGI will be stolen and copied many times, then we should assume that anything it can do, it will be told to do.
If we assume that there’s only a good chance one other state actor will get its hands on it, then we might assume the worst capabilities are unlikely to be used.
If it’s three or more state actors, it depends what states and exactly how well the diplomacy goes...