I believe there is minimal chance that all the superpowers will simultaneously agree on a meaningful ai pause. It seems like you agree the same way. A superpower cannot be stopped by the measures you mentioned, they will train new ai experts, build their own chip fabrication equipment, build lots of spare capacity etc. Iran is not a superpower.
I think there is some dispute over what we even mean by “AGi/ASI”. I am thinking of any system that scores above a numerical threshold on a large benchmark of tasks, where the majority of the score comes from complex multimodal tasks that are withheld. AGI means the machine did at least as well as humans on a broad selection of these tasks, ASI means it beat the best human experts on a broad selection of the tasks.
Any machine able to do the above counts.
Note you can pass such tasks without situational or context awareness or ongoing continuity of existence or self modification or online learning. (All forms of state buildup)
So this is a major split in our models I think. I am thinking of an arms race that builds tool ASI without the state above, and I think you are assuming past a certain point the AI systems will have context awareness and the ability to coordinate among each other?
Like is that what drives your doom assumptions and assumptions that people would stop? Do you think decision-makers would avoid investing in tool AI that they have a high confidence they can control? (The confidence would come from controlling context. An isolated model without context can’t even know it’s not still in training)
So ok here’s some convergence.
I believe there is minimal chance that all the superpowers will simultaneously agree on a meaningful ai pause. It seems like you agree the same way. A superpower cannot be stopped by the measures you mentioned, they will train new ai experts, build their own chip fabrication equipment, build lots of spare capacity etc. Iran is not a superpower.
I think there is some dispute over what we even mean by “AGi/ASI”. I am thinking of any system that scores above a numerical threshold on a large benchmark of tasks, where the majority of the score comes from complex multimodal tasks that are withheld. AGI means the machine did at least as well as humans on a broad selection of these tasks, ASI means it beat the best human experts on a broad selection of the tasks.
Any machine able to do the above counts.
Note you can pass such tasks without situational or context awareness or ongoing continuity of existence or self modification or online learning. (All forms of state buildup)
So this is a major split in our models I think. I am thinking of an arms race that builds tool ASI without the state above, and I think you are assuming past a certain point the AI systems will have context awareness and the ability to coordinate among each other?
Like is that what drives your doom assumptions and assumptions that people would stop? Do you think decision-makers would avoid investing in tool AI that they have a high confidence they can control? (The confidence would come from controlling context. An isolated model without context can’t even know it’s not still in training)