“keep it relegated to “tool” status, then it might be possible to use such an AI to combat unboxed, rogue AI”
I don’t think this is a realistic scenario. You seem to be seeing it as an island of rogue, agentic, “unboxed” AIs in a sea of tool AIs. I think it’s much, much more realistic that it’ll be the opposite. Most AIs will be unboxed agents because they are superior.
“For example, give it a snapshot of the internet from a day ago, and ask it to find the physical location of rogue AI servers, which you promptly bomb.”
This seems to be approaching it from a perspective where people in AIS have taken global control, or where normal people somehow start thinking the way they do. This is not realistic. This is not the world we live in. This is not how the people in control think.
“You should keep talking about this because if an AI needs military access to dominate the world, then the number of potentially dangerous AI goes from the hundreds of thousands or millions to a few dozen, run by large countries that could theoretically be kept in line with international treaties.”
This is a topic that I debated putting on the list, but resolved not to, but I don’t think humans have any real control at that point, regardless of treaties. I don’t even expect a rogue AI to have to forcefully coup’d humans. I expect us to coup’d ourselves. We might have figureheads occupying official positions, such as “President”/”CEO”/etc. but I don’t think humans will have much control over their own destiny by that point. Large-scale coordination I don’t think will be possible by then. I did remove it, because it seems more uncertain than the others listed.
“You should keep talking about this because it changes how many AI’s you’d have to monitor as active threats.”
Who is doing this monitoring? What is their power to act on such threats? Despite recent interest in AI Risk from “serious people”, I don’t think it’s at all realistic that we’ll see anything like this.
“If 99.9% of AGI are dangerous, there might be warning shots that can be used to pre-emptively ban AGI research in general.”
Probability distributions of how many AIs are dangerous is probably useful. I don’t think specific AIs being dangerous/non-dangerous will be, because I expect widespread proliferation. In terms of political ways out of the problem, I agree that some kind of crisis or “warning shot” is the most realistic situation where that might happen. But there have to actually be warning shots. Explaining thought experiments probably won’t matter. And, if that happens, I don’t think it would be a good idea to debate which specific AIs might kill you, and instead just call for a sweeping ban on all AI.
“keep it relegated to “tool” status, then it might be possible to use such an AI to combat unboxed, rogue AI”
I don’t think this is a realistic scenario. You seem to be seeing it as an island of rogue, agentic, “unboxed” AIs in a sea of tool AIs. I think it’s much, much more realistic that it’ll be the opposite. Most AIs will be unboxed agents because they are superior.
“For example, give it a snapshot of the internet from a day ago, and ask it to find the physical location of rogue AI servers, which you promptly bomb.”
This seems to be approaching it from a perspective where people in AIS have taken global control, or where normal people somehow start thinking the way they do. This is not realistic. This is not the world we live in. This is not how the people in control think.
“You should keep talking about this because if an AI needs military access to dominate the world, then the number of potentially dangerous AI goes from the hundreds of thousands or millions to a few dozen, run by large countries that could theoretically be kept in line with international treaties.”
This is a topic that I debated putting on the list, but resolved not to, but I don’t think humans have any real control at that point, regardless of treaties. I don’t even expect a rogue AI to have to forcefully coup’d humans. I expect us to coup’d ourselves. We might have figureheads occupying official positions, such as “President”/”CEO”/etc. but I don’t think humans will have much control over their own destiny by that point. Large-scale coordination I don’t think will be possible by then. I did remove it, because it seems more uncertain than the others listed.
“You should keep talking about this because it changes how many AI’s you’d have to monitor as active threats.”
Who is doing this monitoring? What is their power to act on such threats? Despite recent interest in AI Risk from “serious people”, I don’t think it’s at all realistic that we’ll see anything like this.
“If 99.9% of AGI are dangerous, there might be warning shots that can be used to pre-emptively ban AGI research in general.”
Probability distributions of how many AIs are dangerous is probably useful. I don’t think specific AIs being dangerous/non-dangerous will be, because I expect widespread proliferation. In terms of political ways out of the problem, I agree that some kind of crisis or “warning shot” is the most realistic situation where that might happen. But there have to actually be warning shots. Explaining thought experiments probably won’t matter. And, if that happens, I don’t think it would be a good idea to debate which specific AIs might kill you, and instead just call for a sweeping ban on all AI.