Er, yes. AI risk worriers think AI will cause human extinction . Unless they believe in God, surely all morality stems from humanity, so the extinction of the species must be the ultimate harm—and preventing it surely justifies violence (if it doesn’t, then what does?)
If you hypothetically have a situation where it’s a 100% clear that the human race will go extinct unless a violent act is committed, and it’s seems likely that the violent act would prevent human extinction, then, in that hypothetical case, that would be a strong consideration in favour of committing the violent act.
In reality though, this clarity is extremely unlikely, and unilateral actions are likely to have negative side effects. Moreover, even if you think you have such clarity, it’s likely that you are mistaken, and the negative side effects still apply no matter how well justified you personally thought your actions were, if others don’t agree.
OK, so then AI doomers admit it’s likely they’re mistaken?
(Re side effects, no matter how negative they are, they’re better than the alternative; and it doesn’t even have to be likely that violence would work: if doomers really believe P(doom) is 1, then any action with a non-zero probability of success is worth pursuing.)
You’re assuming “the violence might or might not stop extinction, but then there will be some side-effects (that are unrelated to extinction)”. But, my concrete belief is that most acts of violence you could try to commit would probably make extinction more likely, not less, because a) they wouldn’t work, and b) they destroy the trust and coordination mechanisms necessary for the world to actually deal with the problem.
To spell out a concrete example: someone tries bombing an AI lab. Maybe they succeed, maybe they don’t. Either way, they didn’t actually stop the development of AI because other labs will still continue the work. But now, when people are considering who to listen to about AI safety, the “AI risk is high” people get lumped in with crazy terrorists and sidelined.
I don’t know! I’ve certainly seen people say P(doom) is 1, or extremely close. And anyway, bombing an AI lab wouldn’t stop progress, but would slow it down—and if you think there is a chance alignment will be solved, the more time you buy the better.
If you think P(doom) is 1, you probably don’t believe that terrorist bombing of anything will do enough damage to be useful. That is probably one of EYs cruxes on violence.
I am not an extreme doomer, but part of that is that I expect that people will face things more realistically over time—something that violence, introducing partisanship and division, would set back considerably. But even for an actual doomer, the “make things better through violence” option is not an especially real option.
You may have a fantasy of choosing between these options:
doom
heroically struggle against the doom through glorious violence
But you are actually choosing between:
a dynamic that’s likely by default to lead to doom at some indefinite time in the future by some pathway we can’t predict the details of until it’s too late
make the situation even messier through violence, stirring up negative attitudes towards your cause, especially among AI researchers but also among the public, making it harder to achieve any collective solution later, sealing the fate of humanity even more thoroughly
Let me put it this way. To the extent that you have p(doom) = 1 - epsilon, where is epsilon coming from? If it’s coming from “terrorist attacks successully stop capability research” then I guess violence might make sense from that perspective but I would question your sanity. If relatively more of that epsilon is coming from things like “international agreements to stop AI capabilities” or “AI companies start taking x-risk more seriously”, which I would think would be more realistic, then don’t ruin the chances of that through violence.
Except that violence doesn’t have to stop the AI labs, it just has to slow them down: if you think that international agreements yada yada have a chance of success, and given this takes time, then things like cyber attacks that disrupt AI research can help, no?
I think you are overestimating the efficacy and underestimating the side effects of such things. How much do you expect a cyber attack to slow things down? Maybe a week if it’s very successful? Meanwhile it still stirs up opposition and division, and puts diplomatic efforts back years.
As the gears to ascension notes, non-injurious acts of aggression share many game theoretic properties as physical violence. I would express the key issue here as legitimacy; if you don’t have legitimacy, acting unilaterally puts you in conflict with the rest of humanity and doesn’t get you legitimacy, but once you do have legitimacy you don’t need to act unilaterally, you can get a ritual done that causes words to be written on a piece of paper where people with badges and guns will come to shut down labs that do things forbidden by those words. Cool huh? But if someone just goes ahead and takes illegitimate unilateral action, or appears to be too willing to do so, that puts them into a conflict position where they and people associated with them won’t get to do the legitimate thing.
Everyone has been replying as though you mean physical violence; non-injurious acts of aggression don’t qualify as violence unambiguously, but share many game theoretic properties. If classical liberal coordination can be achieved even temporarily it’s likely to be much more effective at preventing doom.
Even in a crowd of ai doomers, no one person speaks for ai doomers. But plenty think it likely they’re mistaken somehow. I personally just think the big labs aren’t disproportionately likely to be the cause of an extinction strength ai, so violence is overdeterminedly off the table as an effective strategy, before even considering whether it’s justified, legal, or understandable. The only way we solve this is by constructing the better world.
If it’s true AI labs aren’t likely to be the cause of extinction, why is everyone upset at the arms race they’ve begun?
You can’t have it both ways: either the progress these labs are making is scary—in which case anything that disrupts them (and hence slows them down even if it doesn’t stop them) is good—or they’re on the wrong track, in which case we’re all fine.
I refer back to the first sentence of the message you’re replying to. I’m not having it both ways, you’re confusing different people’s opinions. My view is the only thing remarkable about labs is that they get to this slightly sooner by having bigger computers; even killing everyone at every big lab wouldn’t undo how much compute there is in the world, so it at most buys a year at an intense cost to rule morality and to knowledge of how to stop disaster. If you disagree with an argument someone else made, lay it out, please. I probably simply never agreed with the other person’s doom model anyway.
Er, yes. AI risk worriers think AI will cause human extinction . Unless they believe in God, surely all morality stems from humanity, so the extinction of the species must be the ultimate harm—and preventing it surely justifies violence (if it doesn’t, then what does?)
If you hypothetically have a situation where it’s a 100% clear that the human race will go extinct unless a violent act is committed, and it’s seems likely that the violent act would prevent human extinction, then, in that hypothetical case, that would be a strong consideration in favour of committing the violent act.
In reality though, this clarity is extremely unlikely, and unilateral actions are likely to have negative side effects. Moreover, even if you think you have such clarity, it’s likely that you are mistaken, and the negative side effects still apply no matter how well justified you personally thought your actions were, if others don’t agree.
OK, so then AI doomers admit it’s likely they’re mistaken?
(Re side effects, no matter how negative they are, they’re better than the alternative; and it doesn’t even have to be likely that violence would work: if doomers really believe P(doom) is 1, then any action with a non-zero probability of success is worth pursuing.)
You’re assuming “the violence might or might not stop extinction, but then there will be some side-effects (that are unrelated to extinction)”. But, my concrete belief is that most acts of violence you could try to commit would probably make extinction more likely, not less, because a) they wouldn’t work, and b) they destroy the trust and coordination mechanisms necessary for the world to actually deal with the problem.
To spell out a concrete example: someone tries bombing an AI lab. Maybe they succeed, maybe they don’t. Either way, they didn’t actually stop the development of AI because other labs will still continue the work. But now, when people are considering who to listen to about AI safety, the “AI risk is high” people get lumped in with crazy terrorists and sidelined.
But when you say extinction will be more likely, you must believe that the probability of extinction is not 1.
Well… Yeah? Would any of us care to build knowledge that improves our odds if our odds were immovably terrible?
I don’t know! I’ve certainly seen people say P(doom) is 1, or extremely close. And anyway, bombing an AI lab wouldn’t stop progress, but would slow it down—and if you think there is a chance alignment will be solved, the more time you buy the better.
If you think P(doom) is 1, you probably don’t believe that terrorist bombing of anything will do enough damage to be useful. That is probably one of EYs cruxes on violence.
I am not an extreme doomer, but part of that is that I expect that people will face things more realistically over time—something that violence, introducing partisanship and division, would set back considerably. But even for an actual doomer, the “make things better through violence” option is not an especially real option.
You may have a fantasy of choosing between these options:
doom
heroically struggle against the doom through glorious violence
But you are actually choosing between:
a dynamic that’s likely by default to lead to doom at some indefinite time in the future by some pathway we can’t predict the details of until it’s too late
make the situation even messier through violence, stirring up negative attitudes towards your cause, especially among AI researchers but also among the public, making it harder to achieve any collective solution later, sealing the fate of humanity even more thoroughly
Let me put it this way. To the extent that you have p(doom) = 1 - epsilon, where is epsilon coming from? If it’s coming from “terrorist attacks successully stop capability research” then I guess violence might make sense from that perspective but I would question your sanity. If relatively more of that epsilon is coming from things like “international agreements to stop AI capabilities” or “AI companies start taking x-risk more seriously”, which I would think would be more realistic, then don’t ruin the chances of that through violence.
Except that violence doesn’t have to stop the AI labs, it just has to slow them down: if you think that international agreements yada yada have a chance of success, and given this takes time, then things like cyber attacks that disrupt AI research can help, no?
I think you are overestimating the efficacy and underestimating the side effects of such things. How much do you expect a cyber attack to slow things down? Maybe a week if it’s very successful? Meanwhile it still stirs up opposition and division, and puts diplomatic efforts back years.
As the gears to ascension notes, non-injurious acts of aggression share many game theoretic properties as physical violence. I would express the key issue here as legitimacy; if you don’t have legitimacy, acting unilaterally puts you in conflict with the rest of humanity and doesn’t get you legitimacy, but once you do have legitimacy you don’t need to act unilaterally, you can get a ritual done that causes words to be written on a piece of paper where people with badges and guns will come to shut down labs that do things forbidden by those words. Cool huh? But if someone just goes ahead and takes illegitimate unilateral action, or appears to be too willing to do so, that puts them into a conflict position where they and people associated with them won’t get to do the legitimate thing.
Everyone has been replying as though you mean physical violence; non-injurious acts of aggression don’t qualify as violence unambiguously, but share many game theoretic properties. If classical liberal coordination can be achieved even temporarily it’s likely to be much more effective at preventing doom.
Even in a crowd of ai doomers, no one person speaks for ai doomers. But plenty think it likely they’re mistaken somehow. I personally just think the big labs aren’t disproportionately likely to be the cause of an extinction strength ai, so violence is overdeterminedly off the table as an effective strategy, before even considering whether it’s justified, legal, or understandable. The only way we solve this is by constructing the better world.
If it’s true AI labs aren’t likely to be the cause of extinction, why is everyone upset at the arms race they’ve begun?
You can’t have it both ways: either the progress these labs are making is scary—in which case anything that disrupts them (and hence slows them down even if it doesn’t stop them) is good—or they’re on the wrong track, in which case we’re all fine.
I refer back to the first sentence of the message you’re replying to. I’m not having it both ways, you’re confusing different people’s opinions. My view is the only thing remarkable about labs is that they get to this slightly sooner by having bigger computers; even killing everyone at every big lab wouldn’t undo how much compute there is in the world, so it at most buys a year at an intense cost to rule morality and to knowledge of how to stop disaster. If you disagree with an argument someone else made, lay it out, please. I probably simply never agreed with the other person’s doom model anyway.