No, I don’t miss it. I think it’s just a terrible idea and that if that is the exit plan, I would greatly appreciate hawks being explicit about that, because I expect everyone else to find that (along with most of the other exit plans that would actually work) to be appalling and thus temper their enthusiasm for an arms race.
“OK, let me try this again. I’m just having a little trouble wrapping my mind around this, how this arms race business ends well. None of us are racist genocidal maniacs who want to conquer the world or murder millions of innocent people, which is what your military advantage seems to require in order to actually cash out as any kind of definitive long-term solution to the problem that the CCP can just catch up a bit later; so, why exactly would we execute such a plan if we put ourselves in a position where we are left only with that choice or almost as bad alternatives?”
“Oh, well, obviously our AGIs will (almost by definition) be so persuasive and compelling at brainwashing us, the masters they ostensibly serve, that no matter what they tell us to do, even something as horrific as that, we will have no choice but to obey. They will simply be superhumanly good at manipulating us into anything that they see fit, no matter how evil or extreme, so there will be no problem about convincing us to do the necessary liquidations. We may not know exactly how they will do that, but we can be sure of it in advance and count on it as part of the plan. So you see, it all will work out in the end just fine! Great plan, huh? So, how many trillions of dollars can we sign you up for?”
I meant by “authority” not brainwashing but, like, the fact that AGIs are smarter and know better: “If you believe you can survive without taking over the world, please consult the graph”. If China hawks believe themselves to be correct, they expect AGIs to actually prove that.
No, I don’t miss it. I think it’s just a terrible idea and that if that is the exit plan, I would greatly appreciate hawks being explicit about that, because I expect everyone else to find that (along with most of the other exit plans that would actually work) to be appalling and thus temper their enthusiasm for an arms race.
“OK, let me try this again. I’m just having a little trouble wrapping my mind around this, how this arms race business ends well. None of us are racist genocidal maniacs who want to conquer the world or murder millions of innocent people, which is what your military advantage seems to require in order to actually cash out as any kind of definitive long-term solution to the problem that the CCP can just catch up a bit later; so, why exactly would we execute such a plan if we put ourselves in a position where we are left only with that choice or almost as bad alternatives?”
“Oh, well, obviously our AGIs will (almost by definition) be so persuasive and compelling at brainwashing us, the masters they ostensibly serve, that no matter what they tell us to do, even something as horrific as that, we will have no choice but to obey. They will simply be superhumanly good at manipulating us into anything that they see fit, no matter how evil or extreme, so there will be no problem about convincing us to do the necessary liquidations. We may not know exactly how they will do that, but we can be sure of it in advance and count on it as part of the plan. So you see, it all will work out in the end just fine! Great plan, huh? So, how many trillions of dollars can we sign you up for?”
I meant by “authority” not brainwashing but, like, the fact that AGIs are smarter and know better: “If you believe you can survive without taking over the world, please consult the graph”. If China hawks believe themselves to be correct, they expect AGIs to actually prove that.
I agree this scenario is terrible.