I can also come up with a story where obviously it’s cheaper and more effective to disable all of the nuclear weapons than it is to take over the world, so why would the AI do the second thing?
Erm… For preventing nuclear war on the scale of decades… I don’t know what you have in mind for how it would disable all the nukes, but a one-off breaking of all the firing mechanisms isn’t going to work. They could just repair/replace that once they discovered the problem. You could imagine some more drastic thing like blowing up the conventional explosives on the missiles so as to utterly ruin them, but in a way that doesn’t trigger the big chain reaction. But my impression is that, if you have a pile of weapons-grade uranium, then it’s reasonably simple to make a bomb out of it, and since uranium is an element, no conventional explosion can eliminate that from the debris. Maybe you can melt it, mix it with other stuff, and make it super-impure?
But even then, the U.S. and Russia probably have stockpiles of weapons-grade uranium. I suspect they could make nukes out of that within a few months. You would have to ruin all the stockpiles too.
And then there’s the possibility of mining more uranium and enriching it; I feel like this would take a few years at most, possibly much less if one threw a bunch of resources into rushing it. Would you ruin all uranium mines in the world somehow?
No, it seems to me that the only ways to reliably rule out nuclear war involve either using overwhelming physical force to prevent people from using or making nukes (like a drone army watching all the uranium stockpiles), or being able to reliably persuade the governments of all nuclear powers in the world to disarm and never make any new nukes. The power to do either of these things seems tantamount to the power to take over the world.
Interesting. The natural approach is to imagine that you just have a 3-sided die with 2, 4, 6 on the sides, and if you do that, then I compute A = 12 and B = 6[1]. But, as the top Reddit comment’s edit points out, the difference between that problem and the one you posed is that your version heavily weights the probability towards short sequences—that weighting being 1/2^n for a sequence of length n. (Note that the numbers I got, A=12 and B=6, are so much higher than the A≈2.7 and B=3 you get.) It’s an interesting selection effect.
The thing is that, if you roll a 6 and then a non-6, in an “A” sequence you’re likely to just die due to rolling an odd number before you succeed in getting the double 6, and thus exclude the sequence from the surviving set; whereas in a “B” sequence there’s a much higher chance you’ll roll a 6 before dying, and thus include this longer “sequence of 3+ rolls” in the set.
To illustrate with an extreme version, consider:
Obviously that’s one way to reduce A to 2.
Excluding odd rolls completely, so the die has a 1⁄3 chance of rolling 6 and a 2⁄3 chance of rolling an even number that’s not 6, we have:
A = 1 + 1⁄3 * A2 + 2⁄3 * A
Where A2 represents “the expected number of die rolls until you get two 6′s in a row, given that the last roll was a 6”. Subtraction and multiplication then yields:
A = 3 + A2
And if we consider rolling a die from the A2 state, we get:
A2 = 1 + 1⁄3 * 0 + 2⁄3 * A
= 1 + 2⁄3 * A
Substituting:
A = 3 + 1 + 2⁄3 * A
=> (subtract)
1⁄3 * A = 4
=> (multiply)
A = 12
For B, a similar approach yields the equations:
B = 1 + 1⁄3 * B2 + 2⁄3 * B
B2 = 1 + 1⁄3 * 0 + 2⁄3 * B2
And the reader may solve for B = 6.