Typically, this reasoning doesn’t work because we have to update once again based on our current age and on the fact that such technologies do not yet exist, which compensates for the update in the direction of “Life extension is developing and alignment is easy.”
This is easier to understand through the Sleeping Beauty problem. She wakes up once on Monday if it’s heads, and on both Monday and Tuesday if it’s tails. The first update suggests that tails is two times more likely, so the probability becomes 2⁄3. However, as people typically argue, after learning that it is Monday, she needs to update back to 1⁄3, which yields the same probability for both tails and heads.
But in the two-thirders’ position, we reject the second update because Tails-Monday and Tails-Tuesday are not independent events (as was recently discussed on LessWrong in the Sleeping Beauty series).
Typically, this reasoning doesn’t work because we have to update once again based on our current age and on the fact that such technologies do not yet exist, which compensates for the update in the direction of “Life extension is developing and alignment is easy.”
This is easier to understand through the Sleeping Beauty problem. She wakes up once on Monday if it’s heads, and on both Monday and Tuesday if it’s tails. The first update suggests that tails is two times more likely, so the probability becomes 2⁄3. However, as people typically argue, after learning that it is Monday, she needs to update back to 1⁄3, which yields the same probability for both tails and heads.
But in the two-thirders’ position, we reject the second update because Tails-Monday and Tails-Tuesday are not independent events (as was recently discussed on LessWrong in the Sleeping Beauty series).