If you’re just taking into account P(AGI in year t) and P(doom | AGI in year t), I think you should weight by probability times leverage. So weight AGI in year t by P(AGI in year t) * (P(doom | AGI in year t) - P(doom | AGI in year t)^2).
Certainly ignoring P(doom) is wrong, and certainly the asymmetry where you condition on success is wrong (conversely: why not condition on alignment failure because those are the worlds that need you to work on them) (or: notice that you’re giving most weight to the worlds with lowest P(doom), when a world with extremely low P(doom) doesn’t need you much in expectation; you have more influence over worlds with P(doom) close to 50%), it seems to me.
If you’re just taking into account P(AGI in year t) and P(doom | AGI in year t), I think you should weight by probability times leverage. So weight AGI in year t by P(AGI in year t) * (P(doom | AGI in year t) - P(doom | AGI in year t)^2).
Certainly ignoring P(doom) is wrong, and certainly the asymmetry where you condition on success is wrong (conversely: why not condition on alignment failure because those are the worlds that need you to work on them) (or: notice that you’re giving most weight to the worlds with lowest P(doom), when a world with extremely low P(doom) doesn’t need you much in expectation; you have more influence over worlds with P(doom) close to 50%), it seems to me.