At the very least it would need to be on a logistic scale (i.e. in odds form). If something has 2:1 odds, then if I am at 50% then that’s a 25% probability mass movement, but when I am at 99% only a ~1% probability movement.
But my guess is even odds are too hard to assign to stuff like this.
After chatting with our voices, I believe you misunderstood me as asking “How many points of probability does this argument move you?” whereas I meant “If you had not seen any of the other arguments, and you were presented with this argument, what would your absolute probability of an existential catastrophe from AI be?”
I don’t stand by this as the ultimate bestest survey question of all time, and would welcome an improvement.
I’ll say I think the alternative of “What likelihood ratio does this argument provide?” (e.g. 2x, 10x, etc) isn’t great because I believe the likelihood ratios of all the arguments are not conditionally independent.
At the very least it would need to be on a logistic scale (i.e. in odds form). If something has 2:1 odds, then if I am at 50% then that’s a 25% probability mass movement, but when I am at 99% only a ~1% probability movement.
But my guess is even odds are too hard to assign to stuff like this.
After chatting with our voices, I believe you misunderstood me as asking “How many points of probability does this argument move you?” whereas I meant “If you had not seen any of the other arguments, and you were presented with this argument, what would your absolute probability of an existential catastrophe from AI be?”
I don’t stand by this as the ultimate bestest survey question of all time, and would welcome an improvement.
I’ll say I think the alternative of “What likelihood ratio does this argument provide?” (e.g. 2x, 10x, etc) isn’t great because I believe the likelihood ratios of all the arguments are not conditionally independent.