“I tentatively guess that if Singapore were to become a thriving hub for AI risk reduction, this would reduce AI risk by 16%.”
The units on this claim seem bad. There is a big difference between 50% → 34% and 99% → 83%. I’m not sure if I would endorse this if I thought about it more, but maybe good units for a claim like the number of bits of evidence the update is equivalent to. Going from 50% to 33% would be the same as going from 99% to 98% (1 bit).
Hmmm. I’m confused about this. My tentative thoughts are as follows:
--For EUmax we care about the raw difference in probabilities.
--However, I don’t have a good sense of that difference. I don’t know whether base AI risk is 99% or 1%. All I know (or all I guess, even) is that a hub in Singapore reduces AI risk by 16%. So that would be ~16% in the former case and ~0.16% in the latter.
--However, this seems actually fine, because we are choosing between options like “Go to Singapore” and “Stay in the Bay” and the common currency we use to measure both options is how much AI risk reduction we can do.
I interpreted this as a relative reduction of the probability (P_new=0.84*P_old) rather than an absolute decrease of the probability by 0.16. However, this indicates that the claim might be ambiguous which is problematic in another way.
Yep, that seems right. How would one do the calculation in bits? (EDIT: Not sure anymore; DanielFilan reminded me why I chose this metric in the first place...)
The number of bits of evidence is log2(p1/(1−p1))−log2(p0/(1−p0)).
Basically, if you view the probability as a fraction, p/(p+q), then a positive bit of evidence doubles p, while a negative bit of evidence doubles q.
You can imagine the ruler: 3% 6% 11% 20% 33% 50% 67% 80% 89% 94% 97%. Each is one more bit than the last. For probabilities near 0, it is roughly counting doublings, for probabilities near 1, it is counting doublings of the complement.
The units on this claim seem bad. There is a big difference between 50% → 34% and 99% → 83%. I’m not sure if I would endorse this if I thought about it more, but maybe good units for a claim like the number of bits of evidence the update is equivalent to. Going from 50% to 33% would be the same as going from 99% to 98% (1 bit).
But for EU maximization you really do care about the raw difference in probabilities!
Hmmm. I’m confused about this. My tentative thoughts are as follows:
--For EUmax we care about the raw difference in probabilities.
--However, I don’t have a good sense of that difference. I don’t know whether base AI risk is 99% or 1%. All I know (or all I guess, even) is that a hub in Singapore reduces AI risk by 16%. So that would be ~16% in the former case and ~0.16% in the latter.
--However, this seems actually fine, because we are choosing between options like “Go to Singapore” and “Stay in the Bay” and the common currency we use to measure both options is how much AI risk reduction we can do.
Yeah, I had this in mind I said I’m not sure if I would endorse this if I thought about it more. I am still uncertain.
I guess the bit reduction estimate is probably more portable across different people’s models, which is nice?
I interpreted this as a relative reduction of the probability (P_new=0.84*P_old) rather than an absolute decrease of the probability by 0.16. However, this indicates that the claim might be ambiguous which is problematic in another way.
You interpreted it correctly.
I apparently did not!
Yep, that seems right. How would one do the calculation in bits? (EDIT: Not sure anymore; DanielFilan reminded me why I chose this metric in the first place...)
The number of bits of evidence is log2(p1/(1−p1))−log2(p0/(1−p0)).
Basically, if you view the probability as a fraction, p/(p+q), then a positive bit of evidence doubles p, while a negative bit of evidence doubles q.
You can imagine the ruler: 3% 6% 11% 20% 33% 50% 67% 80% 89% 94% 97%. Each is one more bit than the last. For probabilities near 0, it is roughly counting doublings, for probabilities near 1, it is counting doublings of the complement.
Thanks! I hope to use this for raising my daughter. When teaching forecasting and bayesian reasoning.