But it wasn’t rational for me to do that, I was just doing it to prove my seriousness.
My offer was also in this spirit of you proving your seriousness. Feel free to suggest bets which would be rational for you to take. Do you think there is a significant risk of a large AI catastrophe in the next few years? For example, what do you think is the probability of human population decreasing from (mid) 2026 to (mid) 2027?
You are basically asking me to give up money in expectation to prove that I really believe what I’m saying, when I’ve already done literally this multiple times. (And besides, hopefully it’s pretty clear that I am serious from my other actions.) So, I’m leaning against doing this, sorry. If you have an idea for a bet that’s net-positive for me I’m all ears.
Yes I do think there’s a significant risk of large AI catastrophe in the next few years. To answer your specific question, maybe something like 5%? idk.
Good question. I guess I’m at 30%, so 2x higher? Low confidence haven’t thought about it much, there’s a lot of uncertainty about what METR/ARC will classify as success, and I also haven’t reread ARC/METR’s ARA eval to remind myself of how hard it is.
Have your probabilities for AGI on given years changed at all since this breakdown you gave 7 months ago? I, and I’m sure many others, defer quite a lot to your views on timelines, so it would be good to have an updated breakdown.
Thanks, Daniel. That makes sense.
My offer was also in this spirit of you proving your seriousness. Feel free to suggest bets which would be rational for you to take. Do you think there is a significant risk of a large AI catastrophe in the next few years? For example, what do you think is the probability of human population decreasing from (mid) 2026 to (mid) 2027?
You are basically asking me to give up money in expectation to prove that I really believe what I’m saying, when I’ve already done literally this multiple times. (And besides, hopefully it’s pretty clear that I am serious from my other actions.) So, I’m leaning against doing this, sorry. If you have an idea for a bet that’s net-positive for me I’m all ears.
Yes I do think there’s a significant risk of large AI catastrophe in the next few years. To answer your specific question, maybe something like 5%? idk.
Are you much higher than Metaculus’ community on Will ARC find that GPT-5 has autonomous replication capabilities??
Good question. I guess I’m at 30%, so 2x higher? Low confidence haven’t thought about it much, there’s a lot of uncertainty about what METR/ARC will classify as success, and I also haven’t reread ARC/METR’s ARA eval to remind myself of how hard it is.
Have your probabilities for AGI on given years changed at all since this breakdown you gave 7 months ago? I, and I’m sure many others, defer quite a lot to your views on timelines, so it would be good to have an updated breakdown.
15% − 2024
15% − 2025
15% − 2026
10% − 2027
5% − 2028
5% − 2029
3% − 2030
2% − 2031
2% − 2032
2% − 2033
2% − 2034
2% − 2035
My 2024 probability has gone down from 15% to 5%. Other than that things are pretty similar, so just renormalize I guess.