I mostly try to look around to who’s saying what and why and find that the people I consider most thoughtful tend to be more concerned and take “the weak argument” or variations thereof very seriously (as do I). It seems like the “expert consensus” here (as in the poll) is best seen as some sort of evidence rather than a base rate, and one can argue how much to update on it.
That said, there’s a few people who seem less overall concerned about near-term doom and who I take seriously as thinkers on the topic. Carl Shulman being perhaps the most notable.
I mostly try to look around to who’s saying what and why and find that the people I consider most thoughtful tend to be more concerned and take “the weak argument” or variations thereof very seriously
We apparently have different tastes in “people I consider thoughtful”. “Here are some people I like and their opinions” is an argument unlikely to convince me (a stranger).
metaculus did a study where they compared prediction markets with a small number of participants to those with a large number and found that you get most of the benefit at relative small numbers (10 or so). So if you randomly sample 10 AI experts and survey their opinions, you’re doing almost as good as a full prediction market. The fact that multiple AI markets (metaculus, manifold) and surveys all agree on the same 5-10% suggests that none of these methodologies is wildly flawed.
I mean it only suggests that they’re highly correlated. I agree that it seems likely they represent the views of the average “AI expert” in this case. (I should take a look to check who was actually sampled)
My main point regarding this is that we probably shouldn’t be paying this particular prediction market too much attention in place of e.g. the survey you mention. I probably also wouldn’t give the survey too much weight compared to opinions of particularly thoughtful people, but I agree that this needs to be argued.
I mostly try to look around to who’s saying what and why and find that the people I consider most thoughtful tend to be more concerned and take “the weak argument” or variations thereof very seriously (as do I). It seems like the “expert consensus” here (as in the poll) is best seen as some sort of evidence rather than a base rate, and one can argue how much to update on it.
That said, there’s a few people who seem less overall concerned about near-term doom and who I take seriously as thinkers on the topic. Carl Shulman being perhaps the most notable.
We apparently have different tastes in “people I consider thoughtful”. “Here are some people I like and their opinions” is an argument unlikely to convince me (a stranger).
Who do you consider thoughtful on this issue?
It’s more like “here are some people who seem to have good opinions”, and that would certainly move the needle for me.
No one. I trust prediction markets far more than any single human being.
In general, yes—but see the above (I.e. we don’t have a properly functioning prediction market on the issue).
metaculus did a study where they compared prediction markets with a small number of participants to those with a large number and found that you get most of the benefit at relative small numbers (10 or so). So if you randomly sample 10 AI experts and survey their opinions, you’re doing almost as good as a full prediction market. The fact that multiple AI markets (metaculus, manifold) and surveys all agree on the same 5-10% suggests that none of these methodologies is wildly flawed.
I mean it only suggests that they’re highly correlated. I agree that it seems likely they represent the views of the average “AI expert” in this case. (I should take a look to check who was actually sampled)
My main point regarding this is that we probably shouldn’t be paying this particular prediction market too much attention in place of e.g. the survey you mention. I probably also wouldn’t give the survey too much weight compared to opinions of particularly thoughtful people, but I agree that this needs to be argued.