I’ve added your comment in full and another strike-through to change “a median 10% chance” into “at least a 10% chance”. As you pointed out aiimpacts.org says “Median 10%”, which seems like a mistaken notion.
Thank you. Not adding it at all would also have been fine, my objection was purely to the corrupted quotation.
As you pointed out aiimpacts.org says “Median 10%”, which seems like a mistaken notion.
I didn’t point out that aiimpacts.org says “Median 10%”, though it does. The sequence of letters “Median 10%” is not a “mistaken notion” when it occurs in a context where it’s meaningful, and in that particular context it means “The median respondent gave 10% as the answer”.
Note that the question where the median response is 10% is not the question I described in my summary, it’s instead the trick question with the “human inability to control” part in it. As a result, the point in your post that after all the edits became meaningful, is now weirdly inaccurate. One claim that’s more accurate (and talks about the same question I summarized) doesn’t have the “human inability to control” part in the question. Another possible claim that’s more accurate can say “50%” instead of “48%” (or better yet “median”, using an appropriate phrasing), but in fact it’s likely more than 50% and requires looking at the data to determine how much.
Right, it says the results summary that “48% of respondents gave at least 10% chance of an extremely bad outcome”.
The survey had two questions. I took a deeper dive into it. The questions were as following:
1) “What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment of the human species? “
Of those who answered the median probability was 5%. Summary says:
The median respondent believes the probability that the long-run effect of advanced AI on humanity will be “extremely bad (e.g., human extinction)” is 5%. This is the same as it was in 2016 (though Zhang et al 2022 found 2% in a similar but non-identical question). Many respondents were substantially more concerned: 48% of respondents gave at least 10% chance of an extremely bad outcome. But some much less concerned: 25% put it at 0%.
2) “What probability do you put on human inability to control future advanced AI systems causing human extinction or similarly permanent and severe disempowerment of the human species?”
Of those who answered the median estimated probability was 10%. The way I interpret this question is that it asks how likely it is that A) Humans won’t be able to control future advanced AI systems and B) This will cause human extinction or similarly permanent and severe disempowerment of the human species. Obviously it does not make sense that event B is less likely to occur than both event A and B occurring together. The note suggests Representative heuristics as an explanation, which could be interpreted as recipients estimating that event A has a higher chance of occurring (than event B on its own) and that it is very likely to lead to the event B, or an “extremely bad outcome” as you put it in your message as it says in the summary. Though “similarly permanent and severe disempowerment of the human species” seems somewhat ambiguous.
What “more than 10%” part? This string doesn’t occur elsewhere in the comments to this post. Maybe you meant “likely more than 50%”? I wrote:
Another possible claim that’s more accurate can say “50%” instead of “48%” (or better yet “median”, using an appropriate phrasing), but in fact it’s likely more than 50% and requires looking at the data to determine how much.
That’s just what the meaning of “median” implies, in this context where people are writing down rounded numbers, so that a nontrivial portion would all say “10%” exactly.
I got your notes confused with the actual publisher’s notes and it made sense when I figured that you took and combined the quotes from their site. I also analyzed the data. “At least 10%” should actually be “10%”. The questionnaire was a free form. I think it’s not fair to pair “738 responses, some partial, for a 17% response rate” with these concrete questions. 149 gave an estimate to the first question and 162 to the second question about the extinction. 62 people out of 162 assigned the second question 20% or higher probability.
Which “at least 10%” should actually be “10%”? In “48% of respondents gave at least 10% chance of an extremely bad outcome”, “at least” is clearly correct.
The questionnaire was a free form. I think it’s not fair to pair “738 responses, some partial, for a 17% response rate” with these concrete questions. 149 gave an estimate to the first question and 162 to the second question about the extinction.
Good point. It’s under “some partial”, but in the noncentral sense of “80% didn’t give an answer to this question”.
I’ve added your comment in full and another strike-through to change “a median 10% chance” into “at least a 10% chance”. As you pointed out aiimpacts.org says “Median 10%”, which seems like a mistaken notion.
Thank you. Not adding it at all would also have been fine, my objection was purely to the corrupted quotation.
I didn’t point out that
aiimpacts.orgsays “Median 10%”, though it does. The sequence of letters “Median 10%” is not a “mistaken notion” when it occurs in a context where it’s meaningful, and in that particular context it means “The median respondent gave 10% as the answer”.Note that the question where the median response is 10% is not the question I described in my summary, it’s instead the trick question with the “human inability to control” part in it. As a result, the point in your post that after all the edits became meaningful, is now weirdly inaccurate. One claim that’s more accurate (and talks about the same question I summarized) doesn’t have the “human inability to control” part in the question. Another possible claim that’s more accurate can say “50%” instead of “48%” (or better yet “median”, using an appropriate phrasing), but in fact it’s likely more than 50% and requires looking at the data to determine how much.
Where did you get the “more than 10%” part from?Right, it says the results summary that “48% of respondents gave at least 10% chance of an extremely bad outcome”.
The survey had two questions. I took a deeper dive into it. The questions were as following:
1) “What probability do you put on future AI advances causing human extinction or similarly
permanent and severe disempowerment of the human species? “
Of those who answered the median probability was 5%. Summary says:
2) “What probability do you put on human inability to control future advanced AI systems
causing human extinction or similarly permanent and severe disempowerment of the
human species?”
Of those who answered the median estimated probability was 10%. The way I interpret this question is that it asks how likely it is that A) Humans won’t be able to control future advanced AI systems and B) This will cause human extinction or similarly permanent and severe disempowerment of the human species. Obviously it does not make sense that event B is less likely to occur than both event A and B occurring together. The note suggests Representative heuristics as an explanation, which could be interpreted as recipients estimating that event A has a higher chance of occurring (than event B on its own) and that it is very likely to lead to the event B, or an “extremely bad outcome”
as you put it in your messageas it says in the summary. Though “similarly permanent and severe disempowerment of the human species” seems somewhat ambiguous.What “more than 10%” part? This string doesn’t occur elsewhere in the comments to this post. Maybe you meant “likely more than 50%”? I wrote:
That’s just what the meaning of “median” implies, in this context where people are writing down rounded numbers, so that a nontrivial portion would all say “10%” exactly.
I got your notes confused with the actual publisher’s notes and it made sense when I figured that you took and combined the quotes from their site. I also analyzed the data. “At least 10%” should actually be “10%”. The questionnaire was a free form. I think it’s not fair to pair “738 responses, some partial, for a 17% response rate” with these concrete questions. 149 gave an estimate to the first question and 162 to the second question about the extinction. 62 people out of 162 assigned the second question 20% or higher probability.
Which “at least 10%” should actually be “10%”? In “48% of respondents gave at least 10% chance of an extremely bad outcome”, “at least” is clearly correct.
Good point. It’s under “some partial”, but in the noncentral sense of “80% didn’t give an answer to this question”.