If a person was asked point-blank about the risk AI takeover, and they gave an answer that implied the risk was lower than they think it is, in private, I would consider that a lie
[...]
That said, my guess is that many of the people that I’m thinking of, in these policy positions, if they were asked, point blank, might lie in exactly that way. I have no specific evidence of that, but it does seem like the most likely way many of them would respond, given their overall policy about communicating their beliefs.
As a relevant piece of evidence here, Jason Matheny, when asked point-blank in a senate committee hearing about “how concerned should we be about catastrophic risks from AI?” responded with “I don’t know”, which seems like it qualifies as a lie by the standard you set here (which, to be clear, I don’t super agree with and my intention here is partially to poke holes in your definition of a lie, while also sharing object-level relevant information).
Quote (slightly paraphrased because transcription is hard):
Senator Peters: “The last question before we close. We’ve heard thoughts from various experts about the risk of human-like artificial intelligence or Artificial General Intelligence, including various catastrophic projections. So my final question is, what is the risk that Artificial General Intelligence poses, and how likely is that to matter in the near future?”
[...]
Matheny: “As is typically my last words: I don’t know. I think it’s a really difficult question. I think whether AGI is nearer or farther than thought, I think there are things we can do today in either case. Including regulatory frameworks that include standards with third party tests and audits, governance of supply chains so we can understand where large amounts of computing is going, and so that we can prevent large amounts of computing going to places with lower ethical standards that we and other democracies have”
Given my best model of Matheny’s beliefs, this sure does not seem like an answer that accurately summarizes his beliefs here, and represents a kind of response that I think causes people to be quite miscalibrated about the beliefs of experts in the field.
In my experience people raise the hypothetical of “but they would be honest when asked point blank” to argue that people working in the space are not being deceptive. However, I have now seen people being asked point blank, and I haven’t seen them be more honest than their original evasiveness implied, so I think this should substantially increase people’s priors on people doing something more deceptive here.
Jason Matheny is approximately the most powerful person in the AI policy space. I think he is setting a precedent here for making statements that meet at least the definition of lying you set out in your comment (I am still unsure whether to count that as lying, though it sure doesn’t feel honest), and if-anything, if I talk to people in the field, Matheny is generally known as being among the more open and honest people in the space.
If his beliefs are what I would have expected them to be (eg something like “agrees with the basic arguments laid out in Superintelligence, and was motivated to follow his current carrer trajectory by those arguments”), then this answer is at best, misleading and misrepresentation of his actual models.
Seeing this particular example, I’m on the fence about whether to call it a “lie”. He was asked about the state of the world, not about his personal estimates, and he answered in a way that was more about the state of knowable public knowledge rather than his personal estimate. But I agree that seems pretty hair-splitting.
As it is, I notice that I’m confused.
Why wouldn’t he say something to the effect of the following?
I don’t know; this kind of forecasting is very difficult, timelines forecasting is very difficult. I can’t speak with confidence one way or the other. However, my best guess from following the literature on this topic for many years is that the catastrophic concerns are credible. I don’t know how probable it is, but does not seem to me that it is merely outlandish sci fi scenario that AI will lead to human extinction, and is not out of the question that that will happen in the next 10 years.
That doesn’t just seem more transparent, and more cooperative with the questioner, it also seems...like an obvious strategic move?
Does he not, in fact, by the basic arguments in Superingelligence? Is there some etiquette that he feels that he shouldn’t say that?
I think your interpretation is fairly uncharitable. If you have further examples of this deceptive pattern from those sympathetic to AI risk I would change my perspective but the speculation in the post plus this example weren’t compelling:
I watched the video and firstly Senator Peters seems to trail off after the quoted part and ends his question by saying “What’s your assessment of how fast this is going and when do you think we may be faced with those more challenging issues?”. So straightforwardly his question is about timelines not about risk as you frame it. Indeed Matheny (after two minutes) literally responds “it’s a really difficult question. I think whether AGI is nearer or farther than thought …” (emphasis different to yours) so makes it likely to me Matheny is expressing uncertainty about timelines, not risk.
Overall I agree that this was an opportunity for Matheny to discuss AI x-risk and plausibly it wasn’t the best use of time to discuss the uncertainty of the situation. But saying this is dishonesty doesn’t seem well supported
No, the question was about whether there are apocalyptic risks and on what timeline we should be concerned about apocalyptic risks.
The questioner used the term ‘apocalyptic’ specifically. Three people answered the question, and the first two both also alluded to ‘apocalyptic’ risks and sort of said that they didn’t really think we need to think about that possibility. Them referring to apocalyptic risks goes to show that it was a key part of what the questioner wanted to understand — to what extent these risks are real and on what timeline we’ll need to react to them. My read is not that Matheny actively misled the speaker, but that he avoided answering, which is “hiding” rather than “lying” (I don’t agree with the OP that they’re identical).
I think the question was unclear so it was more acceptable to not directly address whether there is apocalyptic risk, but I think many people I know would have definitely said “Oh to be clear I totally disagree with the previous two people, there are definitely apocalyptic risks and we are not prepared for them and cannot deal with them after-the-fact (as you just mentioned being concerned about).”
Extra detail on what happened
Everyone who answered explicitly avoided making timeline predictions and instead talked about where they think the policy focus should be.
The first person roughly said “We have many problems with AI right now, let’s focus on addressing those.”
The middle person said the AI problems are all of the sort “people being sent to jail because of an errant ML system”.
Here’s the middle person in full, clearly responding to the question of whether there’s apocalyptic risks to be worried about:
People ask me what keeps me up at night. AGI does not keep me up at night. And the reason why it doesn’t, is because (as Ms Gibbons mentioned) the problems we are likely to face, with the apocalyptic visions of AGI, are the same problems we are already facing right now, with the systems that are already in play. I worry about people being sent to jail because of an errant ML system. Whether you use some fancy AGI to do the same thing, it’s the same problem… My bet is that the harms we’re going to see, as these more powerful systems come online — even with ChatGPT — are no different from the harms we’re seeing right now. So if we focus our efforts and our energies on governance and regulation and guardrails to address the harms we’re seeing right now, they will be able to adjust as the technology improves. I am not worried that what we put in place today will be out of date or out of sync with the new tech. The new tech is like the old tech, just supercharged.
Matheny didn’t disagree with them and didn’t address the question of whether it’s apocalyptic, just said he was uncertain, and then listed the policies he wanted to see: setting standards with 3rd party audits, and governance of hardware supply chain to track it and control that it doesn’t go to places that aren’t democracies.
To not state that you disagree with the last two positions signals that you agree with them, as the absence of your disagreement is evidence of the absence of disagreement. I don’t think Matheny outright said anything false but I think it is a bit misleading to not say “I totally disagree, I think the new tech will be akin to inventing a whole new superintelligent alien species that may kill us all and take over the universe” if something like that is what you believe.
My read is that he was really trying as hard as he could to not address whether there are apocalyptic risks and instead just focus on encouraging the sorts of policies he thought should be implemented.
My read is that he was really trying as hard as he could to not address whether there are apocalyptic risks and instead just focus on encouraging the sorts of policies he thought should be implemented.
Why, though?
Does he know something we don’t? Does he think that if he expresses that those risks are real he’ll lose political capital? People won’t put him or his friends in positions of power, because he’ll be branded as a kook?
Is he just in the habit of side-stepping the weird possibilities?
This looks to me, from the outside, like an unforced error. They were asking the question, about some core beliefs, pretty directly. It seems like it would help if, in every such instance, the EA people who think that the world might be destroyed by AGI in the next 20 years, say that they think that the world might be destroyed by AGI in the next 20 years.
As Ben said, this seems incongruent with the responses that the other two people gave, neither of which talked that much about timelines, but did seem to directly respond to the concern about catastrophic/apocalyptic risk from AGI.
I do agree that it’s plausible that Matheny somehow understood the question differently from the other two people, and interpreted it in a more timelines focused way, though he also heard the other two people talk, which makes that somewhat less likely. I do agree that the question wasn’t asked in the most cogent way.
Thanks for checking this! I mostly agree with all your original comment now (except the first part suggesting it was point blank, but we’re quibbling over definitions at this point), this does seem like a case of intentionally not discussing risk
the recent TIME article saying there’s no trade off between progress and safety
More generally, for having talked to many AI policy/safety members, I can say it’s a very common pattern. At the eve of the FLI open letter, one of the most senior persons in the AI governance & policy X risk community was explaining that it was stupid to write this letter and that it would make future policy efforts much more difficult etc.
I agree that it is important to be clear about the potential for catastrophic AI risk, and I am somewhat disappointed in the answer above (though I think calling “I don’t know” lying is a bit of a stretch). But on the whole, I think people have been pretty upfront about catastrophic risk, e.g. Dario has given an explicit P(doom) publicly, all the lab heads have signed the CAIS letter, etc.
Notably, though, that’s not what the original post is primarily asking for: it’s asking for people to clearly state that they agree that we should pause/stop AI development, not to clearly state that that they think AI poses a catastrophic risk. I agree that people should clearly state that they think there’s a catastrophic risk, but I disagree that people should clearly state that they think we should pause.
Primarily, that’s because I don’t actually think trying to get governments to enact some sort of a generic pause would make good policy. Analogizing to climate change, I think getting scientists to say publicly that they think climate change is a real risk helped the cause, but putting pressure on scientists to publicly say that environmentalism/degrowth/etc. would solve the problem has substantially hurt the cause (despite the fact that a magic button that halved consumption would probably solve climate change).
As a relevant piece of evidence here, Jason Matheny, when asked point-blank in a senate committee hearing about “how concerned should we be about catastrophic risks from AI?” responded with “I don’t know”, which seems like it qualifies as a lie by the standard you set here (which, to be clear, I don’t super agree with and my intention here is partially to poke holes in your definition of a lie, while also sharing object-level relevant information).
See this video 1:39:00 to 1:43:00: https://www.hsgac.senate.gov/hearings/artificial-intelligence-risks-and-opportunities/
Quote (slightly paraphrased because transcription is hard):
Given my best model of Matheny’s beliefs, this sure does not seem like an answer that accurately summarizes his beliefs here, and represents a kind of response that I think causes people to be quite miscalibrated about the beliefs of experts in the field.
In my experience people raise the hypothetical of “but they would be honest when asked point blank” to argue that people working in the space are not being deceptive. However, I have now seen people being asked point blank, and I haven’t seen them be more honest than their original evasiveness implied, so I think this should substantially increase people’s priors on people doing something more deceptive here.
Jason Matheny is approximately the most powerful person in the AI policy space. I think he is setting a precedent here for making statements that meet at least the definition of lying you set out in your comment (I am still unsure whether to count that as lying, though it sure doesn’t feel honest), and if-anything, if I talk to people in the field, Matheny is generally known as being among the more open and honest people in the space.
If his beliefs are what I would have expected them to be (eg something like “agrees with the basic arguments laid out in Superintelligence, and was motivated to follow his current carrer trajectory by those arguments”), then this answer is at best, misleading and misrepresentation of his actual models.
Seeing this particular example, I’m on the fence about whether to call it a “lie”. He was asked about the state of the world, not about his personal estimates, and he answered in a way that was more about the state of knowable public knowledge rather than his personal estimate. But I agree that seems pretty hair-splitting.
As it is, I notice that I’m confused.
Why wouldn’t he say something to the effect of the following?
That doesn’t just seem more transparent, and more cooperative with the questioner, it also seems...like an obvious strategic move?
Does he not, in fact, by the basic arguments in Superingelligence? Is there some etiquette that he feels that he shouldn’t say that?
What’s missing from my understanding here?
I think your interpretation is fairly uncharitable. If you have further examples of this deceptive pattern from those sympathetic to AI risk I would change my perspective but the speculation in the post plus this example weren’t compelling:
I watched the video and firstly Senator Peters seems to trail off after the quoted part and ends his question by saying “What’s your assessment of how fast this is going and when do you think we may be faced with those more challenging issues?”. So straightforwardly his question is about timelines not about risk as you frame it. Indeed Matheny (after two minutes) literally responds “it’s a really difficult question. I think whether AGI is nearer or farther than thought …” (emphasis different to yours) so makes it likely to me Matheny is expressing uncertainty about timelines, not risk.
Overall I agree that this was an opportunity for Matheny to discuss AI x-risk and plausibly it wasn’t the best use of time to discuss the uncertainty of the situation. But saying this is dishonesty doesn’t seem well supported
No, the question was about whether there are apocalyptic risks and on what timeline we should be concerned about apocalyptic risks.
The questioner used the term ‘apocalyptic’ specifically. Three people answered the question, and the first two both also alluded to ‘apocalyptic’ risks and sort of said that they didn’t really think we need to think about that possibility. Them referring to apocalyptic risks goes to show that it was a key part of what the questioner wanted to understand — to what extent these risks are real and on what timeline we’ll need to react to them. My read is not that Matheny actively misled the speaker, but that he avoided answering, which is “hiding” rather than “lying” (I don’t agree with the OP that they’re identical).
I think the question was unclear so it was more acceptable to not directly address whether there is apocalyptic risk, but I think many people I know would have definitely said “Oh to be clear I totally disagree with the previous two people, there are definitely apocalyptic risks and we are not prepared for them and cannot deal with them after-the-fact (as you just mentioned being concerned about).”
Extra detail on what happened
Everyone who answered explicitly avoided making timeline predictions and instead talked about where they think the policy focus should be.
The first person roughly said “We have many problems with AI right now, let’s focus on addressing those.”
The middle person said the AI problems are all of the sort “people being sent to jail because of an errant ML system”.
Here’s the middle person in full, clearly responding to the question of whether there’s apocalyptic risks to be worried about:
Matheny didn’t disagree with them and didn’t address the question of whether it’s apocalyptic, just said he was uncertain, and then listed the policies he wanted to see: setting standards with 3rd party audits, and governance of hardware supply chain to track it and control that it doesn’t go to places that aren’t democracies.
To not state that you disagree with the last two positions signals that you agree with them, as the absence of your disagreement is evidence of the absence of disagreement. I don’t think Matheny outright said anything false but I think it is a bit misleading to not say “I totally disagree, I think the new tech will be akin to inventing a whole new superintelligent alien species that may kill us all and take over the universe” if something like that is what you believe.
My read is that he was really trying as hard as he could to not address whether there are apocalyptic risks and instead just focus on encouraging the sorts of policies he thought should be implemented.
Why, though?
Does he know something we don’t? Does he think that if he expresses that those risks are real he’ll lose political capital? People won’t put him or his friends in positions of power, because he’ll be branded as a kook?
Is he just in the habit of side-stepping the weird possibilities?
This looks to me, from the outside, like an unforced error. They were asking the question, about some core beliefs, pretty directly. It seems like it would help if, in every such instance, the EA people who think that the world might be destroyed by AGI in the next 20 years, say that they think that the world might be destroyed by AGI in the next 20 years.
As Ben said, this seems incongruent with the responses that the other two people gave, neither of which talked that much about timelines, but did seem to directly respond to the concern about catastrophic/apocalyptic risk from AGI.
I do agree that it’s plausible that Matheny somehow understood the question differently from the other two people, and interpreted it in a more timelines focused way, though he also heard the other two people talk, which makes that somewhat less likely. I do agree that the question wasn’t asked in the most cogent way.
Thanks for checking this! I mostly agree with all your original comment now (except the first part suggesting it was point blank, but we’re quibbling over definitions at this point), this does seem like a case of intentionally not discussing risk
A few other examples off the top of my head:
ARC graph on RSPs with the “safe zone” part
Anthropic calling ASL-4 accidental risks “speculative”
the recent TIME article saying there’s no trade off between progress and safety
More generally, for having talked to many AI policy/safety members, I can say it’s a very common pattern. At the eve of the FLI open letter, one of the most senior persons in the AI governance & policy X risk community was explaining that it was stupid to write this letter and that it would make future policy efforts much more difficult etc.
I agree that it is important to be clear about the potential for catastrophic AI risk, and I am somewhat disappointed in the answer above (though I think calling “I don’t know” lying is a bit of a stretch). But on the whole, I think people have been pretty upfront about catastrophic risk, e.g. Dario has given an explicit P(doom) publicly, all the lab heads have signed the CAIS letter, etc.
Notably, though, that’s not what the original post is primarily asking for: it’s asking for people to clearly state that they agree that we should pause/stop AI development, not to clearly state that that they think AI poses a catastrophic risk. I agree that people should clearly state that they think there’s a catastrophic risk, but I disagree that people should clearly state that they think we should pause.
Primarily, that’s because I don’t actually think trying to get governments to enact some sort of a generic pause would make good policy. Analogizing to climate change, I think getting scientists to say publicly that they think climate change is a real risk helped the cause, but putting pressure on scientists to publicly say that environmentalism/degrowth/etc. would solve the problem has substantially hurt the cause (despite the fact that a magic button that halved consumption would probably solve climate change).