For crying out loud, LLMs are already considered “AIs” by most people! How could they be a worse analogy for AI, across the board, than extraterrestrial beings that we have never come in contract with?
By tending to lead to overconfidence. An aliens analogy is explicitly relying on [we have no idea what this will do]. It’s easy to imagine friendly aliens, just as it’s easy to imagine unfriendly ones, or entirely disinterested ones. The analogy is unlikely to lead to a highly specific, incorrect model.
This is not true for LLMs. It’s easy to assume that particular patterns will continue to hold—e.g. that it’ll be reasonably safe to train systems with something like our current degree of understanding.
To be clear, I’m not saying they’re worse in terms of information content: I’m saying they can be worse in the terms you’re using to object to analogies: “routinely conveying the false impression of a specific, credible model of AI”.
I think it’s correct that we should be very wary of the use of analogies (though they’re likely unavoidable). However, the cases where we need to be the most wary are those that seem most naturally applicable—these are the cases that are most likely to lead to overconfidence. LLMs, [current NNs], or [current AI systems generally] are central examples here.
On asymmetric pushback, I think you’re correct, but that you’ll tend to get an asymmetry everywhere between [bad argument for conclusion most people agree with] and [bad argument for conclusion most people disagree with]. People have limited time. They’ll tend to put a higher value on critiquing invalid-in-their-opinion arguments when those lead to incorrect-in-their-opinion conclusions (at least unless they’re deeply involved in the discussion).
There’s also an asymmetry in terms of consequences-of-mistakes here: if we think that AI will be catastrophic, and are wrong, this causes a delay, a large loss of value, and a small-but-significant increase in x-risk; if we think that AI will be non-catastrophic, and are wrong, we’re dead.
Lack of pushback shouldn’t be taken as a strong indication that people agree with the argumentation used.
Clearly this isn’t ideal. I do think it’s worth thinking about mechanisms to increase the quality of argument. E.g. I think the ability to emoji react to particular comment sections is helpful here—though I don’t think there’s one that’s great for [analogy seems misleading] as things stand. Perhaps there should be a [seems misleading] react?? (I don’t think “locally invalid” covers this)
An aliens analogy is explicitly relying on [we have no idea what this will do]. It’s easy to imagine friendly aliens, just as it’s easy to imagine unfriendly ones, or entirely disinterested ones. The analogy is unlikely to lead to a highly specific, incorrect model.
As a matter of fact I think the word “alien” often evokes a fairly specific caricature that is separate from “something that’s generically different and hard to predict”. But it’s obviously hard for me to prove what’s going on in people’s minds, so I’ll just say what tends to flashes in my mind when I think of aliens:
Beings who have no shared history with us, as they were created under a completely separate evolutionary history
A Hollywood stock image of an alien species that is bent on some goal, such as extermination (i.e. rapacious creatures who will stop at nothing to achieve something)
A being that does not share our social and cultural concepts
I think these things are often actually kind of being jammed into the analogy, intended or not, and the question of how much future AIs will share these properties is still an open question. I think we should not merely assume these things.
By tending to lead to overconfidence.
An aliens analogy is explicitly relying on [we have no idea what this will do]. It’s easy to imagine friendly aliens, just as it’s easy to imagine unfriendly ones, or entirely disinterested ones. The analogy is unlikely to lead to a highly specific, incorrect model.
This is not true for LLMs. It’s easy to assume that particular patterns will continue to hold—e.g. that it’ll be reasonably safe to train systems with something like our current degree of understanding.
To be clear, I’m not saying they’re worse in terms of information content: I’m saying they can be worse in the terms you’re using to object to analogies: “routinely conveying the false impression of a specific, credible model of AI”.
I think it’s correct that we should be very wary of the use of analogies (though they’re likely unavoidable).
However, the cases where we need to be the most wary are those that seem most naturally applicable—these are the cases that are most likely to lead to overconfidence. LLMs, [current NNs], or [current AI systems generally] are central examples here.
On asymmetric pushback, I think you’re correct, but that you’ll tend to get an asymmetry everywhere between [bad argument for conclusion most people agree with] and [bad argument for conclusion most people disagree with].
People have limited time. They’ll tend to put a higher value on critiquing invalid-in-their-opinion arguments when those lead to incorrect-in-their-opinion conclusions (at least unless they’re deeply involved in the discussion).
There’s also an asymmetry in terms of consequences-of-mistakes here: if we think that AI will be catastrophic, and are wrong, this causes a delay, a large loss of value, and a small-but-significant increase in x-risk; if we think that AI will be non-catastrophic, and are wrong, we’re dead.
Lack of pushback shouldn’t be taken as a strong indication that people agree with the argumentation used.
Clearly this isn’t ideal.
I do think it’s worth thinking about mechanisms to increase the quality of argument.
E.g. I think the ability to emoji react to particular comment sections is helpful here—though I don’t think there’s one that’s great for [analogy seems misleading] as things stand. Perhaps there should be a [seems misleading] react?? (I don’t think “locally invalid” covers this)
As a matter of fact I think the word “alien” often evokes a fairly specific caricature that is separate from “something that’s generically different and hard to predict”. But it’s obviously hard for me to prove what’s going on in people’s minds, so I’ll just say what tends to flashes in my mind when I think of aliens:
Beings who have no shared history with us, as they were created under a completely separate evolutionary history
A Hollywood stock image of an alien species that is bent on some goal, such as extermination (i.e. rapacious creatures who will stop at nothing to achieve something)
A being that does not share our social and cultural concepts
I think these things are often actually kind of being jammed into the analogy, intended or not, and the question of how much future AIs will share these properties is still an open question. I think we should not merely assume these things.