As an example, he analyzes a passage from Andrew Yang’s presidential campaign, which claimed, “The smartest people in the world now predict that ⅓ of Americans will lose their job to automation in 12 years.” Pinker labels such reasoning a “mild example of the argument from authority”.
Isn’t this pretty damning? (As in, damning for Steven’s abilities as a rationalist.)
Appeals to authority are not at all categorically weak. A lot of Inadequate Equilibria was about figuring out when it is and is not plausible to out-compete authority. Saying “Professionals say it’s hard to predict Microsoft’s stock price, so you won’t be able to” is also an argument from authority; it’s also an extremely strong argument.
There has to be a name for the fallacy of thinking that a common term with negative association like “argument from authority” automatically means something even when it doesn’t… ? Someone help me out here.
Also—choosing Yang as an example of irrationality in politics? Really? I guess I’m supposed to think that any example is okay as long as it’s legit (this one arguably isn’t, anyway), but in fact, being named as an example is Bayesian evidence that the author thinks you are altogether irrational, and people are probably going to understand it that way.
The smartest people in the world find it hard to predict Microsoft’s stock price, so you won’t be able to.
Wouldn’t the argument-from-authority version of this instead be “The smartest people in the world say it’s hard for anyone to predict Microsoft’s stock price, so you won’t be able to”?
“Smart people can’t do X, therefore average people can’t do X either” seems less fallacious than “Smart people say average people can’t do X, and they must be right because they’re smart.”
Actually, I think the example I wanted to choose was ‘professionals say it’s hard to predict the stock price’. Like, the appeal to authority is most commonly made with respect to supposed experts in the field, not with generally smart people. What Yang said isn’t even a central example of appeal to authority.
I think ‘smart people say average people can’t do X, they must be right because they’re smart’ is probably also not central? Also not sure it’s all that fallacious, probably depends on how you define ‘smart’.
We could substitute any nonsense assertion we like into that Yang quote.
The smartest people in the world now predict that horse-doses of ivermectin are the best treatment for COVID-19.
The smartest people in the world now predict that Kanye West will win the next presidential election.
The smartest people in the world now predict that we should return to the gold standard.
Appending “the smartest people in the world now predict [X}” shouldn’t actually lend any weight to the subsequent claim. It’s just a thing you can say to make that claim sound more substantial. This is because Yang’s not telling us who these people are, exactly, how he knows they’re so smart, or why their intelligence bears very much on the credence we should assign to this remarkably specific prediction.
If instead, Yang had said “Paul Krugman now predicts that 1⁄3 of Americans will lose their job to automation in 12 years” (and note—I used Krugman’s name arbitrarily and have no reason to believe this is what he really thinks), then we could judge for ourselves the credence we assign to Krugman’s predictive abilities. It would become a meaningful claim. As it is, though, I think this Yang quote is a nice, central example of an argument from authority.
We could substitute any nonsense assertion we like into that Yang quote.
Isn’t that trivially false? If Yang said the Kanye thing, that would be a lie, so if you think he’s trying to be honest, he can’t say that. I agree that he doesn’t give you any way to verify his claim, but that’s not the standard I use to decide whether something is an appeal to authority. If you say, ‘some of the smartest people in the world are religious’, that’s an appeal to authority and probably a weak argument even though it’s true.
Yang often uses the phrase ‘my friends in Silicon Valley’; he probably was talking about important people in tech in that quote.. I wouldn’t trust those people, but I certainly think their opinions are evidence.
OK, I think there are two ways to look at this question.
One is to separate the quote from the speaker, and ask if we’d still consider the context-free quote to be a piece of evidence. This is what I am advocating.
The other is to consider who’s speaking as key to interpreting the meaning of the quote, which is what you’re doing.
I think both are valid. For example, in the TV show “Firefly,” one of the characters, Simon Tam, receives letters from his highly intelligent sister. They read as “perfectly normal” to his parents, but their trivial content and occasional misspellings make him suspect—correctly—that they contain a code saying that she’s being harmed at the boarding school she’s been sent to. Here, considering the message in light of the speaker (or sender, in this case), is crucial to understanding it as a piece of evidence that his sister is in danger.
Another example is when my mentor in my MS program tells me that it’s best to automatically accept the predictions our PI makes about research ideas. If he likes them, they’re good. If he doesn’t like them, they’re bad. He’s a credible authority figure to whom we can and should appeal as a strong form of evidence.
Alternatively, there are many cases in which we might find it very difficult to predict how the identity of the speaker, or the context, should influence our interpretation of their quote. In the case of Yang, I have very little insight into whether or not his reference to “the smartest people in the world” is evidence that “this job-loss prediction is believed by more smart and well-qualified analysists than I, AllAmericanBreakfast, had thought prior to reading this Yang quote.”
If it is evidence of this, then yes, I agree that an ideal reasoning process would take it as some evidence that the prediction is true. But realistically, politicians often play fast-and-loose with their evidence. I am often wise to actively choose to not read anything into the quote beyond its context-free content. When I read this quote, I willfully shut off my imagination from trying to conjure up images of the supposed “smartest people in the world” that Yang’s ventriloquizing, to prevent my brain from being tricked into thinking this quote ought to update my belief.
Perhaps a more precise description of the fallacy here is “argument from an illusory authority.” When we say that something is a fallacious argument from authority, we’re implicitly saying that “proper epistemics in this context is to disregard references to the opinions of nonspecific ‘authorities,’ because the rhetoric is designed to trick you into accepting the statement, rather than to convey credible opinions to your mind.”
In response to your question, “isn’t this trivially false,” well—no, it’s trivially true. We can substitute anything we like into that statement. Watch me!
The smartest people in the world say that raisin bran is the best cereal.
The smartest people in the world say that I, AllAmericanBreakfast, am right about all this argument-from-authority stuff.
The smartest people in the world say that it’s trivially true that you can append any nonsense statement to “the smartest people in the world.”
Context-free, you shouldn’t (and, I’m sure, are not) taking “the smartest people in the world” as any evidence at all of the truth-value of my claims. In context, you also aren’t doing that, because you recognize my intent in uttering these quotes is not to convince you of their literal truth.
Note that it’s only in very special circumstances—particular combinations of speaker, referenced authority figure, and object-level claim—in which you would consider the referenced authority figure to lend a greater weight of evidence to the claim. Perhaps most references to authority figures in the context of argument are selected for actually being relevant as evidence. By this, I mean that during actual debate, people may strive to avoid statements like “my uncle Bob says that Apple stock is going to double in price by next year,” because referencing uncle Bob as an authority figure lends no support to their claim, but undermines their general credibility as a debater. So when an authority figure does get referenced, we ought to take it seriously.
But I don’t actually buy that argument. I think vague or non-credible authority figures get referenced all the time, and it’s only in select circumstances when we should actually respect this form of evidence. Generally, I think we should “tag” references to authority figures as fallacious and to be ignored, unless we have made a considered judgment to afford a specific speaker, on a specific topic, referencing a specific authority figure, to be actually useful evidence. A conservative guardedness with occasional permissions, rather than a liberal acceptance with occasional rejections.
In an ideal reasoning process with unlimited compute, we might wish to consider fully the credibility of each referenced authority figure, no matter how vague. But in a real process with our puny minds, I think it’s best to generally choose to ignore and actively (epistemically) punish arguments by authority, unless they’re done right.
I strongly endorse drawing a distinction, but I think I want to draw it a bit differently. The reason is that I feel like I would still defend the smart-people quote as non-fallacy-like if someone else had said it, and if that’s true, it can’t be because I have some respect for Yang as a thinker.
How about this (which I give you total credit for because I only came up with it after reading your comment):
(1) A statement is, by itself, not evidence to a rational observer who doesn’t know the speaker, but it’s possible that adding more information could turn it into evidence
(2) A statement contains enough information for a rational observer to conclude that the argument is fallacious
I would agree that #1 applies to Yang’s quote, and if that’s sufficient for being a fallacy, then Yang’s quote is a fallacy. This has some logic to it because people could mistakenly believe that the quote is evidence by itself, and that would be a mistake. However, I myself often say things for which #1 applies, and I believe that a lot of pretty rational people do as well. Then again, perhaps some don’t. I probably do it much less on LW than on other sites where I put much less effort into my posts.
I think the explanation for my intuition that the inclusion into the book is stupid is that avoiding #1 is a relatively high standard, and in fact lots of politicians routinely fail #2. I bet you could even find clear-cut examples of politicians failing #2 with regard to arguments from authority. There just is a difference between “this is stupid” and “this is incomplete and may or may not be stupid if I hear the rest”, and that difference seems to capture my reaction even on reflection.
Conversely, I feel like the personality of the speaker should not be an input to the fallacy-deciding function. I agree with everything in your last four paragraphs, but I think #1 remains less bad than #2 even if you think it’s unlikely that additional information would make the argument non-stupid.
I think we’re getting closer! Here’s even another alternative.
Let’s first admit that arguments can gain information-content both from their text and from their context. Through text and context, they can define, describe, and make claims and predictions.
An appropriate argument from authority sufficiently defines (via text and context) a meaningful authority figure, describes with sufficient accuracy their level of credibility on the subject, and makes a sufficiently specific claim. We predict that the accuracy of our own prediction will improve if we put in the effort to update our predictions based on this claim. Furthermore, once the prediction resolves, we will use that result to increase or decease the credibility we ascribe to that authority figure.
A fallacious argument from authority fails one or more of these tests of sufficiency, even when taking context into account. The authority figure may be too vaguely referenced, their credibility may be exaggerated, or the claim may be too imprecise. The accuracy of our predictions will be worsened if we update based on this claim, and the resolution of the claim does not allow us to update the credibility we ascribe to the referenced authority figure.
Another possible division is:
A non-fallacious argument from authority creates an expectation that more context about the referenced authority figure would help us better assess the truth-content of the claim.
A fallacious argument from authority creates an expectation that further context would make the quote feel just as stupid as it seemed before.
I think it’s only worth worrying about these divisions and fearing arguments from authority in a relatively serious context. If you’re having dinner with your friends and happen to vaguely reference an authority figure to back your claim, that’s fine. It ain’t that deep.
But if a serious statesman does it on TV in the context of a debate or speech, then we have every right to complain that their claims contain fallacious arguments from authority.
I think that Yang’s quote has a serious-enough context, is stupid-sounding enough on its own to make me uninterested in more context, and fails all these tests of sufficiency. For that reason, I consider it fallacious. It hits all three check boxes on my “fallacy check-list.”
I suspect that when you make potentially-fallacious arguments from authority, they’re usually in a relatively non-serious context and that your audience believes that if they took the time to interrogate you about the authority figures you reference, that they’d feel persuaded that they are plausibly meritorious authority figures even though you were vague in your initial presentation. Hence, you would probably not be committing a fallacy, in the way I am defining it here.
[...] creates an expectation that more context about the referenced authority figure would help us better assess the truth-content of the claim.
[...] creates an expectation that further context would make the quote feel just as stupid as it seemed before.
I want to add 3
[...] contains enough information that you already know the quote is stupid
You implied Yang’s quote is #2, I would say it’s #1. But (more importantly?) I would draw the bigger distinction between #2 and #3, and I don’t see any reason to choose something in #2 as an example of irrationality when examples in #3 are available. This, I think, is still my main argument.
I agree with everything else in your post; in particular, with the importance of context which I hadn’t considered. I concede that my behavior isn’t analogous to that of Yang in this example.
Glad we are converging on some sort of agreement! I find this helpful to talk out with you.
I’m not clear on the distinction between #2 and #3. What’s the difference between predicting the quote will still seem stupid after further research, and finding the quote to be stupid now? By conservation of expected evidence, aren’t they the same thing?
I’m not clear on the distinction between #2 and #3. What’s the difference between predicting the quote will still seem stupid after further research, and finding the quote to be stupid now? By conservation of expected evidence, aren’t they the same thing?
One difference is just uncertainty. Conservation of expected evidence only works if you’re certain. I think I was assuming that your probability on Yang being unable to expand on his claim meaningfully is like ~67%, not like 98%.
What I suspect happened is that he talked to various big names in tech, including the CEOs of companies who make decisions about automation, and they were bullish on the timelines. Would that kind of scenario qualify as being non-stupid?
The other difference is that you can’t demonstrate it. The leading question wasn’t about whether one should update from the quote, it was whether it’s a good idea to choose the quote as a negative example in a book about rationality. Even if Steven Pinker were 100% sure that there is no reason to update here, it’s still not a good example if he can’t prove it. I mean, if you accept that the quote could plausibly be non-stupid, then the honest way to talk about would have been “Here is Yang saying {quote}. If it turns out that ‘some of the smartest people in the world’ is made-up or refers to people aren’t actually impressive, this will have been an example of a vapid appeal-to-authority. I can’t prove that this is so, but here are my reasons for suspecting it.” And again, I can’t imagine it’s hard to find examples of appeals-to-authority that are clear-cut fallacies.
What I suspect happened is that he talked to various big names in tech, including the CEOs of companies who make decisions about automation, and they were bullish on the timelines. Would that kind of scenario qualify as being non-stupid?
No. That’s a good point. It seems like “fallacious argument from authority” lends itself to black-and-white thinking that’s just not appropriate in many cases. Reading the tea leaves has its value. If I had to guess, Pinker was looking for a timely quote by a politician his readers are likely to be sympathetic to, and this one was convenient.
I still think that there are many times when it’s best as a rule to just dismiss statements with the form of “arguments from authority.” This fits the criteria, and it might be that sometimes you throw out the baby with the bathwater this way. Then again, there could be equal value in becoming sensitive to when it’s appropriate to “tune in” to this sort of evidence. That probably depends on the individual and their goals.
It would be interesting to know exactly what Pinker wrote. For instance, imagine that he wrote something like this:
What Yang does here is a mild example of the argument from authority. It may be true that 1⁄3 of Americans will lose their jobs to automation in 12 years, and if it’s true that the smartest people in the world expect that then that should make us think it more likely than we did before. But it’s a long way short of proof—the smartest people in the world may still not have much actual ability to predict what will happen 12 years out. And Yang never says which smartest people in the world, or how he knows that they’re the smartest, or what sort of smart, all of which could make a big difference to how much weight we give to their opinion. (Maybe the smartest people in the world are all theoretical physicists and don’t actually know anything about the economy. Maybe the people Yang thinks are the smartest in the world are the people who seem smart to him because they say a lot of things he agrees with. Maybe he has no idea who the smartest people in the world are and he’s just bullshitting. Or maybe he means that a selection of top economists, technologists and the like sat down and made a serious attempt to predict likely futures for technological development and their impact on the economy, and they estimate that 1⁄3 of Americans will lose their jobs to automation. These scenarios are not all alike.) Some arguments from authority are stronger than this, some are weaker, but the weaknesses here are typical: it’s not clear exactly what authority is being cited, it’s not clear exactly how expert that authority is, and even the most expert it could plausibly be isn’t enough to justify very much confidence in what they say.
I wouldn’t find that damning evidence against Pinker’s expertise in rationality.
FWIW, this was the full paragraph that I pulled the quote from,
The “smartest people in the world” claim from the Yang Gang is a mild example of the argument from authority. The authority being deferred to is often religious, as in the gospel song and bumper sticker “God said it, I believe it, that settles it.” But it can also be political or academic. Intellectual cliques often revolve around a guru whose pronouncements become secular gospel. Many academic disquisitions begin, “As Derrida has taught us . . .”—or Foucault, or Butler, or Marx, or Freud, or Chomsky. Good scientists disavow this way of talking, but they are sometimes raised up as authorities by others. I often get letters taking me to task for worrying about human-caused climate change because, they note, this brilliant physicist or that Nobel laureate denies it. But Einstein was not the only scientific authority whose opinions outside his area of expertise were less than authoritative. In their article “The Nobel Disease: When Intelligence Fails to Protect against Irrationality,” Scott Lilienfeld and his colleagues list the flaky beliefs of a dozen science laureates, including eugenics, megavitamins, telepathy, homeopathy, astrology, herbalism, synchronicity, race pseudoscience, cold fusion, crank autism treatments, and denying that AIDS is caused by HIV.
I maintain that the point is stupid. Putting aside that ‘smart’ could entail rationality, the bigger issue is that Steven argues [it’s non-conclusive] ⇒ [it’s a fallacy]. Certainly intelligent people can believe false things, but as long as intelligence correlates with being accurate, what Yang is is still Bayesian evidence, just as you said in the review.
And it’s not hard to find better examples for irrationality. Julia Galef’s book contains a bunch of those, and they’re all clear-cut.
If all the information you had about a person was some.very generic information about their IQ or rationality quotient,your best option would be to believe the person who has the highest.
But that is almost never the case. Experts have certificates indicating their domaon specific knowledge.
Would you want a random person with an IQ of 180 performing a surgical operation on you?
what Yang is is still Bayesian evidence
But very weak Bayesian evidence. The human brain can’t physically deal with every small.or very large quantities. You’re much better off disregarding very weak evidence.
You’re much better off disregarding very weak evidence.
Yes. This shouldn’t be confused with regarding it as lack of evidence. Occasionally you are better off making use of it after all, if the margins for a decision are slim. Evidence is never itself fallacious, the error is its misrepresentation as something that it isn’t, and an act of labeling something as a fallacy can itself easily fall into such a fallacy, for example implying that weak evidence of certain form should be seen as lack of evidence or as counterevidence.
Would you want a random person with an IQ of 180 performing a surgical operation on you?
If we didn’t have professional surgeons (and I need the surgery), then yes, and we don’t have something analogous to professional surgeons for predicting the future. (Maybe superforecasters, but that standard is definitely not relevant if we’re comparing Yang to the average politician.)
We do have people with expertise relevant to making the sort of prediction Yang’s talking about, though. For instance:
AI researchers probably have a better idea than randomly chosen very smart people of what the state of AI is likely to be a decade from now.
Economists probably have a better idea than randomly chosen very smart people of whether the likely outcome of a given level of AI progress looks more like “oh noes 1⁄3 of all Americans have lost their jobs” or “1/3 of all Americans find that the nature of their work has changed” or “no one actually needs a job any more”.
Damn! If only I’d listened to AI researchers five years ago.
(I know what you meant :-).)
Yes, it’s true that AI researchers’ greater expertise is to some extent counterbalanced by possible biases.I still think it’s likely that a typical eminent AI researcher has a better idea of the likely level of technological obsolescence over the next ~decade than a typical randomly chosen person with (say) an IQ over 160.
(I don’t think corresponding things are always true. For instance, I am not at all convinced that a randomly chosen eminent philosopher-of-religion has a better idea on average of whether there are any gods and what they’re like if so than a randomly chosen very clever person. I think it depends on how much real expertise is possible in a given field. In AI there’s quite a lot.)
I agree that people who are both AI experts and truck drivers (or executives at truck-driving companies) will have a better idea of how many Americans will lose their truck-driving jobs because they get automated away, and likewise for other jobs.
Relatively few people are expert both in AI and in other fields at risk of getting automated away. I think having just expertise in AI gives you a better chance than having neither. I don’t know who Yang’s “smartest people” actually were, but if they aren’t people with either specific AI expertise, or specific expertise in areas likely to fall victim to automation, or maybe expertise in labor economics, then I think their pronouncements about how many Americans are going to lose their jobs to automation in the near future are themselves examples of the phenomenon that xkcd comic is pointing at.
(Also, e.g., truck drivers may well have characteristic biases when they talk about the likely state of the truck driving industry a decade from now, just as AI researchers may.)
There has to be a name for the fallacy of thinking that a common term with negative association like “argument from authority” automatically means something even when it doesn’t… ? Someone help me out here.
The fallacy is inappropriate appeal to authority. A geologist is not an apporiate authority on what is truly a fruit or berry. And you should not disregard your doctor’s advice just because they’re an authority on medicine.
The word “smart” is nowadays used to mean something like “expert on everything”...but how likely is that? It takes half a lifetime to get serious domain-specific knowledge of just one thing.
Isn’t this pretty damning? (As in, damning for Steven’s abilities as a rationalist.)
Appeals to authority are not at all categorically weak. A lot of Inadequate Equilibria was about figuring out when it is and is not plausible to out-compete authority. Saying “Professionals say it’s hard to predict Microsoft’s stock price, so you won’t be able to” is also an argument from authority; it’s also an extremely strong argument.
There has to be a name for the fallacy of thinking that a common term with negative association like “argument from authority” automatically means something even when it doesn’t… ? Someone help me out here.
Also—choosing Yang as an example of irrationality in politics? Really? I guess I’m supposed to think that any example is okay as long as it’s legit (this one arguably isn’t, anyway), but in fact, being named as an example is Bayesian evidence that the author thinks you are altogether irrational, and people are probably going to understand it that way.
Wouldn’t the argument-from-authority version of this instead be “The smartest people in the world say it’s hard for anyone to predict Microsoft’s stock price, so you won’t be able to”?
“Smart people can’t do X, therefore average people can’t do X either” seems less fallacious than “Smart people say average people can’t do X, and they must be right because they’re smart.”
Actually, I think the example I wanted to choose was ‘professionals say it’s hard to predict the stock price’. Like, the appeal to authority is most commonly made with respect to supposed experts in the field, not with generally smart people. What Yang said isn’t even a central example of appeal to authority.
I think ‘smart people say average people can’t do X, they must be right because they’re smart’ is probably also not central? Also not sure it’s all that fallacious, probably depends on how you define ‘smart’.
We could substitute any nonsense assertion we like into that Yang quote.
Appending “the smartest people in the world now predict [X}” shouldn’t actually lend any weight to the subsequent claim. It’s just a thing you can say to make that claim sound more substantial. This is because Yang’s not telling us who these people are, exactly, how he knows they’re so smart, or why their intelligence bears very much on the credence we should assign to this remarkably specific prediction.
If instead, Yang had said “Paul Krugman now predicts that 1⁄3 of Americans will lose their job to automation in 12 years” (and note—I used Krugman’s name arbitrarily and have no reason to believe this is what he really thinks), then we could judge for ourselves the credence we assign to Krugman’s predictive abilities. It would become a meaningful claim. As it is, though, I think this Yang quote is a nice, central example of an argument from authority.
Isn’t that trivially false? If Yang said the Kanye thing, that would be a lie, so if you think he’s trying to be honest, he can’t say that. I agree that he doesn’t give you any way to verify his claim, but that’s not the standard I use to decide whether something is an appeal to authority. If you say, ‘some of the smartest people in the world are religious’, that’s an appeal to authority and probably a weak argument even though it’s true.
Yang often uses the phrase ‘my friends in Silicon Valley’; he probably was talking about important people in tech in that quote.. I wouldn’t trust those people, but I certainly think their opinions are evidence.
OK, I think there are two ways to look at this question.
One is to separate the quote from the speaker, and ask if we’d still consider the context-free quote to be a piece of evidence. This is what I am advocating.
The other is to consider who’s speaking as key to interpreting the meaning of the quote, which is what you’re doing.
I think both are valid. For example, in the TV show “Firefly,” one of the characters, Simon Tam, receives letters from his highly intelligent sister. They read as “perfectly normal” to his parents, but their trivial content and occasional misspellings make him suspect—correctly—that they contain a code saying that she’s being harmed at the boarding school she’s been sent to. Here, considering the message in light of the speaker (or sender, in this case), is crucial to understanding it as a piece of evidence that his sister is in danger.
Another example is when my mentor in my MS program tells me that it’s best to automatically accept the predictions our PI makes about research ideas. If he likes them, they’re good. If he doesn’t like them, they’re bad. He’s a credible authority figure to whom we can and should appeal as a strong form of evidence.
Alternatively, there are many cases in which we might find it very difficult to predict how the identity of the speaker, or the context, should influence our interpretation of their quote. In the case of Yang, I have very little insight into whether or not his reference to “the smartest people in the world” is evidence that “this job-loss prediction is believed by more smart and well-qualified analysists than I, AllAmericanBreakfast, had thought prior to reading this Yang quote.”
If it is evidence of this, then yes, I agree that an ideal reasoning process would take it as some evidence that the prediction is true. But realistically, politicians often play fast-and-loose with their evidence. I am often wise to actively choose to not read anything into the quote beyond its context-free content. When I read this quote, I willfully shut off my imagination from trying to conjure up images of the supposed “smartest people in the world” that Yang’s ventriloquizing, to prevent my brain from being tricked into thinking this quote ought to update my belief.
Perhaps a more precise description of the fallacy here is “argument from an illusory authority.” When we say that something is a fallacious argument from authority, we’re implicitly saying that “proper epistemics in this context is to disregard references to the opinions of nonspecific ‘authorities,’ because the rhetoric is designed to trick you into accepting the statement, rather than to convey credible opinions to your mind.”
In response to your question, “isn’t this trivially false,” well—no, it’s trivially true. We can substitute anything we like into that statement. Watch me!
Context-free, you shouldn’t (and, I’m sure, are not) taking “the smartest people in the world” as any evidence at all of the truth-value of my claims. In context, you also aren’t doing that, because you recognize my intent in uttering these quotes is not to convince you of their literal truth.
Note that it’s only in very special circumstances—particular combinations of speaker, referenced authority figure, and object-level claim—in which you would consider the referenced authority figure to lend a greater weight of evidence to the claim. Perhaps most references to authority figures in the context of argument are selected for actually being relevant as evidence. By this, I mean that during actual debate, people may strive to avoid statements like “my uncle Bob says that Apple stock is going to double in price by next year,” because referencing uncle Bob as an authority figure lends no support to their claim, but undermines their general credibility as a debater. So when an authority figure does get referenced, we ought to take it seriously.
But I don’t actually buy that argument. I think vague or non-credible authority figures get referenced all the time, and it’s only in select circumstances when we should actually respect this form of evidence. Generally, I think we should “tag” references to authority figures as fallacious and to be ignored, unless we have made a considered judgment to afford a specific speaker, on a specific topic, referencing a specific authority figure, to be actually useful evidence. A conservative guardedness with occasional permissions, rather than a liberal acceptance with occasional rejections.
In an ideal reasoning process with unlimited compute, we might wish to consider fully the credibility of each referenced authority figure, no matter how vague. But in a real process with our puny minds, I think it’s best to generally choose to ignore and actively (epistemically) punish arguments by authority, unless they’re done right.
I strongly endorse drawing a distinction, but I think I want to draw it a bit differently. The reason is that I feel like I would still defend the smart-people quote as non-fallacy-like if someone else had said it, and if that’s true, it can’t be because I have some respect for Yang as a thinker.
How about this (which I give you total credit for because I only came up with it after reading your comment):
(1) A statement is, by itself, not evidence to a rational observer who doesn’t know the speaker, but it’s possible that adding more information could turn it into evidence
(2) A statement contains enough information for a rational observer to conclude that the argument is fallacious
I would agree that #1 applies to Yang’s quote, and if that’s sufficient for being a fallacy, then Yang’s quote is a fallacy. This has some logic to it because people could mistakenly believe that the quote is evidence by itself, and that would be a mistake. However, I myself often say things for which #1 applies, and I believe that a lot of pretty rational people do as well. Then again, perhaps some don’t. I probably do it much less on LW than on other sites where I put much less effort into my posts.
I think the explanation for my intuition that the inclusion into the book is stupid is that avoiding #1 is a relatively high standard, and in fact lots of politicians routinely fail #2. I bet you could even find clear-cut examples of politicians failing #2 with regard to arguments from authority. There just is a difference between “this is stupid” and “this is incomplete and may or may not be stupid if I hear the rest”, and that difference seems to capture my reaction even on reflection.
Conversely, I feel like the personality of the speaker should not be an input to the fallacy-deciding function. I agree with everything in your last four paragraphs, but I think #1 remains less bad than #2 even if you think it’s unlikely that additional information would make the argument non-stupid.
I think we’re getting closer! Here’s even another alternative.
Let’s first admit that arguments can gain information-content both from their text and from their context. Through text and context, they can define, describe, and make claims and predictions.
An appropriate argument from authority sufficiently defines (via text and context) a meaningful authority figure, describes with sufficient accuracy their level of credibility on the subject, and makes a sufficiently specific claim. We predict that the accuracy of our own prediction will improve if we put in the effort to update our predictions based on this claim. Furthermore, once the prediction resolves, we will use that result to increase or decease the credibility we ascribe to that authority figure.
A fallacious argument from authority fails one or more of these tests of sufficiency, even when taking context into account. The authority figure may be too vaguely referenced, their credibility may be exaggerated, or the claim may be too imprecise. The accuracy of our predictions will be worsened if we update based on this claim, and the resolution of the claim does not allow us to update the credibility we ascribe to the referenced authority figure.
Another possible division is:
A non-fallacious argument from authority creates an expectation that more context about the referenced authority figure would help us better assess the truth-content of the claim.
A fallacious argument from authority creates an expectation that further context would make the quote feel just as stupid as it seemed before.
I think it’s only worth worrying about these divisions and fearing arguments from authority in a relatively serious context. If you’re having dinner with your friends and happen to vaguely reference an authority figure to back your claim, that’s fine. It ain’t that deep.
But if a serious statesman does it on TV in the context of a debate or speech, then we have every right to complain that their claims contain fallacious arguments from authority.
I think that Yang’s quote has a serious-enough context, is stupid-sounding enough on its own to make me uninterested in more context, and fails all these tests of sufficiency. For that reason, I consider it fallacious. It hits all three check boxes on my “fallacy check-list.”
I suspect that when you make potentially-fallacious arguments from authority, they’re usually in a relatively non-serious context and that your audience believes that if they took the time to interrogate you about the authority figures you reference, that they’d feel persuaded that they are plausibly meritorious authority figures even though you were vague in your initial presentation. Hence, you would probably not be committing a fallacy, in the way I am defining it here.
Taking your second division:
I want to add 3
You implied Yang’s quote is #2, I would say it’s #1. But (more importantly?) I would draw the bigger distinction between #2 and #3, and I don’t see any reason to choose something in #2 as an example of irrationality when examples in #3 are available. This, I think, is still my main argument.
I agree with everything else in your post; in particular, with the importance of context which I hadn’t considered. I concede that my behavior isn’t analogous to that of Yang in this example.
Glad we are converging on some sort of agreement! I find this helpful to talk out with you.
I’m not clear on the distinction between #2 and #3. What’s the difference between predicting the quote will still seem stupid after further research, and finding the quote to be stupid now? By conservation of expected evidence, aren’t they the same thing?
One difference is just uncertainty. Conservation of expected evidence only works if you’re certain. I think I was assuming that your probability on Yang being unable to expand on his claim meaningfully is like ~67%, not like 98%.
What I suspect happened is that he talked to various big names in tech, including the CEOs of companies who make decisions about automation, and they were bullish on the timelines. Would that kind of scenario qualify as being non-stupid?
The other difference is that you can’t demonstrate it. The leading question wasn’t about whether one should update from the quote, it was whether it’s a good idea to choose the quote as a negative example in a book about rationality. Even if Steven Pinker were 100% sure that there is no reason to update here, it’s still not a good example if he can’t prove it. I mean, if you accept that the quote could plausibly be non-stupid, then the honest way to talk about would have been “Here is Yang saying {quote}. If it turns out that ‘some of the smartest people in the world’ is made-up or refers to people aren’t actually impressive, this will have been an example of a vapid appeal-to-authority. I can’t prove that this is so, but here are my reasons for suspecting it.” And again, I can’t imagine it’s hard to find examples of appeals-to-authority that are clear-cut fallacies.
No. That’s a good point. It seems like “fallacious argument from authority” lends itself to black-and-white thinking that’s just not appropriate in many cases. Reading the tea leaves has its value. If I had to guess, Pinker was looking for a timely quote by a politician his readers are likely to be sympathetic to, and this one was convenient.
I still think that there are many times when it’s best as a rule to just dismiss statements with the form of “arguments from authority.” This fits the criteria, and it might be that sometimes you throw out the baby with the bathwater this way. Then again, there could be equal value in becoming sensitive to when it’s appropriate to “tune in” to this sort of evidence. That probably depends on the individual and their goals.
It would be interesting to know exactly what Pinker wrote. For instance, imagine that he wrote something like this:
I wouldn’t find that damning evidence against Pinker’s expertise in rationality.
FWIW, this was the full paragraph that I pulled the quote from,
I maintain that the point is stupid. Putting aside that ‘smart’ could entail rationality, the bigger issue is that Steven argues [it’s non-conclusive] ⇒ [it’s a fallacy]. Certainly intelligent people can believe false things, but as long as intelligence correlates with being accurate, what Yang is is still Bayesian evidence, just as you said in the review.
And it’s not hard to find better examples for irrationality. Julia Galef’s book contains a bunch of those, and they’re all clear-cut.
If all the information you had about a person was some.very generic information about their IQ or rationality quotient,your best option would be to believe the person who has the highest.
But that is almost never the case. Experts have certificates indicating their domaon specific knowledge.
Would you want a random person with an IQ of 180 performing a surgical operation on you?
But very weak Bayesian evidence. The human brain can’t physically deal with every small.or very large quantities. You’re much better off disregarding very weak evidence.
Yes. This shouldn’t be confused with regarding it as lack of evidence. Occasionally you are better off making use of it after all, if the margins for a decision are slim. Evidence is never itself fallacious, the error is its misrepresentation as something that it isn’t, and an act of labeling something as a fallacy can itself easily fall into such a fallacy, for example implying that weak evidence of certain form should be seen as lack of evidence or as counterevidence.
If we didn’t have professional surgeons (and I need the surgery), then yes, and we don’t have something analogous to professional surgeons for predicting the future. (Maybe superforecasters, but that standard is definitely not relevant if we’re comparing Yang to the average politician.)
We do have people with expertise relevant to making the sort of prediction Yang’s talking about, though. For instance:
AI researchers probably have a better idea than randomly chosen very smart people of what the state of AI is likely to be a decade from now.
Economists probably have a better idea than randomly chosen very smart people of whether the likely outcome of a given level of AI progress looks more like “oh noes 1⁄3 of all Americans have lost their jobs” or “1/3 of all Americans find that the nature of their work has changed” or “no one actually needs a job any more”.
As a field AI researchers are in AI research because they believe in it’s potential. While they do have expertise there’s also a bias.
If you would have listened to AI researchers five years ago we would have driverless cars by now.
Damn! If only I’d listened to AI researchers five years ago.
(I know what you meant :-).)
Yes, it’s true that AI researchers’ greater expertise is to some extent counterbalanced by possible biases.I still think it’s likely that a typical eminent AI researcher has a better idea of the likely level of technological obsolescence over the next ~decade than a typical randomly chosen person with (say) an IQ over 160.
(I don’t think corresponding things are always true. For instance, I am not at all convinced that a randomly chosen eminent philosopher-of-religion has a better idea on average of whether there are any gods and what they’re like if so than a randomly chosen very clever person. I think it depends on how much real expertise is possible in a given field. In AI there’s quite a lot.)
Knowing whether AI will make a field obsolate takes both expertise of AI and expertise of the given field.
There’s an xkcd for that https://xkcd.com/793/
I agree that people who are both AI experts and truck drivers (or executives at truck-driving companies) will have a better idea of how many Americans will lose their truck-driving jobs because they get automated away, and likewise for other jobs.
Relatively few people are expert both in AI and in other fields at risk of getting automated away. I think having just expertise in AI gives you a better chance than having neither. I don’t know who Yang’s “smartest people” actually were, but if they aren’t people with either specific AI expertise, or specific expertise in areas likely to fall victim to automation, or maybe expertise in labor economics, then I think their pronouncements about how many Americans are going to lose their jobs to automation in the near future are themselves examples of the phenomenon that xkcd comic is pointing at.
(Also, e.g., truck drivers may well have characteristic biases when they talk about the likely state of the truck driving industry a decade from now, just as AI researchers may.)
Fallacy fallacy
The fallacy is inappropriate appeal to authority. A geologist is not an apporiate authority on what is truly a fruit or berry. And you should not disregard your doctor’s advice just because they’re an authority on medicine.
The word “smart” is nowadays used to mean something like “expert on everything”...but how likely is that? It takes half a lifetime to get serious domain-specific knowledge of just one thing.