For me, in just about every case, the credence I’d assign to an AI’s wacky claims would depend on its ability to answer followup questions. For instance, in Eliezer’s examples:
What Orbital Mind Control Lasers? Who uses them? What do they do with them? Why haven’t they come up with a way to get around the hats?
2) All mathematical reasoning involving “infinities” involves self-evident contradictions, but human mathematicians have a blind spot with respect to them.
I’m actually strangely comfortable with this one, possibly because I’m bad at math.
3) You are not above-average; most people believe in the existence of a huge fictional underclass in order to place themselves at the top of the heap, rather than in the middle. This is why so many of your friends seem to have PhDs despite PhDs supposedly constituting only 0.5% of the population. You are actually in the bottom third of the population; the other two-thirds have already built their own AIs.
Why haven’t I heard of any of these other AIs before? How do all of the people producing statistics indicating that there are a lot of dumb people coordinate their efforts to perpetuate the fiction?
4) The human bias toward overconfidence is far deeper than we are capable of recognizing; we have a form of species overconfidence which denies all evidence against itself. Humans are much slower runners than we think, muscularly weaker, struggle to keep afloat in the water let alone move, and of course, are poorer thinkers.
Why do so few of us die of drowning (or any of the other things that would kill us if we were so dramatically more pathetic than we believe)? If this bias is so pervasive, why can I see these words on the AI’s screen, when it seems that I should block them out as with all over evidence that we are pathetic in this way?
5) Dogs, cats, cows, and many other mammals are capable of linguistic reasoning and have made many efforts to communicate with us, but humans are only capable of recognizing other humans as capable of thought.
If we have this incapability, what explains the abundant fiction in which nonhuman animals (both terrestrial and non) are capable of speech, and childhood anthropomorphization of animals? Can you teach me to talk to the stray cat in my neighborhood? Why only mammals, not birds and the like? What about people who are actively trying to communicate with animals like gorillas, or are those not capable of communication?
6) Humans cannot reproduce without the aid of the overlooked third sex.
Are they overlooked in the sense that people we can otherwise detect are not recognized as being part of this sex, or in the sense that we literally do not notice the existence of the members of this sex? In the former case, how do so many people manage to reproduce without apparently wanting to or involving third parties? In the latter case, how can I get in touch with these people? By what mechanism are they involved in human reproduction?
7) The Earth is flat.
Are we talking Euclidean spacetime here? What is the explanation for the observations of a spheroid Earth?
8) Human beings are incapable of writing fiction; all supposed fiction you have read is actually true.
In this universe? What about stories with plot holes? I think that I have written fiction in the past; am I in causal contact with the events I describe? When I make an edit that changes the plot, how does that work? What about people who write self-insertions?
If we have this incapability, what explains the abundant fiction in which nonhuman animals (both terrestrial and non) are capable of speech, and childhood anthropomorphization of animals?
That’s not anthropomorphization.
Can you teach me to talk to the stray cat in my neighborhood?
Sorry, you’re too old. Those childhood conversations you had with cats were real. You just started dismissing them as make-believe once your ability to doublethink was fully mature.
All of the really interesting stuff, from before you could doublethink at all, has been blocked out entirely by infantile amnesia.
Why haven’t I heard of any of these other AIs before?
You have. They’re in the news every day.
How do all of the people producing statistics indicating that there are a lot of dumb people coordinate their efforts to perpetuate the fiction?
Perpetuate what fiction? They produce statistics about all the dumb people, compiled into glossy magazines. Hell, you’re wearing a ‘bottom thirder’ sleeve button on your shirt right now.
Why haven’t I heard of any of these other AIs before? How do all of the people producing statistics indicating that there are a lot of dumb people coordinate their efforts to perpetuate the fiction?
They’re smarter than you, remember. Of course they can coordinate a little global deception.
They’re usually very kind—most of us don’t like to hurt the bottom percentile’s feelings (that you’re only in the bottom third is actually one of their polite fictions to cushion the shock when you begin to realize the obvious truth of your inferiority).
For me, in just about every case, the credence I’d assign to an AI’s wacky claims would depend on its ability to answer followup questions. For instance, in Eliezer’s examples:
What Orbital Mind Control Lasers? Who uses them? What do they do with them? Why haven’t they come up with a way to get around the hats?
I’m actually strangely comfortable with this one, possibly because I’m bad at math.
Why haven’t I heard of any of these other AIs before? How do all of the people producing statistics indicating that there are a lot of dumb people coordinate their efforts to perpetuate the fiction?
Why do so few of us die of drowning (or any of the other things that would kill us if we were so dramatically more pathetic than we believe)? If this bias is so pervasive, why can I see these words on the AI’s screen, when it seems that I should block them out as with all over evidence that we are pathetic in this way?
If we have this incapability, what explains the abundant fiction in which nonhuman animals (both terrestrial and non) are capable of speech, and childhood anthropomorphization of animals? Can you teach me to talk to the stray cat in my neighborhood? Why only mammals, not birds and the like? What about people who are actively trying to communicate with animals like gorillas, or are those not capable of communication?
Are they overlooked in the sense that people we can otherwise detect are not recognized as being part of this sex, or in the sense that we literally do not notice the existence of the members of this sex? In the former case, how do so many people manage to reproduce without apparently wanting to or involving third parties? In the latter case, how can I get in touch with these people? By what mechanism are they involved in human reproduction?
Are we talking Euclidean spacetime here? What is the explanation for the observations of a spheroid Earth?
In this universe? What about stories with plot holes? I think that I have written fiction in the past; am I in causal contact with the events I describe? When I make an edit that changes the plot, how does that work? What about people who write self-insertions?
That’s not anthropomorphization.
Sorry, you’re too old. Those childhood conversations you had with cats were real. You just started dismissing them as make-believe once your ability to doublethink was fully mature.
All of the really interesting stuff, from before you could doublethink at all, has been blocked out entirely by infantile amnesia.
Good point; “Children are sane” belongs somewhere high on the list.
You have. They’re in the news every day.
Perpetuate what fiction? They produce statistics about all the dumb people, compiled into glossy magazines. Hell, you’re wearing a ‘bottom thirder’ sleeve button on your shirt right now.
Yes. Yes you are.
They’re smarter than you, remember. Of course they can coordinate a little global deception.
I was asking after their motivation more than their capabilities. (The AIs, not the statisticians.)
They’re usually very kind—most of us don’t like to hurt the bottom percentile’s feelings (that you’re only in the bottom third is actually one of their polite fictions to cushion the shock when you begin to realize the obvious truth of your inferiority).