I’m questioning whether the Turing test is closely related to machine thinking, for machines calibrated to pass the Turing test.
If “Can machines think?” is a meaningless question, then “Does the Turing test tell us whether machines can think?” must be equally meaningless. (That is, until we figure out what the heck we’re asking when we ask “Can this machine think?”, investigating how closely related the Turing test is to said issue is futile.)
Now, the following is my interpretation and not a paraphrasing of Turing. I think what Turing meant when he said that “can machines think” and “can a machine win the imitation game” are related questions is this: just like “can machines think” depends strongly on just what we mean by “think”, which depends on societal views and prevailing attitudes (see quote below[1]), so is our interpretation of the imitation game, and what an agent’s performance on it implies about that agent, closely related to societal attitudes and perception of agents as “thinking”, “conscious”, etc.
Turing thought that societal attitudes shaped what we think “thinking” is, and what kinds of things “think”. He also thought that success in the imitation game would herald a change in societal attitudes, such that people would think of machines as being able to “think”, and that this would render the philosophical discussion moot. At no point in this process do we ever define “thinking” or undertake any systematic way of determining whether an agent “thinks” or not.
Personally, I think he was right: at some point, past some threshold of apparent similarity of machines to humans, societal attitudes will shift and this whole business of administering tests to detect some magic spark will be rendered moot. Of course, that hasn’t happened yet, so here we are, still lacking anything resembling a proper definition of “thinking”. The Turing test does not help us in that regard.
[1] The promised quote:
I propose to consider the question “Can machines think?” This should begin with definitions of the terms “machine” and “think.” The definitions might be framed so as to reflect so far as possible the normal use of the words, but this attitude is dangerous. If the meaning of the words “machine” and “think” are to be found by examining how they are commonly used it is difficult to escape the conclusion that the meaning and answer to the question, “Can machines think?” is to be sought in a statistical survey such as a Gallup poll. But this is absurd. Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words.
I take your point, but Turing’s paper wasn’t simply an exercise in applied sociology. And the Turing test does help detect thinking, without having to define it: just consider applying it to a whole brain emulation. The Turing test and the definition of thinking are related; Truing was being disingenuous if he was pretending otherwise. He was actually proposing a definition of thinking, and stating that it would become the universally accepted one, the one that would be the “correct” simplification of the currently muddle concept.
The following is an extract from the typewritten script of a BBC radio broadcast
entitled ‘Can Automatic Calculating Machines Be Said To Think’, recorded in
January 1952. In response to the introductory remarks
We’re here today to discuss whether calculating machines can be said to think
in any proper sense of the word. … Turing, … [h]ave you a mechanical defini-
tion?,
Turing replies:
I don’t want to give a definition of thinking, but if I had to I should probably be
unable to say anything more about it than that it was a sort of buzzing that went
on inside my head. But I don’t really see that we need to agree on a definition
at all. The important thing is to try to draw a line between the properties of a
brain, or of a man, that we want to discuss, and those that we don’t. To take an
extreme case, we are not interested in the fact that the brain has the consistency
of cold porridge. We don’t want to say ‘This machine’s quite hard, so it isn’t
a brain, and so it can’t think.’ I would like to suggest a particular kind of
test that one might apply to a machine. You might call it a test to see whether the
machine thinks, but it would be better to avoid begging the question, and say
that the machines that pass are (let’s say) ‘Grade A’ machines.
[...]
Well, that’s my test. Of course I am not saying at present either that machines
really could pass the test, or that they couldn’t. My suggestion is just that this
is the question we should discuss. It’s not the same as ‘Do machines think,’
but it seems near enough for our present purpose, and raises much the same
difficulties.
Certainly the Turing test can be viewed as an operationalization of “does this machine think?”. No argument there. I also agree with you concerning what Turing probably had in mind.
The problem is that if we have in mind (perhaps not even explicitly) some different definition of thinking or, gods forbid, some other property entirely, like “consciousness”, then the Turing test immediately stops being of much use.
Here is a related thing. John Searle, in his essay “Minds, Brains, and Programs” (where he presents the famous “Chinese room” thought experiment), claims that even if you a) place the execution of the “Chinese room” program into a robot body, which is then able to converse with you in Chinese, or b) simulate the entire brain of a native Chinese speaker neuron-by-neuron, and optionally put that into a robot body, you will still not have a system that possesses true understanding of Chinese.
Now, taken to its logical extreme, this is surely an absurd position to take in practice. We can imagine a scenario where Searle meets a man on the street, strikes up a conversation (perhaps in Chinese), and spends some time discoursing with the articulate stranger on various topics from analytic philosophy to dietary preferences, getting to know the man and being impressed with his depth of knowledge and originality of thought, until at some point, the stranger reaches up and presses a hidden button behind his ear, causing the top of his skull to pop open and reveal that he is in fact a robot with an electronic brain! Dun dun dun! He then hands Searle a booklet detailing his design specs and also containing the entirety of his brain’s source code (in very fine print), at which point Searle declares that the stranger’s half of the entire conversation up to that point has been nothing but the meaningless blatherings of a mindless machine, devoid entirely of any true understanding.
It seems fairly obvious to me that such entities would, like humans, be beneficiaries of what Turing called “the polite convention” that people do, in fact, think (which is what lets us not be troubled by the problem of other minds in day-to-day life). But if someone like John Searle were to insist that we nonetheless have no direct evidence for the proposition that the robots in question do “think”, I don’t see that we would have a good answer for him. (Searle’s insistence that we shouldn’t question whether humans can think is, of course, hypocritical, but that is not relevant here.) Social conventions to treat something as being true do not constitute a demonstration that said thing is actually true.
at which point Searle declares that the stranger’s half of the entire conversation up to that point has been nothing but the meaningless blatherings of a mindless machine, devoid entirely of any true understanding.
It is perhaps worth noting that Searle explicitly posits in that essay that the system is functioning as a Giant Lookup Table.
If faced with an actual GLUT Chinese Room… well, honestly, I’m more inclined to believe that I’m being spoofed than trust the evidence of my senses.
But leaving that aside, if faced with something I somehow am convinced is a GLUT Chinese Room, I have to rethink my whole notion of how complicated conversation actually is, and yeah, I would probably conclude that the entire conversation up to that point has been devoid entirely of any true understanding. (I would also have to rethink my grounds for believing that humans have true understanding.)
Actually, Searle’s description of the thought experiment does include a “program”, a set of rules for manipulating the Chinese symbols provided to the room’s occupant. Searle also addresses a version of the contrary position (the pro-AI position, as it were) that posits a simulation of an actual brain (to which I alluded in the grandparent). He doesn’t think that would possess true understanding, either.
I think that if we’ve gotten to the point where we’re rethinking whether humans have true understanding, we should instead admit that we haven’t the first clue what “true understanding” is or what relation, if any, said mysterious property has to do with whatever we’re detecting in our test subjects.
Wouldn’t such a GLUT by necessity require someone possessing immensely fine understanding of Chinese and English both, though? You could then say that the person+GLUT system as a whole understands Chinese, as it combines both the person’s symbol-manipulation capabilities and the actual understanding represented by the GLUT.
You might still not possess understanding of Chinese, but that does not mean a meaningful conversation has not taken place.
I have no idea whether a GLUT-based Chinese Room would require someone possessing immensely fine understanding of Chinese and English both. As far as I can tell, a GLUT-based Chinese Room is impossible, and asking what is or isn’t required to bring about an impossible situation seems a silly question. Conversely, if it turns out that a GLUT-based Chinese Room is not impossible, I don’t trust my intuitions about what is or isn’t required to construct one.
I have no problem with saying a Chinese-speaking-person+GLUT system as a whole understands Chinese, in much the same sense that I have no problem saying that a Chinese-speaking-person+tuna-fish-sandwich system as a whole understands Chinese. I’m not sure how interesting that is.
I’m perfectly content to posit an artificial system capable of understanding Chinese and having a meaningful conversation. I’m unable to conceive specifically of a GLUT that can do so.
I’m perfectly content to posit an artificial system capable of understanding Chinese and having a meaningful conversation. I’m unable to conceive specifically of a GLUT that can do so.
I don’t think it’s that hard to conceive of. Imagine that the Simulation Argument is true; then, we could easily imagine a GLUT that exists outside of our own simulation, using additional resources; then our Chinese Room could just be an interface for such a GLUT.
As you said though, I don’t find the proposal very interesting, especially since I’m not a big fan of the Simulation Argument anyway.
I find I am unable, on brief consideration, to conceive of a GLUT sitting in some real world within which my observable universe is being computed… I have no sense of what such a thing might be like, or what its existence implies about the real world and how it differs from my observed simulation, or really much of anything interesting.
It’s possible that I might be able to if I thought about it for noticeably longer than I’m inclined to.
Not necessarily. Theoretically, one could have very specific knowledge of Chinese, possibly acquired from very limited but deep experience. Imagine one person who has spoken Chinese only at the harbor, and has complete and total mastery of the maritime vocabulary of Chinese but would lack all but the simplest verbs relevant to the conversations happening just a mile further inland. Conceivably, a series of experts in a very localized domain could separately contribute their understanding, perhaps governed by a person who understands (in English) every conceivable key to the GLUT, but does not understand the values which must be placed in it.
Then, imagine someone whose entire knowledge of Chinese is the translation of the phrase: “Does my reply make sense in the context of this conversation?” This person takes an arbitrary amount of time, randomly combining phonemes and carrying out every conceivable conversation with an unlimited supply of Chinese speakers. (This is substantially more realistic if there are many people working in a field with fewer potential combinations than language). Through perhaps the least efficient trial and error possible, they learn to carry on a conversation by rote, keeping only those conversational threads which, through pure chance, make sense throughout the entire dialogue.
In neither of these human experts do we find a real understanding of Chinese. It could be said that the understandings of the domain experts combine to form one great understanding, but the inefficient trial-and-error GLUT manufacturers certainly do not have any understanding, merely memory.
I agree on the basic point, but then my deeper point was that somewhere down the line you’ll find the intelligence(s) that created a high-fidelity converter for an arbitrary amount of information from one format to another. Sarle is free to claim that the system does not understand Chinese, but its very function could only have been imparted by parties who collectively speak Chinese very well, making the room at very least a medium of communication utilizing this understanding.
And this is before we mention the entirely plausible claim that the room-person system as a whole understands Chinese, even though neither of its two parts does. Any system you’ll take apart to sufficient degrees will stop displaying the properties of the whole, so having us peer inside an electronic brain asking “but where does the intelligence/understanding reside?” misses the point entirely.
Theoretically, one could have very specific knowledge of Chinese, possibly acquired from very limited but deep experience. Imagine one person who has spoken Chinese only at the harbor, and has complete and total mastery of the maritime vocabulary of Chinese but would lack all but the simplest verbs relevant to the conversations happening just a mile further inland. Conceivably, a series of experts in a very localized domain could separately contribute their understanding, perhaps governed by a person who understands (in English) every conceivable key to the GLUT, but does not understand the values which must be placed in it.
This does not pass the simplest plausibility test. Do you imagine that being at a harbor causes people to have only conversations which are uniquely applicable to harbor activities? Does one not need words and phrases for concepts like “person”, “weather”, “hello”, “food”, “where”, “friend”, “tomorrow”, “city”, “want”, etc., not to mention rules of Chinese grammar and syntax? Such a “harbor-only” Chinese speaker may lack certain specific vocabulary, but he certainly will not lack a general understanding of Chinese.
Your other example is even sillier, especially given that the number of possible conversations in a human language is infinite. For one thing, a conversation where one person is constantly asking “Does my reply make sense?” is very, very different from the “same” conversation without such constant verbal monitoring. (Not to mention the specific fact that your imaginary expert would not be able to understand his interlocutor’s response to his question about whether his utterances made sense.)
A more realistic version would be for for an observer to record all conversations between two Chinese speakers with length N, where N is some arbitrarily large but still finite conversation length. (If a GLUT were to capture every possible conversation, you are correct in saying that it would have to be infinite).
From a sufficiently large sample size (though it is implausible to capture every probable conversation in any realistic amount of time, not to mention in any amount of time during which the language is relatively stable and unchanging), a tree of conversations could be built, with an arbitrarily large probability of including a given conversation within it.
From this, one could built a GLUT (though it would probably be more efficient as a tree) of the possible questions given context and the appropriate responses. Though it would be utterly unfeasible to build, that is a limitation of the availability of data, rather than the GLUT structure itself. It would not be perfect—one cannot build an infinite GLUT, nor can one acquire the infinite amount of data with which to fill it—but it could, perhaps, surpass even a native speaker by some measures.
Consider: what would the table contain as appropriate responses for the following questions? (Each question would certainly appear many, many times in our record of all conversations up to length N.)
“Hello, what is your name?”
“Where do you live?”
“What do you look like?”
“Tell me about your favorite television show.”
Remember that a GLUT, by definition, matches each input to one output. If you have to algorithmically consider context, whether environmental (what year is it? where are we?), personal (who am I?), or conversation history (what’s been said up to this point?), then that is not a GLUT, it is a program. You can of course convert any program that deterministically gives output for given input into a GLUT, but to do that successfully, you really do need all possible inputs and their outputs; and “input” here means “question, plus conversation history, plus complete description of world-state” (complete because we don’t know what context we’ll need in order to give an appropriate response).
In other words, to construct such a GLUT, you would have to be well-nigh omniscient. But, admittedly, you would not then have to “know” any Chinese.
But leaving that aside, if faced with something I somehow am convinced is a GLUT Chinese Room … I would probably conclude that the entire conversation up to that point has been devoid entirely of any true understanding.
I wouldn’t; or, at least, not necessarily.
Just for simplicity, let’s say we the agent and the experimenter are conversing via text-based chat. The agent and the experimenter take turns outputting a single line of text (it could be a very long line); the experimenter always goes first, saying “hello”.
In this case, the agent’s side of the conversation can be modeled as the function F(H(t), t) where t is the current line number, and H(t) is the sequence of inputs that the agent received up to point t. Thus, H(0) is always {”hello”}, as per above. H(2) might be something like {”hello”, “your name sounds funny”}, etc.
We know that, since F is a function, it is a relation that maps each possible input to a single output—so it’s basically a lookup table. In the ideal case, the number of possible inputs is infinite (trivially, we could say “hello”, “helloo”, “hellooo”, and so on, and infinitum), and thus the lookup table would need to be infinitely large. However, we humans are finite creatures, and thus in practice the lookup table would only need to be finitely large.
Of course, practically speaking, you’d probably still need a storage device larger than the Universe to encode even a finite lookup table of sufficient size; but this is a practical objection, which does not a priori prohibit us from implementing a Turing-grade agent as a GLUT.
You’re correct that it doesn’t a priori prohibit such a thing. It does, however, bring my prior probability of encountering such a thing vanishingly low. Faced with an event that somehow causes me to update that vanishingly small probability to the point of convincing me it’s true I am vastly surprised, and that vast surprise colors all of my intuitions about interactions with nominally intelligent systems. Given that, it’s not clear to me why I should keep believing that I was having an intelligent conversation a moment earlier.
...and that vast surprise colors all of my intuitions about interactions with nominally intelligent systems. Given that, it’s not clear to me why I should keep believing that I was having an intelligent conversation a moment earlier.
What do you mean by “intelligent conversation” ? Do you mean, “a conversation with an intelligent agent”, or “a conversation whose contents satisfy certain criteria”, and if so, which ones ? I’ll assume you mean the former for now.
Let’s say that you had a text-only chat with the agent, and found it intellectually stimulating. You thought that the agent was responding quite cleverly to your comments, had a distinct “writer’s voice”, etc.
Now, let’s imagine two separate worlds. In world A, you learned that the agent was in fact a GLUT. Surprised and confused, you confronted it in conversation, and it responded to your comments as it did before, with apparent intelligence and wit. But, of course, now you knew better than to fall for such obvious attempts to fool you.
In world B, the exact same thing happened, and the rest of your conversation proceeded as before, with one minor difference: unbeknownst to you, the person who told you that the agent was a GLUT was himself a troll. He totally gaslighted you. The agent isn’t a GLUT or even an AI; it’s just a regular human (and the troll’s accomplice), a la the good old Mechanical Turk.
It sounds to me like if you were in world B, you’d still disbelieve that you were having a conversation with an intelligent agent. But in world B, you’d be wrong. In world A, you would of course be right.
Is there any way for you to tell which world you’re in (I mean, without waterboarding that pesky troll or taking apart the Chinese Room to see who’s inside, etc.) ? If there is no way for you to tell the difference, then what’s the difference ?
By the way, I do agree with you that, in our real world, the probability of such a GLUT existing is pretty much zero. I am merely questioning the direction of your (hypothetical) belief update.
By construction, there’s no way for me to tell… that is, I’ve already posited that some event (somehow, implausibly) convinced me my interlocutor is a GLUT.
In world A, “I” was correct to be convinced; my interlocutor really was (somehow, implausibly) a GLUT, impossible as that seems. In world B, “I” was (somehow, implausibly) incorrectly convinced.
There’s all kinds of things I can be fooled about, and knowing that I can be fooled about those things should (and does) make me more difficult to convince of them. But if, even taking that increased skeptcism into account, I’m convinced anyway… well, what more is there to say? At that point I’ve been (somehow, implausibly) convinced, and should behave accordingly.
To say “Even if I’ve (somehow, implausibly) been exposed to a convincing event, I don’t update my beliefs” is simply another way of saying that no such convincing event can exist—of fighting the hypothetical.
Mind you, I agree that no such convincing event can exist, and that the hypothetical simply is not going to happen. But that’s precisely my point: if it does anyway, then I am clearly deeply confused about how the universe works; I should at that point sharply lower my confidence in all judgments even vaguely related to the nonexistence of GLUT Chinese Rooms, including “I can tell whether I’m talking to an intelligent system just by talking to them”.
The extent of my confidence in X ought to be proportional to the extent of my confusion if I come (somehow) to believe that X is false.
I think I see what you’re saying—discovering that something as unlikely as a GLUT actually exists would shake your beliefs in pretty much everything, including the Turing Test. This position makes sense, but I think it’s somewhat orthogonal to the current topic. Presumably, you’d feel the same way if you became convinced that gods exist, or that Pi has a finite number of digits after all, or something.
discovering that something as unlikely as a GLUT actually exists would shake your beliefs in pretty much everything, including the Turing Test
Not quite.
Discovering that something as unlikely as a conversation-having GLUT exists would shake my beliefs in everything related to conversation-having GLUTs. My confidence that I’m wearing socks right now would not decrease much, but my confidence that I can usefully infer attributes of a system by conversing with it would decrease enormously. Since Turing Tests are directly about the latter, my confidence about Turing Tests would also decrease enormously.
More generally, any event that causes me to sharply alter my confidence in a proposition P will also tend to alter my confidence in other propositions related to P, to an extent proportional to their relation.
An event which made me confident that pi was a terminating decimal after all, or that some religion’s account of its god(s) was accurate, etc. probably would not reduce my confidence in the Turing Test nearly as much, though it would reduce my confidence in other things more.
My confidence that I’m wearing socks right now would not decrease much...
Why not ? Encountering a bona-fide GLUT that could pass the Turing test would be tantamount to a miracle. I personally would begin questioning everything if something like that were to happen. After all, socks are objects that I had previously thought of as “physical”, but the GLUT would shake the very notions of what a “physical” object even is.
Since Turing Tests are directly about the latter, my confidence about Turing Tests would also decrease enormously.
my confidence about Turing Tests would also decrease enormously. Why that, and not your confidence about GLUTs ?
Of course my confidence about GLUTs would also decrease enormously in this scenario… sorry if that wasn’t clear.
More generally, my point here is that a conversation-having GLUT would not alter my confidence in all propositions equally, but rather would alter my confidence in propositions to a degree proportional to their relation to conversation-having GLUTs, and “I can usefully infer attributes of a system by conversing with it” (P1) is far more closely related to conversation-having GLUTs than “I’m wearing socks” (P2).
If your point is that my confidence in P2 should nevertheless be significant, even if much less than P1… well, maybe. Offhand, I’m not sure my brain is capable of spanning a broad enough span of orders-of-magnitude of confidence-shift to be able to consistently represent the updates of both P1 and P2, but I’m not confident either way.
I think the differences between us are rather small, in fact. I do have a different definition of thinking, which is not fully explicit. It would go along the lines of “a thinking machine should demonstrate human-like abilities in most situations and not be extremely stupid in some areas”. The intuition is that if there is a general intelligence, rather than simply a list of specific rules, then it’s competence shouldn’t completely collapse when facing unusual situations.
The “test systems on situations they’re not optimised” approach was trying to establish whether there would be such a collapse in skill. Of course you can’t test for every situation, but you can get a good idea this way.
Searle’s steadfast refusal to consider perfectly reasonable replies to his position, and his general recalcitrance in the debate on this and related questions, makes him unusually vulnerable to slightly uncharitable readings. The fact that his justification seems to be “human brains have unspecified magic that make humans conscious, and no I will not budge from that position because I have very strong intuitions” means, I think, that my reading is not even very uncharitable.
Oh, and on the subject of whole brain emulations: Greg Egan’s recent novel Zendegi (despite being, imo, rather poor overall), does make a somewhat convincing case that an emulation of a person’s brain/consciousness/personality might pass something like a Turing test and still not possess subjective consciousness or true general intelligence on a human level.
When this topic comes up I’m always reminded of a bit in John Varley’s Golden Globe, where our hero asks an advanced AI whether it’s actually conscious, and it replies “I’ve thought about that question a lot and have concluded that I’m probably not.”
If “Can machines think?” is a meaningless question, then “Does the Turing test tell us whether machines can think?” must be equally meaningless. (That is, until we figure out what the heck we’re asking when we ask “Can this machine think?”, investigating how closely related the Turing test is to said issue is futile.)
Now, the following is my interpretation and not a paraphrasing of Turing. I think what Turing meant when he said that “can machines think” and “can a machine win the imitation game” are related questions is this: just like “can machines think” depends strongly on just what we mean by “think”, which depends on societal views and prevailing attitudes (see quote below[1]), so is our interpretation of the imitation game, and what an agent’s performance on it implies about that agent, closely related to societal attitudes and perception of agents as “thinking”, “conscious”, etc.
Turing thought that societal attitudes shaped what we think “thinking” is, and what kinds of things “think”. He also thought that success in the imitation game would herald a change in societal attitudes, such that people would think of machines as being able to “think”, and that this would render the philosophical discussion moot. At no point in this process do we ever define “thinking” or undertake any systematic way of determining whether an agent “thinks” or not.
Personally, I think he was right: at some point, past some threshold of apparent similarity of machines to humans, societal attitudes will shift and this whole business of administering tests to detect some magic spark will be rendered moot. Of course, that hasn’t happened yet, so here we are, still lacking anything resembling a proper definition of “thinking”. The Turing test does not help us in that regard.
[1] The promised quote:
I take your point, but Turing’s paper wasn’t simply an exercise in applied sociology. And the Turing test does help detect thinking, without having to define it: just consider applying it to a whole brain emulation. The Turing test and the definition of thinking are related; Truing was being disingenuous if he was pretending otherwise. He was actually proposing a definition of thinking, and stating that it would become the universally accepted one, the one that would be the “correct” simplification of the currently muddle concept.
Well, there’s this:
http://swarma.org/thesis/doc/jake_224.pdf
Certainly the Turing test can be viewed as an operationalization of “does this machine think?”. No argument there. I also agree with you concerning what Turing probably had in mind.
The problem is that if we have in mind (perhaps not even explicitly) some different definition of thinking or, gods forbid, some other property entirely, like “consciousness”, then the Turing test immediately stops being of much use.
Here is a related thing. John Searle, in his essay “Minds, Brains, and Programs” (where he presents the famous “Chinese room” thought experiment), claims that even if you a) place the execution of the “Chinese room” program into a robot body, which is then able to converse with you in Chinese, or b) simulate the entire brain of a native Chinese speaker neuron-by-neuron, and optionally put that into a robot body, you will still not have a system that possesses true understanding of Chinese.
Now, taken to its logical extreme, this is surely an absurd position to take in practice. We can imagine a scenario where Searle meets a man on the street, strikes up a conversation (perhaps in Chinese), and spends some time discoursing with the articulate stranger on various topics from analytic philosophy to dietary preferences, getting to know the man and being impressed with his depth of knowledge and originality of thought, until at some point, the stranger reaches up and presses a hidden button behind his ear, causing the top of his skull to pop open and reveal that he is in fact a robot with an electronic brain! Dun dun dun! He then hands Searle a booklet detailing his design specs and also containing the entirety of his brain’s source code (in very fine print), at which point Searle declares that the stranger’s half of the entire conversation up to that point has been nothing but the meaningless blatherings of a mindless machine, devoid entirely of any true understanding.
It seems fairly obvious to me that such entities would, like humans, be beneficiaries of what Turing called “the polite convention” that people do, in fact, think (which is what lets us not be troubled by the problem of other minds in day-to-day life). But if someone like John Searle were to insist that we nonetheless have no direct evidence for the proposition that the robots in question do “think”, I don’t see that we would have a good answer for him. (Searle’s insistence that we shouldn’t question whether humans can think is, of course, hypocritical, but that is not relevant here.) Social conventions to treat something as being true do not constitute a demonstration that said thing is actually true.
It is perhaps worth noting that Searle explicitly posits in that essay that the system is functioning as a Giant Lookup Table.
If faced with an actual GLUT Chinese Room… well, honestly, I’m more inclined to believe that I’m being spoofed than trust the evidence of my senses.
But leaving that aside, if faced with something I somehow am convinced is a GLUT Chinese Room, I have to rethink my whole notion of how complicated conversation actually is, and yeah, I would probably conclude that the entire conversation up to that point has been devoid entirely of any true understanding. (I would also have to rethink my grounds for believing that humans have true understanding.)
I don’t expect that to happen, though.
Actually, Searle’s description of the thought experiment does include a “program”, a set of rules for manipulating the Chinese symbols provided to the room’s occupant. Searle also addresses a version of the contrary position (the pro-AI position, as it were) that posits a simulation of an actual brain (to which I alluded in the grandparent). He doesn’t think that would possess true understanding, either.
I think that if we’ve gotten to the point where we’re rethinking whether humans have true understanding, we should instead admit that we haven’t the first clue what “true understanding” is or what relation, if any, said mysterious property has to do with whatever we’re detecting in our test subjects.
Oh, and: GAZP vs. GLUT.
Wouldn’t such a GLUT by necessity require someone possessing immensely fine understanding of Chinese and English both, though? You could then say that the person+GLUT system as a whole understands Chinese, as it combines both the person’s symbol-manipulation capabilities and the actual understanding represented by the GLUT.
You might still not possess understanding of Chinese, but that does not mean a meaningful conversation has not taken place.
I have no idea whether a GLUT-based Chinese Room would require someone possessing immensely fine understanding of Chinese and English both. As far as I can tell, a GLUT-based Chinese Room is impossible, and asking what is or isn’t required to bring about an impossible situation seems a silly question. Conversely, if it turns out that a GLUT-based Chinese Room is not impossible, I don’t trust my intuitions about what is or isn’t required to construct one.
I have no problem with saying a Chinese-speaking-person+GLUT system as a whole understands Chinese, in much the same sense that I have no problem saying that a Chinese-speaking-person+tuna-fish-sandwich system as a whole understands Chinese. I’m not sure how interesting that is.
I’m perfectly content to posit an artificial system capable of understanding Chinese and having a meaningful conversation. I’m unable to conceive specifically of a GLUT that can do so.
I don’t think it’s that hard to conceive of. Imagine that the Simulation Argument is true; then, we could easily imagine a GLUT that exists outside of our own simulation, using additional resources; then our Chinese Room could just be an interface for such a GLUT.
As you said though, I don’t find the proposal very interesting, especially since I’m not a big fan of the Simulation Argument anyway.
I find I am unable, on brief consideration, to conceive of a GLUT sitting in some real world within which my observable universe is being computed… I have no sense of what such a thing might be like, or what its existence implies about the real world and how it differs from my observed simulation, or really much of anything interesting.
It’s possible that I might be able to if I thought about it for noticeably longer than I’m inclined to.
If you can do so easily, good for you.
Not necessarily. Theoretically, one could have very specific knowledge of Chinese, possibly acquired from very limited but deep experience. Imagine one person who has spoken Chinese only at the harbor, and has complete and total mastery of the maritime vocabulary of Chinese but would lack all but the simplest verbs relevant to the conversations happening just a mile further inland. Conceivably, a series of experts in a very localized domain could separately contribute their understanding, perhaps governed by a person who understands (in English) every conceivable key to the GLUT, but does not understand the values which must be placed in it.
Then, imagine someone whose entire knowledge of Chinese is the translation of the phrase: “Does my reply make sense in the context of this conversation?” This person takes an arbitrary amount of time, randomly combining phonemes and carrying out every conceivable conversation with an unlimited supply of Chinese speakers. (This is substantially more realistic if there are many people working in a field with fewer potential combinations than language). Through perhaps the least efficient trial and error possible, they learn to carry on a conversation by rote, keeping only those conversational threads which, through pure chance, make sense throughout the entire dialogue.
In neither of these human experts do we find a real understanding of Chinese. It could be said that the understandings of the domain experts combine to form one great understanding, but the inefficient trial-and-error GLUT manufacturers certainly do not have any understanding, merely memory.
I agree on the basic point, but then my deeper point was that somewhere down the line you’ll find the intelligence(s) that created a high-fidelity converter for an arbitrary amount of information from one format to another. Sarle is free to claim that the system does not understand Chinese, but its very function could only have been imparted by parties who collectively speak Chinese very well, making the room at very least a medium of communication utilizing this understanding.
And this is before we mention the entirely plausible claim that the room-person system as a whole understands Chinese, even though neither of its two parts does. Any system you’ll take apart to sufficient degrees will stop displaying the properties of the whole, so having us peer inside an electronic brain asking “but where does the intelligence/understanding reside?” misses the point entirely.
This does not pass the simplest plausibility test. Do you imagine that being at a harbor causes people to have only conversations which are uniquely applicable to harbor activities? Does one not need words and phrases for concepts like “person”, “weather”, “hello”, “food”, “where”, “friend”, “tomorrow”, “city”, “want”, etc., not to mention rules of Chinese grammar and syntax? Such a “harbor-only” Chinese speaker may lack certain specific vocabulary, but he certainly will not lack a general understanding of Chinese.
Your other example is even sillier, especially given that the number of possible conversations in a human language is infinite. For one thing, a conversation where one person is constantly asking “Does my reply make sense?” is very, very different from the “same” conversation without such constant verbal monitoring. (Not to mention the specific fact that your imaginary expert would not be able to understand his interlocutor’s response to his question about whether his utterances made sense.)
You make some valid points.
A more realistic version would be for for an observer to record all conversations between two Chinese speakers with length N, where N is some arbitrarily large but still finite conversation length. (If a GLUT were to capture every possible conversation, you are correct in saying that it would have to be infinite).
From a sufficiently large sample size (though it is implausible to capture every probable conversation in any realistic amount of time, not to mention in any amount of time during which the language is relatively stable and unchanging), a tree of conversations could be built, with an arbitrarily large probability of including a given conversation within it.
From this, one could built a GLUT (though it would probably be more efficient as a tree) of the possible questions given context and the appropriate responses. Though it would be utterly unfeasible to build, that is a limitation of the availability of data, rather than the GLUT structure itself. It would not be perfect—one cannot build an infinite GLUT, nor can one acquire the infinite amount of data with which to fill it—but it could, perhaps, surpass even a native speaker by some measures.
I remain dubious.
Consider: what would the table contain as appropriate responses for the following questions? (Each question would certainly appear many, many times in our record of all conversations up to length N.)
“Hello, what is your name?”
“Where do you live?”
“What do you look like?”
“Tell me about your favorite television show.”
Remember that a GLUT, by definition, matches each input to one output. If you have to algorithmically consider context, whether environmental (what year is it? where are we?), personal (who am I?), or conversation history (what’s been said up to this point?), then that is not a GLUT, it is a program. You can of course convert any program that deterministically gives output for given input into a GLUT, but to do that successfully, you really do need all possible inputs and their outputs; and “input” here means “question, plus conversation history, plus complete description of world-state” (complete because we don’t know what context we’ll need in order to give an appropriate response).
In other words, to construct such a GLUT, you would have to be well-nigh omniscient. But, admittedly, you would not then have to “know” any Chinese.
I wouldn’t; or, at least, not necessarily.
Just for simplicity, let’s say we the agent and the experimenter are conversing via text-based chat. The agent and the experimenter take turns outputting a single line of text (it could be a very long line); the experimenter always goes first, saying “hello”.
In this case, the agent’s side of the conversation can be modeled as the function F(H(t), t) where t is the current line number, and H(t) is the sequence of inputs that the agent received up to point t. Thus, H(0) is always {”hello”}, as per above. H(2) might be something like {”hello”, “your name sounds funny”}, etc.
We know that, since F is a function, it is a relation that maps each possible input to a single output—so it’s basically a lookup table. In the ideal case, the number of possible inputs is infinite (trivially, we could say “hello”, “helloo”, “hellooo”, and so on, and infinitum), and thus the lookup table would need to be infinitely large. However, we humans are finite creatures, and thus in practice the lookup table would only need to be finitely large.
Of course, practically speaking, you’d probably still need a storage device larger than the Universe to encode even a finite lookup table of sufficient size; but this is a practical objection, which does not a priori prohibit us from implementing a Turing-grade agent as a GLUT.
You’re correct that it doesn’t a priori prohibit such a thing. It does, however, bring my prior probability of encountering such a thing vanishingly low. Faced with an event that somehow causes me to update that vanishingly small probability to the point of convincing me it’s true I am vastly surprised, and that vast surprise colors all of my intuitions about interactions with nominally intelligent systems. Given that, it’s not clear to me why I should keep believing that I was having an intelligent conversation a moment earlier.
What do you mean by “intelligent conversation” ? Do you mean, “a conversation with an intelligent agent”, or “a conversation whose contents satisfy certain criteria”, and if so, which ones ? I’ll assume you mean the former for now.
Let’s say that you had a text-only chat with the agent, and found it intellectually stimulating. You thought that the agent was responding quite cleverly to your comments, had a distinct “writer’s voice”, etc.
Now, let’s imagine two separate worlds. In world A, you learned that the agent was in fact a GLUT. Surprised and confused, you confronted it in conversation, and it responded to your comments as it did before, with apparent intelligence and wit. But, of course, now you knew better than to fall for such obvious attempts to fool you.
In world B, the exact same thing happened, and the rest of your conversation proceeded as before, with one minor difference: unbeknownst to you, the person who told you that the agent was a GLUT was himself a troll. He totally gaslighted you. The agent isn’t a GLUT or even an AI; it’s just a regular human (and the troll’s accomplice), a la the good old Mechanical Turk.
It sounds to me like if you were in world B, you’d still disbelieve that you were having a conversation with an intelligent agent. But in world B, you’d be wrong. In world A, you would of course be right.
Is there any way for you to tell which world you’re in (I mean, without waterboarding that pesky troll or taking apart the Chinese Room to see who’s inside, etc.) ? If there is no way for you to tell the difference, then what’s the difference ?
By the way, I do agree with you that, in our real world, the probability of such a GLUT existing is pretty much zero. I am merely questioning the direction of your (hypothetical) belief update.
By construction, there’s no way for me to tell… that is, I’ve already posited that some event (somehow, implausibly) convinced me my interlocutor is a GLUT.
In world A, “I” was correct to be convinced; my interlocutor really was (somehow, implausibly) a GLUT, impossible as that seems. In world B, “I” was (somehow, implausibly) incorrectly convinced.
There’s all kinds of things I can be fooled about, and knowing that I can be fooled about those things should (and does) make me more difficult to convince of them. But if, even taking that increased skeptcism into account, I’m convinced anyway… well, what more is there to say? At that point I’ve been (somehow, implausibly) convinced, and should behave accordingly.
To say “Even if I’ve (somehow, implausibly) been exposed to a convincing event, I don’t update my beliefs” is simply another way of saying that no such convincing event can exist—of fighting the hypothetical.
Mind you, I agree that no such convincing event can exist, and that the hypothetical simply is not going to happen. But that’s precisely my point: if it does anyway, then I am clearly deeply confused about how the universe works; I should at that point sharply lower my confidence in all judgments even vaguely related to the nonexistence of GLUT Chinese Rooms, including “I can tell whether I’m talking to an intelligent system just by talking to them”.
The extent of my confidence in X ought to be proportional to the extent of my confusion if I come (somehow) to believe that X is false.
I think I see what you’re saying—discovering that something as unlikely as a GLUT actually exists would shake your beliefs in pretty much everything, including the Turing Test. This position makes sense, but I think it’s somewhat orthogonal to the current topic. Presumably, you’d feel the same way if you became convinced that gods exist, or that Pi has a finite number of digits after all, or something.
Not quite.
Discovering that something as unlikely as a conversation-having GLUT exists would shake my beliefs in everything related to conversation-having GLUTs. My confidence that I’m wearing socks right now would not decrease much, but my confidence that I can usefully infer attributes of a system by conversing with it would decrease enormously. Since Turing Tests are directly about the latter, my confidence about Turing Tests would also decrease enormously.
More generally, any event that causes me to sharply alter my confidence in a proposition P will also tend to alter my confidence in other propositions related to P, to an extent proportional to their relation.
An event which made me confident that pi was a terminating decimal after all, or that some religion’s account of its god(s) was accurate, etc. probably would not reduce my confidence in the Turing Test nearly as much, though it would reduce my confidence in other things more.
Why not ? Encountering a bona-fide GLUT that could pass the Turing test would be tantamount to a miracle. I personally would begin questioning everything if something like that were to happen. After all, socks are objects that I had previously thought of as “physical”, but the GLUT would shake the very notions of what a “physical” object even is.
Why that, and not your confidence about GLUTs ?
Of course my confidence about GLUTs would also decrease enormously in this scenario… sorry if that wasn’t clear.
More generally, my point here is that a conversation-having GLUT would not alter my confidence in all propositions equally, but rather would alter my confidence in propositions to a degree proportional to their relation to conversation-having GLUTs, and “I can usefully infer attributes of a system by conversing with it” (P1) is far more closely related to conversation-having GLUTs than “I’m wearing socks” (P2).
If your point is that my confidence in P2 should nevertheless be significant, even if much less than P1… well, maybe. Offhand, I’m not sure my brain is capable of spanning a broad enough span of orders-of-magnitude of confidence-shift to be able to consistently represent the updates of both P1 and P2, but I’m not confident either way.
I agree with you concerning Searle’s errors (see my takes on Searle at http://lesswrong.com/lw/ghj/searles_cobol_room/ http://lesswrong.com/lw/gyx/ai_prediction_case_study_3_searles_chinese_room/ )
I think the differences between us are rather small, in fact. I do have a different definition of thinking, which is not fully explicit. It would go along the lines of “a thinking machine should demonstrate human-like abilities in most situations and not be extremely stupid in some areas”. The intuition is that if there is a general intelligence, rather than simply a list of specific rules, then it’s competence shouldn’t completely collapse when facing unusual situations.
The “test systems on situations they’re not optimised” approach was trying to establish whether there would be such a collapse in skill. Of course you can’t test for every situation, but you can get a good idea this way.
This seems like a slightly uncharitable reading of Searle’s position.
Searle’s steadfast refusal to consider perfectly reasonable replies to his position, and his general recalcitrance in the debate on this and related questions, makes him unusually vulnerable to slightly uncharitable readings. The fact that his justification seems to be “human brains have unspecified magic that make humans conscious, and no I will not budge from that position because I have very strong intuitions” means, I think, that my reading is not even very uncharitable.
Oh, and on the subject of whole brain emulations: Greg Egan’s recent novel Zendegi (despite being, imo, rather poor overall), does make a somewhat convincing case that an emulation of a person’s brain/consciousness/personality might pass something like a Turing test and still not possess subjective consciousness or true general intelligence on a human level.
When this topic comes up I’m always reminded of a bit in John Varley’s Golden Globe, where our hero asks an advanced AI whether it’s actually conscious, and it replies “I’ve thought about that question a lot and have concluded that I’m probably not.”