at which point Searle declares that the stranger’s half of the entire conversation up to that point has been nothing but the meaningless blatherings of a mindless machine, devoid entirely of any true understanding.
It is perhaps worth noting that Searle explicitly posits in that essay that the system is functioning as a Giant Lookup Table.
If faced with an actual GLUT Chinese Room… well, honestly, I’m more inclined to believe that I’m being spoofed than trust the evidence of my senses.
But leaving that aside, if faced with something I somehow am convinced is a GLUT Chinese Room, I have to rethink my whole notion of how complicated conversation actually is, and yeah, I would probably conclude that the entire conversation up to that point has been devoid entirely of any true understanding. (I would also have to rethink my grounds for believing that humans have true understanding.)
Actually, Searle’s description of the thought experiment does include a “program”, a set of rules for manipulating the Chinese symbols provided to the room’s occupant. Searle also addresses a version of the contrary position (the pro-AI position, as it were) that posits a simulation of an actual brain (to which I alluded in the grandparent). He doesn’t think that would possess true understanding, either.
I think that if we’ve gotten to the point where we’re rethinking whether humans have true understanding, we should instead admit that we haven’t the first clue what “true understanding” is or what relation, if any, said mysterious property has to do with whatever we’re detecting in our test subjects.
Wouldn’t such a GLUT by necessity require someone possessing immensely fine understanding of Chinese and English both, though? You could then say that the person+GLUT system as a whole understands Chinese, as it combines both the person’s symbol-manipulation capabilities and the actual understanding represented by the GLUT.
You might still not possess understanding of Chinese, but that does not mean a meaningful conversation has not taken place.
I have no idea whether a GLUT-based Chinese Room would require someone possessing immensely fine understanding of Chinese and English both. As far as I can tell, a GLUT-based Chinese Room is impossible, and asking what is or isn’t required to bring about an impossible situation seems a silly question. Conversely, if it turns out that a GLUT-based Chinese Room is not impossible, I don’t trust my intuitions about what is or isn’t required to construct one.
I have no problem with saying a Chinese-speaking-person+GLUT system as a whole understands Chinese, in much the same sense that I have no problem saying that a Chinese-speaking-person+tuna-fish-sandwich system as a whole understands Chinese. I’m not sure how interesting that is.
I’m perfectly content to posit an artificial system capable of understanding Chinese and having a meaningful conversation. I’m unable to conceive specifically of a GLUT that can do so.
I’m perfectly content to posit an artificial system capable of understanding Chinese and having a meaningful conversation. I’m unable to conceive specifically of a GLUT that can do so.
I don’t think it’s that hard to conceive of. Imagine that the Simulation Argument is true; then, we could easily imagine a GLUT that exists outside of our own simulation, using additional resources; then our Chinese Room could just be an interface for such a GLUT.
As you said though, I don’t find the proposal very interesting, especially since I’m not a big fan of the Simulation Argument anyway.
I find I am unable, on brief consideration, to conceive of a GLUT sitting in some real world within which my observable universe is being computed… I have no sense of what such a thing might be like, or what its existence implies about the real world and how it differs from my observed simulation, or really much of anything interesting.
It’s possible that I might be able to if I thought about it for noticeably longer than I’m inclined to.
Not necessarily. Theoretically, one could have very specific knowledge of Chinese, possibly acquired from very limited but deep experience. Imagine one person who has spoken Chinese only at the harbor, and has complete and total mastery of the maritime vocabulary of Chinese but would lack all but the simplest verbs relevant to the conversations happening just a mile further inland. Conceivably, a series of experts in a very localized domain could separately contribute their understanding, perhaps governed by a person who understands (in English) every conceivable key to the GLUT, but does not understand the values which must be placed in it.
Then, imagine someone whose entire knowledge of Chinese is the translation of the phrase: “Does my reply make sense in the context of this conversation?” This person takes an arbitrary amount of time, randomly combining phonemes and carrying out every conceivable conversation with an unlimited supply of Chinese speakers. (This is substantially more realistic if there are many people working in a field with fewer potential combinations than language). Through perhaps the least efficient trial and error possible, they learn to carry on a conversation by rote, keeping only those conversational threads which, through pure chance, make sense throughout the entire dialogue.
In neither of these human experts do we find a real understanding of Chinese. It could be said that the understandings of the domain experts combine to form one great understanding, but the inefficient trial-and-error GLUT manufacturers certainly do not have any understanding, merely memory.
I agree on the basic point, but then my deeper point was that somewhere down the line you’ll find the intelligence(s) that created a high-fidelity converter for an arbitrary amount of information from one format to another. Sarle is free to claim that the system does not understand Chinese, but its very function could only have been imparted by parties who collectively speak Chinese very well, making the room at very least a medium of communication utilizing this understanding.
And this is before we mention the entirely plausible claim that the room-person system as a whole understands Chinese, even though neither of its two parts does. Any system you’ll take apart to sufficient degrees will stop displaying the properties of the whole, so having us peer inside an electronic brain asking “but where does the intelligence/understanding reside?” misses the point entirely.
Theoretically, one could have very specific knowledge of Chinese, possibly acquired from very limited but deep experience. Imagine one person who has spoken Chinese only at the harbor, and has complete and total mastery of the maritime vocabulary of Chinese but would lack all but the simplest verbs relevant to the conversations happening just a mile further inland. Conceivably, a series of experts in a very localized domain could separately contribute their understanding, perhaps governed by a person who understands (in English) every conceivable key to the GLUT, but does not understand the values which must be placed in it.
This does not pass the simplest plausibility test. Do you imagine that being at a harbor causes people to have only conversations which are uniquely applicable to harbor activities? Does one not need words and phrases for concepts like “person”, “weather”, “hello”, “food”, “where”, “friend”, “tomorrow”, “city”, “want”, etc., not to mention rules of Chinese grammar and syntax? Such a “harbor-only” Chinese speaker may lack certain specific vocabulary, but he certainly will not lack a general understanding of Chinese.
Your other example is even sillier, especially given that the number of possible conversations in a human language is infinite. For one thing, a conversation where one person is constantly asking “Does my reply make sense?” is very, very different from the “same” conversation without such constant verbal monitoring. (Not to mention the specific fact that your imaginary expert would not be able to understand his interlocutor’s response to his question about whether his utterances made sense.)
A more realistic version would be for for an observer to record all conversations between two Chinese speakers with length N, where N is some arbitrarily large but still finite conversation length. (If a GLUT were to capture every possible conversation, you are correct in saying that it would have to be infinite).
From a sufficiently large sample size (though it is implausible to capture every probable conversation in any realistic amount of time, not to mention in any amount of time during which the language is relatively stable and unchanging), a tree of conversations could be built, with an arbitrarily large probability of including a given conversation within it.
From this, one could built a GLUT (though it would probably be more efficient as a tree) of the possible questions given context and the appropriate responses. Though it would be utterly unfeasible to build, that is a limitation of the availability of data, rather than the GLUT structure itself. It would not be perfect—one cannot build an infinite GLUT, nor can one acquire the infinite amount of data with which to fill it—but it could, perhaps, surpass even a native speaker by some measures.
Consider: what would the table contain as appropriate responses for the following questions? (Each question would certainly appear many, many times in our record of all conversations up to length N.)
“Hello, what is your name?”
“Where do you live?”
“What do you look like?”
“Tell me about your favorite television show.”
Remember that a GLUT, by definition, matches each input to one output. If you have to algorithmically consider context, whether environmental (what year is it? where are we?), personal (who am I?), or conversation history (what’s been said up to this point?), then that is not a GLUT, it is a program. You can of course convert any program that deterministically gives output for given input into a GLUT, but to do that successfully, you really do need all possible inputs and their outputs; and “input” here means “question, plus conversation history, plus complete description of world-state” (complete because we don’t know what context we’ll need in order to give an appropriate response).
In other words, to construct such a GLUT, you would have to be well-nigh omniscient. But, admittedly, you would not then have to “know” any Chinese.
But leaving that aside, if faced with something I somehow am convinced is a GLUT Chinese Room … I would probably conclude that the entire conversation up to that point has been devoid entirely of any true understanding.
I wouldn’t; or, at least, not necessarily.
Just for simplicity, let’s say we the agent and the experimenter are conversing via text-based chat. The agent and the experimenter take turns outputting a single line of text (it could be a very long line); the experimenter always goes first, saying “hello”.
In this case, the agent’s side of the conversation can be modeled as the function F(H(t), t) where t is the current line number, and H(t) is the sequence of inputs that the agent received up to point t. Thus, H(0) is always {”hello”}, as per above. H(2) might be something like {”hello”, “your name sounds funny”}, etc.
We know that, since F is a function, it is a relation that maps each possible input to a single output—so it’s basically a lookup table. In the ideal case, the number of possible inputs is infinite (trivially, we could say “hello”, “helloo”, “hellooo”, and so on, and infinitum), and thus the lookup table would need to be infinitely large. However, we humans are finite creatures, and thus in practice the lookup table would only need to be finitely large.
Of course, practically speaking, you’d probably still need a storage device larger than the Universe to encode even a finite lookup table of sufficient size; but this is a practical objection, which does not a priori prohibit us from implementing a Turing-grade agent as a GLUT.
You’re correct that it doesn’t a priori prohibit such a thing. It does, however, bring my prior probability of encountering such a thing vanishingly low. Faced with an event that somehow causes me to update that vanishingly small probability to the point of convincing me it’s true I am vastly surprised, and that vast surprise colors all of my intuitions about interactions with nominally intelligent systems. Given that, it’s not clear to me why I should keep believing that I was having an intelligent conversation a moment earlier.
...and that vast surprise colors all of my intuitions about interactions with nominally intelligent systems. Given that, it’s not clear to me why I should keep believing that I was having an intelligent conversation a moment earlier.
What do you mean by “intelligent conversation” ? Do you mean, “a conversation with an intelligent agent”, or “a conversation whose contents satisfy certain criteria”, and if so, which ones ? I’ll assume you mean the former for now.
Let’s say that you had a text-only chat with the agent, and found it intellectually stimulating. You thought that the agent was responding quite cleverly to your comments, had a distinct “writer’s voice”, etc.
Now, let’s imagine two separate worlds. In world A, you learned that the agent was in fact a GLUT. Surprised and confused, you confronted it in conversation, and it responded to your comments as it did before, with apparent intelligence and wit. But, of course, now you knew better than to fall for such obvious attempts to fool you.
In world B, the exact same thing happened, and the rest of your conversation proceeded as before, with one minor difference: unbeknownst to you, the person who told you that the agent was a GLUT was himself a troll. He totally gaslighted you. The agent isn’t a GLUT or even an AI; it’s just a regular human (and the troll’s accomplice), a la the good old Mechanical Turk.
It sounds to me like if you were in world B, you’d still disbelieve that you were having a conversation with an intelligent agent. But in world B, you’d be wrong. In world A, you would of course be right.
Is there any way for you to tell which world you’re in (I mean, without waterboarding that pesky troll or taking apart the Chinese Room to see who’s inside, etc.) ? If there is no way for you to tell the difference, then what’s the difference ?
By the way, I do agree with you that, in our real world, the probability of such a GLUT existing is pretty much zero. I am merely questioning the direction of your (hypothetical) belief update.
By construction, there’s no way for me to tell… that is, I’ve already posited that some event (somehow, implausibly) convinced me my interlocutor is a GLUT.
In world A, “I” was correct to be convinced; my interlocutor really was (somehow, implausibly) a GLUT, impossible as that seems. In world B, “I” was (somehow, implausibly) incorrectly convinced.
There’s all kinds of things I can be fooled about, and knowing that I can be fooled about those things should (and does) make me more difficult to convince of them. But if, even taking that increased skeptcism into account, I’m convinced anyway… well, what more is there to say? At that point I’ve been (somehow, implausibly) convinced, and should behave accordingly.
To say “Even if I’ve (somehow, implausibly) been exposed to a convincing event, I don’t update my beliefs” is simply another way of saying that no such convincing event can exist—of fighting the hypothetical.
Mind you, I agree that no such convincing event can exist, and that the hypothetical simply is not going to happen. But that’s precisely my point: if it does anyway, then I am clearly deeply confused about how the universe works; I should at that point sharply lower my confidence in all judgments even vaguely related to the nonexistence of GLUT Chinese Rooms, including “I can tell whether I’m talking to an intelligent system just by talking to them”.
The extent of my confidence in X ought to be proportional to the extent of my confusion if I come (somehow) to believe that X is false.
I think I see what you’re saying—discovering that something as unlikely as a GLUT actually exists would shake your beliefs in pretty much everything, including the Turing Test. This position makes sense, but I think it’s somewhat orthogonal to the current topic. Presumably, you’d feel the same way if you became convinced that gods exist, or that Pi has a finite number of digits after all, or something.
discovering that something as unlikely as a GLUT actually exists would shake your beliefs in pretty much everything, including the Turing Test
Not quite.
Discovering that something as unlikely as a conversation-having GLUT exists would shake my beliefs in everything related to conversation-having GLUTs. My confidence that I’m wearing socks right now would not decrease much, but my confidence that I can usefully infer attributes of a system by conversing with it would decrease enormously. Since Turing Tests are directly about the latter, my confidence about Turing Tests would also decrease enormously.
More generally, any event that causes me to sharply alter my confidence in a proposition P will also tend to alter my confidence in other propositions related to P, to an extent proportional to their relation.
An event which made me confident that pi was a terminating decimal after all, or that some religion’s account of its god(s) was accurate, etc. probably would not reduce my confidence in the Turing Test nearly as much, though it would reduce my confidence in other things more.
My confidence that I’m wearing socks right now would not decrease much...
Why not ? Encountering a bona-fide GLUT that could pass the Turing test would be tantamount to a miracle. I personally would begin questioning everything if something like that were to happen. After all, socks are objects that I had previously thought of as “physical”, but the GLUT would shake the very notions of what a “physical” object even is.
Since Turing Tests are directly about the latter, my confidence about Turing Tests would also decrease enormously.
my confidence about Turing Tests would also decrease enormously. Why that, and not your confidence about GLUTs ?
Of course my confidence about GLUTs would also decrease enormously in this scenario… sorry if that wasn’t clear.
More generally, my point here is that a conversation-having GLUT would not alter my confidence in all propositions equally, but rather would alter my confidence in propositions to a degree proportional to their relation to conversation-having GLUTs, and “I can usefully infer attributes of a system by conversing with it” (P1) is far more closely related to conversation-having GLUTs than “I’m wearing socks” (P2).
If your point is that my confidence in P2 should nevertheless be significant, even if much less than P1… well, maybe. Offhand, I’m not sure my brain is capable of spanning a broad enough span of orders-of-magnitude of confidence-shift to be able to consistently represent the updates of both P1 and P2, but I’m not confident either way.
It is perhaps worth noting that Searle explicitly posits in that essay that the system is functioning as a Giant Lookup Table.
If faced with an actual GLUT Chinese Room… well, honestly, I’m more inclined to believe that I’m being spoofed than trust the evidence of my senses.
But leaving that aside, if faced with something I somehow am convinced is a GLUT Chinese Room, I have to rethink my whole notion of how complicated conversation actually is, and yeah, I would probably conclude that the entire conversation up to that point has been devoid entirely of any true understanding. (I would also have to rethink my grounds for believing that humans have true understanding.)
I don’t expect that to happen, though.
Actually, Searle’s description of the thought experiment does include a “program”, a set of rules for manipulating the Chinese symbols provided to the room’s occupant. Searle also addresses a version of the contrary position (the pro-AI position, as it were) that posits a simulation of an actual brain (to which I alluded in the grandparent). He doesn’t think that would possess true understanding, either.
I think that if we’ve gotten to the point where we’re rethinking whether humans have true understanding, we should instead admit that we haven’t the first clue what “true understanding” is or what relation, if any, said mysterious property has to do with whatever we’re detecting in our test subjects.
Oh, and: GAZP vs. GLUT.
Wouldn’t such a GLUT by necessity require someone possessing immensely fine understanding of Chinese and English both, though? You could then say that the person+GLUT system as a whole understands Chinese, as it combines both the person’s symbol-manipulation capabilities and the actual understanding represented by the GLUT.
You might still not possess understanding of Chinese, but that does not mean a meaningful conversation has not taken place.
I have no idea whether a GLUT-based Chinese Room would require someone possessing immensely fine understanding of Chinese and English both. As far as I can tell, a GLUT-based Chinese Room is impossible, and asking what is or isn’t required to bring about an impossible situation seems a silly question. Conversely, if it turns out that a GLUT-based Chinese Room is not impossible, I don’t trust my intuitions about what is or isn’t required to construct one.
I have no problem with saying a Chinese-speaking-person+GLUT system as a whole understands Chinese, in much the same sense that I have no problem saying that a Chinese-speaking-person+tuna-fish-sandwich system as a whole understands Chinese. I’m not sure how interesting that is.
I’m perfectly content to posit an artificial system capable of understanding Chinese and having a meaningful conversation. I’m unable to conceive specifically of a GLUT that can do so.
I don’t think it’s that hard to conceive of. Imagine that the Simulation Argument is true; then, we could easily imagine a GLUT that exists outside of our own simulation, using additional resources; then our Chinese Room could just be an interface for such a GLUT.
As you said though, I don’t find the proposal very interesting, especially since I’m not a big fan of the Simulation Argument anyway.
I find I am unable, on brief consideration, to conceive of a GLUT sitting in some real world within which my observable universe is being computed… I have no sense of what such a thing might be like, or what its existence implies about the real world and how it differs from my observed simulation, or really much of anything interesting.
It’s possible that I might be able to if I thought about it for noticeably longer than I’m inclined to.
If you can do so easily, good for you.
Not necessarily. Theoretically, one could have very specific knowledge of Chinese, possibly acquired from very limited but deep experience. Imagine one person who has spoken Chinese only at the harbor, and has complete and total mastery of the maritime vocabulary of Chinese but would lack all but the simplest verbs relevant to the conversations happening just a mile further inland. Conceivably, a series of experts in a very localized domain could separately contribute their understanding, perhaps governed by a person who understands (in English) every conceivable key to the GLUT, but does not understand the values which must be placed in it.
Then, imagine someone whose entire knowledge of Chinese is the translation of the phrase: “Does my reply make sense in the context of this conversation?” This person takes an arbitrary amount of time, randomly combining phonemes and carrying out every conceivable conversation with an unlimited supply of Chinese speakers. (This is substantially more realistic if there are many people working in a field with fewer potential combinations than language). Through perhaps the least efficient trial and error possible, they learn to carry on a conversation by rote, keeping only those conversational threads which, through pure chance, make sense throughout the entire dialogue.
In neither of these human experts do we find a real understanding of Chinese. It could be said that the understandings of the domain experts combine to form one great understanding, but the inefficient trial-and-error GLUT manufacturers certainly do not have any understanding, merely memory.
I agree on the basic point, but then my deeper point was that somewhere down the line you’ll find the intelligence(s) that created a high-fidelity converter for an arbitrary amount of information from one format to another. Sarle is free to claim that the system does not understand Chinese, but its very function could only have been imparted by parties who collectively speak Chinese very well, making the room at very least a medium of communication utilizing this understanding.
And this is before we mention the entirely plausible claim that the room-person system as a whole understands Chinese, even though neither of its two parts does. Any system you’ll take apart to sufficient degrees will stop displaying the properties of the whole, so having us peer inside an electronic brain asking “but where does the intelligence/understanding reside?” misses the point entirely.
This does not pass the simplest plausibility test. Do you imagine that being at a harbor causes people to have only conversations which are uniquely applicable to harbor activities? Does one not need words and phrases for concepts like “person”, “weather”, “hello”, “food”, “where”, “friend”, “tomorrow”, “city”, “want”, etc., not to mention rules of Chinese grammar and syntax? Such a “harbor-only” Chinese speaker may lack certain specific vocabulary, but he certainly will not lack a general understanding of Chinese.
Your other example is even sillier, especially given that the number of possible conversations in a human language is infinite. For one thing, a conversation where one person is constantly asking “Does my reply make sense?” is very, very different from the “same” conversation without such constant verbal monitoring. (Not to mention the specific fact that your imaginary expert would not be able to understand his interlocutor’s response to his question about whether his utterances made sense.)
You make some valid points.
A more realistic version would be for for an observer to record all conversations between two Chinese speakers with length N, where N is some arbitrarily large but still finite conversation length. (If a GLUT were to capture every possible conversation, you are correct in saying that it would have to be infinite).
From a sufficiently large sample size (though it is implausible to capture every probable conversation in any realistic amount of time, not to mention in any amount of time during which the language is relatively stable and unchanging), a tree of conversations could be built, with an arbitrarily large probability of including a given conversation within it.
From this, one could built a GLUT (though it would probably be more efficient as a tree) of the possible questions given context and the appropriate responses. Though it would be utterly unfeasible to build, that is a limitation of the availability of data, rather than the GLUT structure itself. It would not be perfect—one cannot build an infinite GLUT, nor can one acquire the infinite amount of data with which to fill it—but it could, perhaps, surpass even a native speaker by some measures.
I remain dubious.
Consider: what would the table contain as appropriate responses for the following questions? (Each question would certainly appear many, many times in our record of all conversations up to length N.)
“Hello, what is your name?”
“Where do you live?”
“What do you look like?”
“Tell me about your favorite television show.”
Remember that a GLUT, by definition, matches each input to one output. If you have to algorithmically consider context, whether environmental (what year is it? where are we?), personal (who am I?), or conversation history (what’s been said up to this point?), then that is not a GLUT, it is a program. You can of course convert any program that deterministically gives output for given input into a GLUT, but to do that successfully, you really do need all possible inputs and their outputs; and “input” here means “question, plus conversation history, plus complete description of world-state” (complete because we don’t know what context we’ll need in order to give an appropriate response).
In other words, to construct such a GLUT, you would have to be well-nigh omniscient. But, admittedly, you would not then have to “know” any Chinese.
I wouldn’t; or, at least, not necessarily.
Just for simplicity, let’s say we the agent and the experimenter are conversing via text-based chat. The agent and the experimenter take turns outputting a single line of text (it could be a very long line); the experimenter always goes first, saying “hello”.
In this case, the agent’s side of the conversation can be modeled as the function F(H(t), t) where t is the current line number, and H(t) is the sequence of inputs that the agent received up to point t. Thus, H(0) is always {”hello”}, as per above. H(2) might be something like {”hello”, “your name sounds funny”}, etc.
We know that, since F is a function, it is a relation that maps each possible input to a single output—so it’s basically a lookup table. In the ideal case, the number of possible inputs is infinite (trivially, we could say “hello”, “helloo”, “hellooo”, and so on, and infinitum), and thus the lookup table would need to be infinitely large. However, we humans are finite creatures, and thus in practice the lookup table would only need to be finitely large.
Of course, practically speaking, you’d probably still need a storage device larger than the Universe to encode even a finite lookup table of sufficient size; but this is a practical objection, which does not a priori prohibit us from implementing a Turing-grade agent as a GLUT.
You’re correct that it doesn’t a priori prohibit such a thing. It does, however, bring my prior probability of encountering such a thing vanishingly low. Faced with an event that somehow causes me to update that vanishingly small probability to the point of convincing me it’s true I am vastly surprised, and that vast surprise colors all of my intuitions about interactions with nominally intelligent systems. Given that, it’s not clear to me why I should keep believing that I was having an intelligent conversation a moment earlier.
What do you mean by “intelligent conversation” ? Do you mean, “a conversation with an intelligent agent”, or “a conversation whose contents satisfy certain criteria”, and if so, which ones ? I’ll assume you mean the former for now.
Let’s say that you had a text-only chat with the agent, and found it intellectually stimulating. You thought that the agent was responding quite cleverly to your comments, had a distinct “writer’s voice”, etc.
Now, let’s imagine two separate worlds. In world A, you learned that the agent was in fact a GLUT. Surprised and confused, you confronted it in conversation, and it responded to your comments as it did before, with apparent intelligence and wit. But, of course, now you knew better than to fall for such obvious attempts to fool you.
In world B, the exact same thing happened, and the rest of your conversation proceeded as before, with one minor difference: unbeknownst to you, the person who told you that the agent was a GLUT was himself a troll. He totally gaslighted you. The agent isn’t a GLUT or even an AI; it’s just a regular human (and the troll’s accomplice), a la the good old Mechanical Turk.
It sounds to me like if you were in world B, you’d still disbelieve that you were having a conversation with an intelligent agent. But in world B, you’d be wrong. In world A, you would of course be right.
Is there any way for you to tell which world you’re in (I mean, without waterboarding that pesky troll or taking apart the Chinese Room to see who’s inside, etc.) ? If there is no way for you to tell the difference, then what’s the difference ?
By the way, I do agree with you that, in our real world, the probability of such a GLUT existing is pretty much zero. I am merely questioning the direction of your (hypothetical) belief update.
By construction, there’s no way for me to tell… that is, I’ve already posited that some event (somehow, implausibly) convinced me my interlocutor is a GLUT.
In world A, “I” was correct to be convinced; my interlocutor really was (somehow, implausibly) a GLUT, impossible as that seems. In world B, “I” was (somehow, implausibly) incorrectly convinced.
There’s all kinds of things I can be fooled about, and knowing that I can be fooled about those things should (and does) make me more difficult to convince of them. But if, even taking that increased skeptcism into account, I’m convinced anyway… well, what more is there to say? At that point I’ve been (somehow, implausibly) convinced, and should behave accordingly.
To say “Even if I’ve (somehow, implausibly) been exposed to a convincing event, I don’t update my beliefs” is simply another way of saying that no such convincing event can exist—of fighting the hypothetical.
Mind you, I agree that no such convincing event can exist, and that the hypothetical simply is not going to happen. But that’s precisely my point: if it does anyway, then I am clearly deeply confused about how the universe works; I should at that point sharply lower my confidence in all judgments even vaguely related to the nonexistence of GLUT Chinese Rooms, including “I can tell whether I’m talking to an intelligent system just by talking to them”.
The extent of my confidence in X ought to be proportional to the extent of my confusion if I come (somehow) to believe that X is false.
I think I see what you’re saying—discovering that something as unlikely as a GLUT actually exists would shake your beliefs in pretty much everything, including the Turing Test. This position makes sense, but I think it’s somewhat orthogonal to the current topic. Presumably, you’d feel the same way if you became convinced that gods exist, or that Pi has a finite number of digits after all, or something.
Not quite.
Discovering that something as unlikely as a conversation-having GLUT exists would shake my beliefs in everything related to conversation-having GLUTs. My confidence that I’m wearing socks right now would not decrease much, but my confidence that I can usefully infer attributes of a system by conversing with it would decrease enormously. Since Turing Tests are directly about the latter, my confidence about Turing Tests would also decrease enormously.
More generally, any event that causes me to sharply alter my confidence in a proposition P will also tend to alter my confidence in other propositions related to P, to an extent proportional to their relation.
An event which made me confident that pi was a terminating decimal after all, or that some religion’s account of its god(s) was accurate, etc. probably would not reduce my confidence in the Turing Test nearly as much, though it would reduce my confidence in other things more.
Why not ? Encountering a bona-fide GLUT that could pass the Turing test would be tantamount to a miracle. I personally would begin questioning everything if something like that were to happen. After all, socks are objects that I had previously thought of as “physical”, but the GLUT would shake the very notions of what a “physical” object even is.
Why that, and not your confidence about GLUTs ?
Of course my confidence about GLUTs would also decrease enormously in this scenario… sorry if that wasn’t clear.
More generally, my point here is that a conversation-having GLUT would not alter my confidence in all propositions equally, but rather would alter my confidence in propositions to a degree proportional to their relation to conversation-having GLUTs, and “I can usefully infer attributes of a system by conversing with it” (P1) is far more closely related to conversation-having GLUTs than “I’m wearing socks” (P2).
If your point is that my confidence in P2 should nevertheless be significant, even if much less than P1… well, maybe. Offhand, I’m not sure my brain is capable of spanning a broad enough span of orders-of-magnitude of confidence-shift to be able to consistently represent the updates of both P1 and P2, but I’m not confident either way.