It sounds like you’re pointing out that people often overestimate the difficulty of passing the Turing Test. Is that what you mean to say?
Yes. I think the Turing Test is useful, but that there are too many quite distinct tests mapping to “Turing Test”, and details matter. College students as volunteers will lead to markedly different results than a randomly iid drawn human from anywhere on Earth.
As is the case so often, many disagreements I’ve seen boil down to (usually unrecognized) definitional squabbles. Without clarification, the statement “A Turing Test is a reasonable test for intelligence” just isn’t well defined enough. Which Turing Test? Reasonable in terms of optimality, in terms of feasability, or in what way? Intelligence in some LW “optimizing power above certain threshold” sense (if so, what threshold?), or some other notion?
You thankfully narrowed it down to the specific Turing version I mentioned, but in truth I don’t have only one concept of intelligence I find useful, in the sense of that I can see various concepts of intelligence being useful in different contexts. I pay no special homage to “intelligence1” over “intelligence2″. Concerning this discussion:
I think that human-level intelligence—and the Turing Test is invariably centered on humans as the benchmark—shouldn’t be defined by educated gifted people, but by an average. An “average human” Turing Test being passed is surely interesting, not least from a historical perspective. However, it’s not clear whether such an algorithm would be powerful enough to foom, or to do that many theoretically interesting tasks. Many less privileged humans can’t do that many interesting tasks better than machines, apart from recognizing tanks and cats on pictures.
So should we focus on a Turing Test tuned to an AGI on par with fooling the best researchers into believing it to be a fellow AI researcher? Maybe, although if we had a “winner”, we’d probably know just by looking out the window, before we even set up the test (or we’d know by the AI looking in …).
All considered, I’d focus on a Turing Test which can fool average humans in the civilized world, which seems to be the lowest Turing Test level at which such a chatbot would have a transformative influence on social human interactions.
Don’t forget that the goal in the Turing Test is not to appear intelligent, but to appear human. If an interrogator asks “what question would you ask in the Turing test?”, and the answer is “uh, I don’t know”, then that is perfectly consistent with the responder being human. A smart interrogator won’t jump to a conclusion.
It sounds like you’re pointing out that people often overestimate the difficulty of passing the Turing Test. Is that what you mean to say?
Do you think the Turing Test (at the difficulty level you describe) is a reasonable test of intelligence?
Yes. I think the Turing Test is useful, but that there are too many quite distinct tests mapping to “Turing Test”, and details matter. College students as volunteers will lead to markedly different results than a randomly iid drawn human from anywhere on Earth.
As is the case so often, many disagreements I’ve seen boil down to (usually unrecognized) definitional squabbles. Without clarification, the statement “A Turing Test is a reasonable test for intelligence” just isn’t well defined enough. Which Turing Test? Reasonable in terms of optimality, in terms of feasability, or in what way? Intelligence in some LW “optimizing power above certain threshold” sense (if so, what threshold?), or some other notion?
You thankfully narrowed it down to the specific Turing version I mentioned, but in truth I don’t have only one concept of intelligence I find useful, in the sense of that I can see various concepts of intelligence being useful in different contexts. I pay no special homage to “intelligence1” over “intelligence2″. Concerning this discussion:
I think that human-level intelligence—and the Turing Test is invariably centered on humans as the benchmark—shouldn’t be defined by educated gifted people, but by an average. An “average human” Turing Test being passed is surely interesting, not least from a historical perspective. However, it’s not clear whether such an algorithm would be powerful enough to foom, or to do that many theoretically interesting tasks. Many less privileged humans can’t do that many interesting tasks better than machines, apart from recognizing tanks and cats on pictures.
So should we focus on a Turing Test tuned to an AGI on par with fooling the best researchers into believing it to be a fellow AI researcher? Maybe, although if we had a “winner”, we’d probably know just by looking out the window, before we even set up the test (or we’d know by the AI looking in …).
All considered, I’d focus on a Turing Test which can fool average humans in the civilized world, which seems to be the lowest Turing Test level at which such a chatbot would have a transformative influence on social human interactions.
Don’t forget that the goal in the Turing Test is not to appear intelligent, but to appear human. If an interrogator asks “what question would you ask in the Turing test?”, and the answer is “uh, I don’t know”, then that is perfectly consistent with the responder being human. A smart interrogator won’t jump to a conclusion.
Thank you, that was a fantastic answer to my questions (and more)!