We want a test to tell us when an AI is intelligent and powerful. But we’ll know a powerful AI even without a test, because we’ll see it using its power and achieving things that really matter [...] narrow AIs [can be made that are] useless for anything other than passing the test. A new test might repeat the same story. Or it might turn out to be too hard and only be achieved long after many other AI capabilities that would greatly change the world.
I think we’ll see (arguably have already seen) AI changing the world before we see a general AI passing the Turing test. But I don’t think that makes the Turing test useless, or a red herring.
Narrow AI is plenty powerful. It drives cars, flies military drones, runs short-term trading systems, and plays chess, and does (or will shortly do) them all better than the best humans in their domains. Right now that hasn’t dramatically changed the world, but I don’t think it’s too much of a stretch to imagine a world that has been transformed by narrow AI applications.
But there are still things the Turing test or a successor would be useful for. For one thing, as AI techniques advance, I expect the line between narrow and general AI to blur. I can’t rule out purpose-built AGI before this becomes significant, but if that doesn’t make the problem completely irrelevant, then the Turing test serves as a pretty good marker of generalizability: if your trading system (that scrapes Reuters for headlines and does some sophisticated NLP and concept-mapping stuff that you’re pretty proud of) starts asking you hilariously bizarre questions about business ethics, you’re probably well on your way to dealing with something that can no longer be described as narrow AI. If it starts asking you good questions about business ethics… well, you’re probably very lucky.
Less significantly from an AGI perspective, but still interestingly, there’s a bunch of semi-narrow AI applications that focus tightly on interaction with humans. Siri, Google Now, and Cortana are probably the most salient examples right now, along with all those godawful customer-service phone systems; we could also imagine things like automated hotel concierges or caretakers for the elderly. The Turing test is an excellent benchmark for their performance; I no longer think we can take a pass as evidence of strong general intelligence, but humanlike responses are so useful in these roles that I still think it’s a good thing to shoot for. A successor test in this role gives us a less gameable objective.
the Turing test serves as a pretty good marker of generalizability
That argues any sufficiently general system could pass the Turing test. But maybe it’s really impossible to pass the test without investing a lot of ‘narrow’ resources in that specific goal. Even if an AGI could self-modify to pass for human, it would not bother unless that were an instrumental goal (i.e. to trick humans), at which point it’s probably too late for you from a FAI viewpoint.
We should be able to recognize a powerful, smart, general intelligence without requiring that it be good at pretending to be a complete different kind of powerful, smart, general intelligence that has a lot of social quirks and cues.
The Turing test is an excellent benchmark for their performance; I no longer think we can take a pass as evidence of strong general intelligence, but humanlike responses are so useful in these roles that I still think it’s a good thing to shoot for.
Again, I don’t think the Turing test is necessary in this example. Siri can fulfill every objective of its designers without being able to trick humans who really want to know if it’s an AI or not. A robotic hotel concierge wants to make guests comfortable and serve their needs; there is no reason that should involve tricking them.
I think we’ll see (arguably have already seen) AI changing the world before we see a general AI passing the Turing test. But I don’t think that makes the Turing test useless, or a red herring.
Narrow AI is plenty powerful. It drives cars, flies military drones, runs short-term trading systems, and plays chess, and does (or will shortly do) them all better than the best humans in their domains. Right now that hasn’t dramatically changed the world, but I don’t think it’s too much of a stretch to imagine a world that has been transformed by narrow AI applications.
But there are still things the Turing test or a successor would be useful for. For one thing, as AI techniques advance, I expect the line between narrow and general AI to blur. I can’t rule out purpose-built AGI before this becomes significant, but if that doesn’t make the problem completely irrelevant, then the Turing test serves as a pretty good marker of generalizability: if your trading system (that scrapes Reuters for headlines and does some sophisticated NLP and concept-mapping stuff that you’re pretty proud of) starts asking you hilariously bizarre questions about business ethics, you’re probably well on your way to dealing with something that can no longer be described as narrow AI. If it starts asking you good questions about business ethics… well, you’re probably very lucky.
Less significantly from an AGI perspective, but still interestingly, there’s a bunch of semi-narrow AI applications that focus tightly on interaction with humans. Siri, Google Now, and Cortana are probably the most salient examples right now, along with all those godawful customer-service phone systems; we could also imagine things like automated hotel concierges or caretakers for the elderly. The Turing test is an excellent benchmark for their performance; I no longer think we can take a pass as evidence of strong general intelligence, but humanlike responses are so useful in these roles that I still think it’s a good thing to shoot for. A successor test in this role gives us a less gameable objective.
That argues any sufficiently general system could pass the Turing test. But maybe it’s really impossible to pass the test without investing a lot of ‘narrow’ resources in that specific goal. Even if an AGI could self-modify to pass for human, it would not bother unless that were an instrumental goal (i.e. to trick humans), at which point it’s probably too late for you from a FAI viewpoint.
We should be able to recognize a powerful, smart, general intelligence without requiring that it be good at pretending to be a complete different kind of powerful, smart, general intelligence that has a lot of social quirks and cues.
Again, I don’t think the Turing test is necessary in this example. Siri can fulfill every objective of its designers without being able to trick humans who really want to know if it’s an AI or not. A robotic hotel concierge wants to make guests comfortable and serve their needs; there is no reason that should involve tricking them.