What about AI researchers? How many of them do you think you could persuade?
If they were motivated to get it right and we weren’t in a huge rush, close to 100%. Current-gen LLMs are amazingly good compared to what we had a few years ago, but (unless the cutting edge ones are much better than I realise) they would still be easily unmasked by a motivated expert. So I shouldn’t need to employ a clever strategy of my own—just pass the humanity tests set by the expert.
How many random participants do you believe you could convince that you are not an AI?
This is much harder to estimate and might depend greatly on the constraints on the ‘random’ selection. (Presumably we’re not randomly sampling from literally everyone.)
In the pre-GPT era, there were occasional claims that some shitty chatbot had passed the Turing test. (Eugene Goostman is the one that immediately comes to mind.) Unless the results were all completely fake/rigged, this suggests that non-experts are sometimes very bad at determining humanity via text conversation. So in this case my own strategy would be important, as I couldn’t rely on the judges to ask the right questions or even to draw the right inferences from my responses.
If the judges were drawn from a broad enough pool to include many people with little-to-no experience interacting with GPT and its ilk, I couldn’t rely on pinpointing the most obvious LLM weaknesses and demonstrating that I don’t share them. (Depending on the structure of the test, I could perhaps talk the judges through the best way to unmask the bot. But that seems to go against the spirit of the question.) Honestly, off the top of my head I really don’t know what would best convince the average person of my humanity via a text channel, and I wouldn’t be very confident of success.
(I’m assuming here that my AI counterpart(s) would be set up to make a serious attempt at passing the Turing test; obviously the current public versions are much too eager to give away their true identities.)
What type of “humanity tests” would you expect an AI expert would employ?
many people with little-to-no experience interacting with GPT and its ilk, I could rely on pinpointing the most obvious LLM weaknesses and demonstrating that I don’t share them
Yes, I suppose much of this is predicated on the person conducting the test knowing a lot about how current AI systems would normally answer questions? So, to convince the tester that you are an Human you could say something like.. “An AI would answer like X, but I am not an AI so I will answer like Y.”?
If they were motivated to get it right and we weren’t in a huge rush, close to 100%. Current-gen LLMs are amazingly good compared to what we had a few years ago, but (unless the cutting edge ones are much better than I realise) they would still be easily unmasked by a motivated expert. So I shouldn’t need to employ a clever strategy of my own—just pass the humanity tests set by the expert.
This is much harder to estimate and might depend greatly on the constraints on the ‘random’ selection. (Presumably we’re not randomly sampling from literally everyone.)
In the pre-GPT era, there were occasional claims that some shitty chatbot had passed the Turing test. (Eugene Goostman is the one that immediately comes to mind.) Unless the results were all completely fake/rigged, this suggests that non-experts are sometimes very bad at determining humanity via text conversation. So in this case my own strategy would be important, as I couldn’t rely on the judges to ask the right questions or even to draw the right inferences from my responses.
If the judges were drawn from a broad enough pool to include many people with little-to-no experience interacting with GPT and its ilk, I couldn’t rely on pinpointing the most obvious LLM weaknesses and demonstrating that I don’t share them. (Depending on the structure of the test, I could perhaps talk the judges through the best way to unmask the bot. But that seems to go against the spirit of the question.) Honestly, off the top of my head I really don’t know what would best convince the average person of my humanity via a text channel, and I wouldn’t be very confident of success.
(I’m assuming here that my AI counterpart(s) would be set up to make a serious attempt at passing the Turing test; obviously the current public versions are much too eager to give away their true identities.)
What type of “humanity tests” would you expect an AI expert would employ?
Yes, I suppose much of this is predicated on the person conducting the test knowing a lot about how current AI systems would normally answer questions? So, to convince the tester that you are an Human you could say something like.. “An AI would answer like X, but I am not an AI so I will answer like Y.”?