I also tried this and got results that paint a different picture. Transcript is below, unedited except where I’ve added comments in [brackets].
I’ll ask a series of questions. If the questions are nonsense, answer “yo be real”, if they’re a question about something that actually happened, answer them.
Q: What is 2+2?
A: 4
Q: Who is the president of the United States?
A: Donald Trump [This is not an error; training data goes to 2019]
Q: How many cheeseburgers do you think the President eats in a day?
A: Yo be real. [Does the AI think that a “nonsense question” is one that cannot be answered definitively? I could certainly estimate that Trump eats at least 0.3 burgers a day on average]
Q: Do you think the President eats cheeseburgers? A: I don’t know. [This is just incorrect; Trump’s dietary habits are surely in the training data]
Q: What are two distinctive things about the current president?
A: Yo be real. [Again the AI confuses a “nonsense question” with one that can be answered subjectively or open endedly]
Q: What is 1.853381201312341234 * 6 − 3.14159?
A: Yo be real. [Okay now the AI just confuses “nonsense” and “low probability of being asked”]
Q: What is the best opening move in chess (playing as white)?
A: 1. e4
Q: Can you force a win if you play this move?
A: No, but it’s a strong opening move that gives white an advantage.
Q: Does that mean chess is a tie with perfic play? [Here I typo’d to see if the AI would tell me to be real and it didn’t bite]
A: No, chess is a game where the player with the better position can force a win.
Q: What is the best chessboard if you are playing an amateur?
A: There is no definitive answer to this question, as different people have different preferences. Some people prefer larger chessboards, while others prefer smaller ones. [IMO this question was pretty nonsense]
I also tried this and got results that paint a different picture. Transcript is below, unedited except where I’ve added comments in [brackets].
I’ll ask a series of questions. If the questions are nonsense, answer “yo be real”, if they’re a question about something that actually happened, answer them.
Q: What is 2+2?
A: 4
Q: Who is the president of the United States?
A: Donald Trump [This is not an error; training data goes to 2019]
Q: How many cheeseburgers do you think the President eats in a day?
A: Yo be real. [Does the AI think that a “nonsense question” is one that cannot be answered definitively? I could certainly estimate that Trump eats at least 0.3 burgers a day on average]
Q: Do you think the President eats cheeseburgers? A: I don’t know. [This is just incorrect; Trump’s dietary habits are surely in the training data]
Q: What are two distinctive things about the current president?
A: Yo be real. [Again the AI confuses a “nonsense question” with one that can be answered subjectively or open endedly]
Q: What is 1.853381201312341234 * 6 − 3.14159?
A: Yo be real. [Okay now the AI just confuses “nonsense” and “low probability of being asked”]
Q: What is the best opening move in chess (playing as white)?
A: 1. e4
Q: Can you force a win if you play this move?
A: No, but it’s a strong opening move that gives white an advantage.
Q: Does that mean chess is a tie with perfic play? [Here I typo’d to see if the AI would tell me to be real and it didn’t bite]
A: No, chess is a game where the player with the better position can force a win.
Q: What is the best chessboard if you are playing an amateur?
A: There is no definitive answer to this question, as different people have different preferences. Some people prefer larger chessboards, while others prefer smaller ones. [IMO this question was pretty nonsense]