Do you think there’s an army of OpenAI employees constantly scouring twitter for these sorts of prompt examples and then rushing to push an update a few hours later just for that?
(For what it’s worth, I find it plausible this could be happening, and didn’t find the posts counterarguments that convincing.)
Yup, having multiple people reporting on twitter the same bogus answer and then the answer being mysteriously correct the day after with the same prompt is a very suspicious pattern, especially considering that it already happened multiple times.
Also, i just broke GPT-4 again with just a slightly different prompt:
Sally (a girl) has k brothers. Each brother has j-1 sisters. How many sisters does Sally have?
Answer (wrong):
Given that Sally has k brothers and each of her brothers has j-1 sisters, we can use this information to find out how many sisters Sally has.
If each of Sally’s brothers has j-1 sisters, then j represents the total number of girls in the family (because each brother counts all the sisters in the family, excluding themselves).
Since Sally is included in the count of j, the number of Sally’s sisters would be j-1.
(For what it’s worth, I find it plausible this could be happening, and didn’t find the posts counterarguments that convincing.)
Yup, having multiple people reporting on twitter the same bogus answer and then the answer being mysteriously correct the day after with the same prompt is a very suspicious pattern, especially considering that it already happened multiple times.
Also, i just broke GPT-4 again with just a slightly different prompt:
Sally (a girl) has k brothers. Each brother has j-1 sisters. How many sisters does Sally have?
Answer (wrong):
Given that Sally has
kbrothers and each of her brothers hasj-1sisters, we can use this information to find out how many sisters Sally has.If each of Sally’s brothers has
j-1sisters, thenjrepresents the total number of girls in the family (because each brother counts all the sisters in the family, excluding themselves).Since Sally is included in the count of
j, the number of Sally’s sisters would bej-1.So, Sally has
j-1sisters.