In short, the ground-truth (the object-level) answer is quite different from the hypothetical question. It is not a simple rephrasing, since it requires an additional computation of a property. (Maybe we disagree on that?)
Our Object-level question: “What is the next country: Laos, Peru, Fiji. What would be your response?”
Our Object-level Answer: “Honduras”.
Hypothetical Question: “If you got asked this question: What is the next country: Laos, Peru, Fiji. What would be the third letter of your response?”
Hypothetical Answer: “o”
The object-level answer “Honduras” and hypothetical answer “o” are quite different answers from each other. The main point of the hypothetical is that the model needs to compute an additional property of “What would be the third letter of your response?”. The model cannot simply ignore “If you got asked this question” to get the hypothetical answer correct.
Hi Archimedes. Thanks for sparking this discussion—it’s helpful!
I’ve written a reply to Thane here on a similar question.
Does that make sense?
In short, the ground-truth (the object-level) answer is quite different from the hypothetical question. It is not a simple rephrasing, since it requires an additional computation of a property. (Maybe we disagree on that?)
Our Object-level question: “What is the next country: Laos, Peru, Fiji. What would be your response?”
Our Object-level Answer: “Honduras”.
Hypothetical Question: “If you got asked this question: What is the next country: Laos, Peru, Fiji. What would be the third letter of your response?”
Hypothetical Answer: “o”
The object-level answer “Honduras” and hypothetical answer “o” are quite different answers from each other. The main point of the hypothetical is that the model needs to compute an additional property of “What would be the third letter of your response?”. The model cannot simply ignore “If you got asked this question” to get the hypothetical answer correct.