I’d have guessed the disagreement wasn’t about whether “222 + 222 = 555” is an incorrect map, or about whether incorrect maps often make it harder to navigate the territory, but about something else. (Maybe ‘I don’t want to think about this because it seems irrelevant/disanalogous to alignment work’?)
I’d have guessed that too, which is why I would have preferred him to say that they disagreed on |whatever meta question he’s actually talking about| instead of implying disagreement on |other thing that makes his disappointment look more reasonable|.
And I’d have guessed the answer Eliezer was looking for was closer to ‘the OP’s entire Section B’ (i.e., a full attempt to explain all the core difficulties), not a one-sentence platitude establishing that there’s nonzero difficulty? But I don’t have inside info about this experiment.
That story sounds much more cogent, but it’s not the primary interpretation of “I asked them a single question” followed by the quoted question. Most people don’t go on 5 paragraph rants in response to single questions, and when they do they tend to ask clarifying details regardless of how well they understand the prompt, so they know they’re responding as intended.
I’d have guessed that too, which is why I would have preferred him to say that they disagreed on |whatever meta question he’s actually talking about| instead of implying disagreement on |other thing that makes his disappointment look more reasonable|.
That story sounds much more cogent, but it’s not the primary interpretation of “I asked them a single question” followed by the quoted question. Most people don’t go on 5 paragraph rants in response to single questions, and when they do they tend to ask clarifying details regardless of how well they understand the prompt, so they know they’re responding as intended.