Speaking for myself, I think this research was worth publishing because its benefits to understanding LLMs outweigh its costs from advancing capabilities.
In particular, the reversal curse shows us how LLM cognition differs from human cognition in important ways, which can help us understand the “psychology” of LLMs. I don’t think this finding will to advance capabilities a lot because:
It doesn’t seem like a strong impediment to LLM performance (as indicated by the fact that people hadn’t noticed it until now).
Many facts are presented in both directions during training, so the reversal curse is likely not a big deal in practice.
Bidirectional LLMs (e.g. BERT) likely do not suffer from the reversal curse.[1] If solving the reversal curse confers substantial capabilities gains, people could have taken advantage of this by switching from autoregressive LLMs to bidirectional ones.
Speaking for myself, I think this research was worth publishing because its benefits to understanding LLMs outweigh its costs from advancing capabilities.
In particular, the reversal curse shows us how LLM cognition differs from human cognition in important ways, which can help us understand the “psychology” of LLMs. I don’t think this finding will to advance capabilities a lot because:
It doesn’t seem like a strong impediment to LLM performance (as indicated by the fact that people hadn’t noticed it until now).
Many facts are presented in both directions during training, so the reversal curse is likely not a big deal in practice.
Bidirectional LLMs (e.g. BERT) likely do not suffer from the reversal curse.[1] If solving the reversal curse confers substantial capabilities gains, people could have taken advantage of this by switching from autoregressive LLMs to bidirectional ones.
Since they have to predict “_ is B” in addition to “A is _”.