The operationalization of myopia with large language models here seems more like a compelling metaphor than a useful technical concept. It’s not clear that “myopia” in next-word prediction within a sentence corresponds usefully to myopia on action-relevant timescales. For example, it would be trivial to remove almost all within-sentence myopia by doing decoding with beam search, but it’s hard to believe that beam search would meaningfully impact alignment outcomes.
Hi Adly, thanks for your comment. Megan and I were thinking about what you said and discussing it. Would you care to elaborate? To be honest, I’m not sure we are understanding exactly what you’re saying.
A couple thoughts we had after reading your comment that might be jumping off points or help you see where the disconnect is:
How do you expect to have myopia over “action-relevant timescales” if you don’t first have myopia over next-token prediction?
The operationalization of myopia with large language models here seems more like a compelling metaphor than a useful technical concept. It’s not clear that “myopia” in next-word prediction within a sentence corresponds usefully to myopia on action-relevant timescales. For example, it would be trivial to remove almost all within-sentence myopia by doing decoding with beam search, but it’s hard to believe that beam search would meaningfully impact alignment outcomes.
Hi Adly, thanks for your comment. Megan and I were thinking about what you said and discussing it. Would you care to elaborate? To be honest, I’m not sure we are understanding exactly what you’re saying.
A couple thoughts we had after reading your comment that might be jumping off points or help you see where the disconnect is:
How do you expect to have myopia over “action-relevant timescales” if you don’t first have myopia over next-token prediction?
What do you mean by “action-relevant timescales”?