The problem is that there are a bunch of intermediate positions between LUT and strong AI. The doubt about an LLM is whether it has a world model, when it is only designed as a word model.
That claim is unjustified and unjustifiable. Everything is fundamentally a black box until proven otherwise. And we will never find any conclusive proof. (I want to tell you to look up Hume’s problem of induction and Karl Popper’s solution, although I feel that making such a remark would be insulting your intelligence.) Our ability to imagine systems behaving in ways that are 100% predictable and our ability to test systems so as to ensure that they behave predictably does not change the fact that everything is always fundamentally a black box.
This might a key crux: I think that white box is a rare state, and interpretability is not the default case here.
Indeed I see the opposite: Black box strategies like Model Free RL work a lot better currently than white box strategies, and I think black boxes are the default scenario. You have to do a lot of work to interpret things enough to make it white box.
The problem is that there are a bunch of intermediate positions between LUT and strong AI. The doubt about an LLM is whether it has a world model, when it is only designed as a word model.
Nothing is fundamentally a black box.
That claim is unjustified and unjustifiable. Everything is fundamentally a black box until proven otherwise. And we will never find any conclusive proof. (I want to tell you to look up Hume’s problem of induction and Karl Popper’s solution, although I feel that making such a remark would be insulting your intelligence.) Our ability to imagine systems behaving in ways that are 100% predictable and our ability to test systems so as to ensure that they behave predictably does not change the fact that everything is always fundamentally a black box.
Nothing complex is a black box , because it has components, which can potentially be understood.
Nothing artificial is a black box to the person who built it.
An LLM is , of course, complex and artificial.
What justifies that claim?
I wasn’t arguing on that basis.
This might a key crux: I think that white box is a rare state, and interpretability is not the default case here.
Indeed I see the opposite: Black box strategies like Model Free RL work a lot better currently than white box strategies, and I think black boxes are the default scenario. You have to do a lot of work to interpret things enough to make it white box.
Any system of sufficient complexity is incomprehensible to an observer of insufficient intelligence. But that’s not fundamental.