Note that the distinction between hinge beliefs and free beliefs does not supervene on the black-box behaviour of NNs/LLMs. It depends on how the belief is implemented, how the belief is learned, how the belief might change, etc.
(2)
“The second model uses a matrix that will always be symmetric, no matter what it’s learned.” might make it seem that the two models are more similar than they actually are.
You might think that both models store an n×n matrix A, and the architecture of both models is xTAy, but Model 1 has a slightly symmetric matrix A whereas Model 2 has an exactly symmetric matrix A. But this isn’t true. The second model doesn’t store a symmetric matrix — it stores an upper triangle.
Yep, exactly!
Two things to note:
(1)
Note that the distinction between hinge beliefs and free beliefs does not supervene on the black-box behaviour of NNs/LLMs. It depends on how the belief is implemented, how the belief is learned, how the belief might change, etc.
(2)
“The second model uses a matrix that will always be symmetric, no matter what it’s learned.” might make it seem that the two models are more similar than they actually are.
You might think that both models store an n×n matrix A, and the architecture of both models is xTAy, but Model 1 has a slightly symmetric matrix A whereas Model 2 has an exactly symmetric matrix A. But this isn’t true. The second model doesn’t store a symmetric matrix — it stores an upper triangle.