This is an interesting point—when we did our causality studies across layers, we also found that the board state features in the middle layers are mostly used causally—not the deep layers. However, the probe accuracy does increase with depth.
I don’t know how this translates to the fact that SAEs also find more of these features in the middle layers. Like, the “natural features” in some sense in the last few layers found by the SAEs do not have to contain much information about the board state but just partial information to make the decision.
This is an interesting point—when we did our causality studies across layers, we also found that the board state features in the middle layers are mostly used causally—not the deep layers. However, the probe accuracy does increase with depth.
I don’t know how this translates to the fact that SAEs also find more of these features in the middle layers. Like, the “natural features” in some sense in the last few layers found by the SAEs do not have to contain much information about the board state but just partial information to make the decision.