Yes, this is what I meant, reposting here insights @Arthur Conmy gave me on twitter
In general I expect the encoder directions to basically behave like the decoder direction with noise. This is because the encoder has to figure out how much features fire while keeping track of interfering features due to superposition. And this adjustment will make it messier
Yes, this is what I meant, reposting here insights @Arthur Conmy gave me on twitter