Beforehand I was very confident that vector additions would work here, even though I knew that the fully connected additions didn’t work. Before showing him the results, but after showing the results for the fully connected network, I asked TurnTrout for his prediction. He gave 85% that the additions would work.
I want to clarify that I had skimmed the original results and concluded that they “worked” in that 3-1 vectors got e.g. 1s to be classified as 3s. (This is not trivial, since not all 1 activations are the same!) However, those results “didn’t work” in that they destroyed performance on non-1 images.
I thought I was making predictions on whether 3-1 vectors get 1s to be classified as 3s by this residual network. I guess I’m going to mark my prediction here as “ambiguous”, in that case.
I want to clarify that I had skimmed the original results and concluded that they “worked” in that 3-1 vectors got e.g. 1s to be classified as 3s. (This is not trivial, since not all 1 activations are the same!) However, those results “didn’t work” in that they destroyed performance on non-1 images.
I thought I was making predictions on whether 3-1 vectors get 1s to be classified as 3s by this residual network. I guess I’m going to mark my prediction here as “ambiguous”, in that case.
Oh, sorry. Editing post with correction.