I realized the previous experiment might be importantly misleading because it’s on a small 12 layer model. In larger models it would still be a big deal if the effective layer horizon was like 20 layers.
Previously the code was too slow to run on larger models. But I made an faster version and ran the same experiment on GPT-2 large (48 layers):
We clearly see the same pattern again. As TurnTrout predicted, there seems be something like an exponential decay in the importance of previous layers as you go futher back. I expect that on large models an the effective layer horizon is an importnat consideration.
I realized the previous experiment might be importantly misleading because it’s on a small 12 layer model. In larger models it would still be a big deal if the effective layer horizon was like 20 layers.
Previously the code was too slow to run on larger models. But I made an faster version and ran the same experiment on GPT-2 large (48 layers):
We clearly see the same pattern again. As TurnTrout predicted, there seems be something like an exponential decay in the importance of previous layers as you go futher back. I expect that on large models an the effective layer horizon is an importnat consideration.
Updated source: https://gist.github.com/UFO-101/41b7ff0b250babe69bf16071e76658a6