Robert_AIZI comments on GPT-4: What we (I) know about it

[Robert_AIZI]

That’s true, but for the long run behavior, the more expensive dense attention layers should still dominate, I think.