Do they mention context window size somewhere? I couldn’t find it. Though in principle I think it could be computed from the other hyperparameters they list.
It’s strongly implied to be 2048, as in Gopher.
Do they mention context window size somewhere? I couldn’t find it. Though in principle I think it could be computed from the other hyperparameters they list.
It’s strongly implied to be 2048, as in Gopher.