Yep, you’re correct. The original argument in the Waluigi mega-post was sloppy.
If μ updated the amplitudes in a perfectly bayesian way and the context window was infinite, then the amplitudes of each premise must be a martingale. But the finite context breaks this.
Regarding your other comment, a long context window doesn’t mean that the waluigis won’t appear quickly. Even with an infinite context window, the waluigi might appear immediately. The assumption that the context window is short/finite is only necessary to establish that the waluigi is an absorbing state but luigi isn’t.
Yep, you’re correct. The original argument in the Waluigi mega-post was sloppy.
If μ updated the amplitudes in a perfectly bayesian way and the context window was infinite, then the amplitudes of each premise must be a martingale. But the finite context breaks this.
Here is a toy model which shows how the finite context window leads to Waluigi Effect. Basically, the finite context window biases the Dynamic LLM towards premises which can be evidenced by short strings (e.g. waluigi), and biases away from premises which can’t be evidenced by short strings (e.g. luigis).
Regarding your other comment, a long context window doesn’t mean that the waluigis won’t appear quickly. Even with an infinite context window, the waluigi might appear immediately. The assumption that the context window is short/finite is only necessary to establish that the waluigi is an absorbing state but luigi isn’t.