Disclaimer: I have not read the Wentworth’s post or the linked one but I know (little) about finite-sample and asymptotic bounds.
(He had another few versions, allegedly with fuller proofs, though I was not able to understand them and focused on this one.)
I think the key point of the statement is “any finite-entropy function f(X)”. This makes sure that the “infinity” in the sampling goes away. That being said, it should be possible to extend the proof to non-independent samples, Cosma Shalizi has done a ton of work on this.
Disclaimer: I have not read the Wentworth’s post or the linked one but I know (little) about finite-sample and asymptotic bounds.
I think the key point of the statement is “any finite-entropy function f(X)”. This makes sure that the “infinity” in the sampling goes away. That being said, it should be possible to extend the proof to non-independent samples, Cosma Shalizi has done a ton of work on this.