If the trials are independent then the limit will not (with probability P→0) converge after an arbitrarily long stretch of erratic behavior.
Be aware that a decidable agent deciding to do or not do another trial itself means the trials are not independent. (Consider the case of an environment that simulates the agent and behaves erratically if and only if the agent would decide to continue.)
Also be aware that the presence of an embedded agent with memory in the world itself means the trials are not independent.
Limit convergence doesn’t need to be proved because limit convergence is a mathematical (as opposed to a scientific) fact.
...under certain assumptions that people tend to forget.
The limit of the mean of a Cauchy distribution does not converge, for instance.
I find the use of ‘mathematical fact’ as an argument surprisingly[1] strong evidence that someone has overgeneralized, in practice. This instance is no exception.
Be aware that a decidable agent deciding to do or not do another trial itself means the trials are not independent. (Consider the case of an environment that simulates the agent and behaves erratically if and only if the agent would decide to continue.)
Also be aware that the presence of an embedded agent with memory in the world itself means the trials are not independent.
...under certain assumptions that people tend to forget.
The limit of the mean of a Cauchy distribution does not converge, for instance.
I find the use of ‘mathematical fact’ as an argument surprisingly[1] strong evidence that someone has overgeneralized, in practice. This instance is no exception.
In the sense of ‘this works far more often than I would expect’.