I’d just like to point out that the construction of this article is an exact ascension into the meta of the construction of the problem within it (except that nobody is paying you the utility you spend on finding the highest-high metastrategy back). This is suggestive of the fact that you need a generalized solution to the strategy-picking problem that picks itself. (If it didn’t pick itself, it wouldn’t be fully general, or it would be making worse choices than some other strategy, inwhichcase it’s insufficiently maximal.) A defined (but not necessarily finite) set of strategies which pick among themselves would also be satisfactory, provided that any one of that set of strategies converges in finite time to the maximal among that set of strategies.
Which if you think about it, is exactly the problem to be solved to produce self-modifying FAI; finding the set of strategy-selecting strategies which select strategies only within their own set, and for which each member of that set converges on the best strategy within that set. (Although FAI doesn’t necessarily have to do it in finite time.)
I’d just like to point out that the construction of this article is an exact ascension into the meta of the construction of the problem within it (except that nobody is paying you the utility you spend on finding the highest-high metastrategy back). This is suggestive of the fact that you need a generalized solution to the strategy-picking problem that picks itself. (If it didn’t pick itself, it wouldn’t be fully general, or it would be making worse choices than some other strategy, inwhichcase it’s insufficiently maximal.) A defined (but not necessarily finite) set of strategies which pick among themselves would also be satisfactory, provided that any one of that set of strategies converges in finite time to the maximal among that set of strategies.
Which if you think about it, is exactly the problem to be solved to produce self-modifying FAI; finding the set of strategy-selecting strategies which select strategies only within their own set, and for which each member of that set converges on the best strategy within that set. (Although FAI doesn’t necessarily have to do it in finite time.)