This is a reply to Alex’s comment 792 but I’m placing it here since for some weird reason the website doesn’t let me reply to 792
I think that the idea that contradictions should lead to infinite utility is probably something that doesn’t work for real models of logical uncertainty.
Why not?
So, I started writing an explanation why it doesn’t work, tried to anticipate the loopholes you would point out in this explanation and ended up with the conclusion it actually does work :)
First, note that in logical uncertainty the boolean divide between “contradiction” and “consistency” is replaced by a continuum. Logical conditional expectations become less and less stable as the probability of the condition goes to zero (see this; for a generalization to probabilistic algorithms see this). What we can do in the spirit of your proposal is e.g. maximize E[U∣A()=a]−TlogPr[A()=a] for some small constant T (in the optimal predictor formalism we probably want T to be a function of k that goes to 0 as k goes to infinity).
The problem was that the self-referential nature of UDT requies optimal predictors for reflective systems and the construction I knew for the latter yielded probabilistic optimal predictors since it uses the Kakutani fixed point theorem and we need to form mixtures to apply it. With probabilistic optimal predictors things get hairy since the stability condition “Prlogical[A()=a]>ϵ” is replaced by the condition “lowest eigenvalue of Elogical[Prindexical[A()=a]Prindexical[A()=b]]>ϵ”. There seems to be no way to stabilize this new condition. There are superficially appealing analogues that in the degenerate case reduce to choosing the action most unlikely in the normal distribution with mean Elogical[Prindexical[A()=a]] and covariance Elogical[Prindexical[A()=a]Prindexical[A()=b]]. Unfortunately it doesn’t work since there might be several actions with similar likelihoods that get chosen with different indexical probabilities consistently with the above mean and covariance. Indeed it would be impossible for it to work since in particular it would allow getting non-negligible logical variance of a quantity that depends on no parameters, which cannot be (since it is always possible to hardcode such a quantity).
However, recently I discovered reflective systems that are deterministic (and which seem the right thing to use for real agents because of independent reasons). For these systems the “naive” method works! This again caches out into some sort of pseudorandomization but this way the pseudorandomization arises naturally instead of having to insert an arbitrary pseudorandom function by hand. Moreover it looks like it solves some issues with making the formalism truly “updateless” (i.e. dealing correctly with scenarios similar to counterfactual mugging).
This is a reply to Alex’s comment 792 but I’m placing it here since for some weird reason the website doesn’t let me reply to 792
So, I started writing an explanation why it doesn’t work, tried to anticipate the loopholes you would point out in this explanation and ended up with the conclusion it actually does work :)
First, note that in logical uncertainty the boolean divide between “contradiction” and “consistency” is replaced by a continuum. Logical conditional expectations become less and less stable as the probability of the condition goes to zero (see this; for a generalization to probabilistic algorithms see this). What we can do in the spirit of your proposal is e.g. maximize E[U∣A()=a]−TlogPr[A()=a] for some small constant T (in the optimal predictor formalism we probably want T to be a function of k that goes to 0 as k goes to infinity).
The problem was that the self-referential nature of UDT requies optimal predictors for reflective systems and the construction I knew for the latter yielded probabilistic optimal predictors since it uses the Kakutani fixed point theorem and we need to form mixtures to apply it. With probabilistic optimal predictors things get hairy since the stability condition “Prlogical[A()=a]>ϵ” is replaced by the condition “lowest eigenvalue of Elogical[Prindexical[A()=a]Prindexical[A()=b]]>ϵ”. There seems to be no way to stabilize this new condition. There are superficially appealing analogues that in the degenerate case reduce to choosing the action most unlikely in the normal distribution with mean Elogical[Prindexical[A()=a]] and covariance Elogical[Prindexical[A()=a]Prindexical[A()=b]]. Unfortunately it doesn’t work since there might be several actions with similar likelihoods that get chosen with different indexical probabilities consistently with the above mean and covariance. Indeed it would be impossible for it to work since in particular it would allow getting non-negligible logical variance of a quantity that depends on no parameters, which cannot be (since it is always possible to hardcode such a quantity).
However, recently I discovered reflective systems that are deterministic (and which seem the right thing to use for real agents because of independent reasons). For these systems the “naive” method works! This again caches out into some sort of pseudorandomization but this way the pseudorandomization arises naturally instead of having to insert an arbitrary pseudorandom function by hand. Moreover it looks like it solves some issues with making the formalism truly “updateless” (i.e. dealing correctly with scenarios similar to counterfactual mugging).
Very pleased with this development!