interstice comments on Clarifying The Malignity of the Universal Prior: The Lexical Update

[interstice]

Thanks, that makes sense. Here is my rephrasing of the argument:

Let the ‘importance function’ $Q(X,Y)$ take as inputs machines $X$and $Y$, and output all places where $Y$ is being used as a universal prior, weighted by their effect on $X$-short programs. Suppose for the sake of argument that there is some short program computing $Q$; this is probably the most ‘natural’ program of this form that we could hope for.

Even given such a program, we’ll still lose to the aliens: in $U^{\prime }$, directly specifying our important decisions on Earth using $Q$ will require both $U$ and $U^{\prime }$ to be fed into $Q$, costing $K_{U^{\prime }}\left(U\right)+K_{U^{\prime }}\left(U^{\prime }\right)$ bits, then $K_{U^{\prime }}\left(us|Q\right(U,U^{\prime }\left)\right)$ bits to specify us. For the aliens, getting them to be motivated to control $U$-short programs costs $K_{U^{\prime }}\left(U\right)$ bits, but then they can skip directly to specifying us given $Q(U,U^{\prime })$, so they save $K_{U^{\prime }}\left(U^{\prime }\right)$ bits over the direct explanation. So the lexical update works.

(I went wrong in thinking that the aliens would need to both update their notion of importance to match ours *and* locate our world; but if we assume the ‘importance function’ exists then the aliens can just pick out our world using our notion of importance)