I’ve been using computable to mean a total function (each instance is computable in finite time).
I’m thinking of an agent outside a universe about to take an action, and each action will cause that universe to run a particular TM. (You could maybe frame this as “the agent chooses the tape for the TM to run on”.) For me, this is analogous to acting in the world and causing the world to shift toward some outcomes over others.
By asserting that U should be the computable one, I’m asserting that “how much do I like this outcome” is a more tractable question than “which actions result in this outcome”.
An intuition pump in a human setting:
I can check whether given states of a Go board are victories for one player or the other, or if the game is not yet finished (this is analogous to U being a total computable function). But it’s much more difficult to choose, for an unfinished game where I’m told I have a winning strategy, a move such that I still have a winning strategy. The best I can really do as a human is calculate a bit and then guess at how the leaves will probably resolve if we go down them (this is analogous to eval being an enumerable but not necessarily computable function).
In general, individual humans are much better at figuring out what outcomes we want than we are at figuring out exactly how to achieve those outcomes. (It would be quite weird if the opposite were the case.) We’re not good at either in an absolute sense, of course.
I’ve been using computable to mean a total function (each instance is computable in finite time).
I’m thinking of an agent outside a universe about to take an action, and each action will cause that universe to run a particular TM. (You could maybe frame this as “the agent chooses the tape for the TM to run on”.) For me, this is analogous to acting in the world and causing the world to shift toward some outcomes over others.
By asserting that U should be the computable one, I’m asserting that “how much do I like this outcome” is a more tractable question than “which actions result in this outcome”.
An intuition pump in a human setting:
I can check whether given states of a Go board are victories for one player or the other, or if the game is not yet finished (this is analogous to U being a total computable function). But it’s much more difficult to choose, for an unfinished game where I’m told I have a winning strategy, a move such that I still have a winning strategy. The best I can really do as a human is calculate a bit and then guess at how the leaves will probably resolve if we go down them (this is analogous to eval being an enumerable but not necessarily computable function).
In general, individual humans are much better at figuring out what outcomes we want than we are at figuring out exactly how to achieve those outcomes. (It would be quite weird if the opposite were the case.) We’re not good at either in an absolute sense, of course.