Say I have a computer that will simulate an arbitrary Turing machine T, and will award me one utilon when that machine halts, and do nothing for me until that happens. With some clever cryptocurrency scheme, this is a scenario I could actually build today.
No, you can’t do that today. You could produce a contraption that will deposit 1 BTC into a certain bitcoin wallet if and when some computer program halts, but this won’t do the wallet’s owner much good if they die before the program halts. If you reflect on what it means to award someone a utilon, rather than a bitcoin, I maintain that it isn’t obvious that this is even possible in theory.
Why in the world would one expect a utility function over an uncountable domain to be computable?
There is a notion of computability in the continuous setting.
As far as I can see, the motivation for requiring a utility function to be computable is that this would make optimization for said utility function to be a great deal easier.
This seems like a strawman to me. A better motivation would be that agents that actually exist are computable, and a utility function is determined by judgements rendered by the agent, which is incapable of thinking uncomputable thoughts.
Clearly, there is a kind of utility function action that is computable. Clearly the kind of UF that is defined in terms of preferences over fine-grained world-states isn’t computable. So, clearly, “utility function” is being used to mean different things.
No, you can’t do that today. You could produce a contraption that will deposit 1 BTC into a certain bitcoin wallet if and when some computer program halts, but this won’t do the wallet’s owner much good if they die before the program halts. If you reflect on what it means to award someone a utilon, rather than a bitcoin, I maintain that it isn’t obvious that this is even possible in theory.
There is a notion of computability in the continuous setting.
This seems like a strawman to me. A better motivation would be that agents that actually exist are computable, and a utility function is determined by judgements rendered by the agent, which is incapable of thinking uncomputable thoughts.
Clearly, there is a kind of utility function action that is computable. Clearly the kind of UF that is defined in terms of preferences over fine-grained world-states isn’t computable. So, clearly, “utility function” is being used to mean different things.