An AIXI-like agent doesn’t understand that it lives within the universe that it’s trying to affect, so it can unwittingly destroy its own hardware with its mining claws (to borrow a phrase from Tim Tyler).
It looks like the newer version of the paper tries to deal with this explicitly, by introducing the concept of “agent implementation”. But I can’t verify whether the solution actually works, since the probability function P is left undefined.
I think the paper suffers from what may be a common failure mode in AI design: problems with the overall approach/framework can always be hidden with additional degrees of freedom “to be filled in later”. (The paper actually introduces two unspecified probability functions.) In a way UDT does the same thing: the only version that doesn’t have obvious problems uses an unspecified “math intuition module”.
Of course I still think UDT is the better approach, but it seems hard to make an argument beyond “it just makes more intuitive sense”. Well, I’m glad I’m a hobbyist who can work on whatever I want and not have to justify myself to a funding committee. :)
It looks like the newer version of the paper tries to deal with this explicitly, by introducing the concept of “agent implementation”. But I can’t verify whether the solution actually works, since the probability function P is left undefined.
I think the paper suffers from what may be a common failure mode in AI design: problems with the overall approach/framework can always be hidden with additional degrees of freedom “to be filled in later”. (The paper actually introduces two unspecified probability functions.) In a way UDT does the same thing: the only version that doesn’t have obvious problems uses an unspecified “math intuition module”.
Of course I still think UDT is the better approach, but it seems hard to make an argument beyond “it just makes more intuitive sense”. Well, I’m glad I’m a hobbyist who can work on whatever I want and not have to justify myself to a funding committee. :)