The post isn’t completely meaningless, it’s just the sort of mental scratchwork you generate when you’re kicking around a new idea. Let me try to explain.
If you look at decision-making from the point of view of the AI doing the decision-making, you’ll notice that some mathematical facts that are supposed to be “fixed” (like the return value of your algorithm) become kind of vague and “controllable”. The deterministic operation of the AI can make mathematical facts come out a certain way which maximizes the AI’s utility. For example, an AI could have a tiny bit of control over the probabilities of certain bitstrings under the universal prior (which is after all a mixture of all possible programs including the AI), or the bits of Chaitin’s omega, or the truth values of certain statements in the arithmetical hierarchy. That last part freaked Will out, then Paul came along and said there’s nothing to worry about, exerting control over math is business as usual.
The post isn’t completely meaningless, it’s just the sort of mental scratchwork you generate when you’re kicking around a new idea. Let me try to explain.
If you look at decision-making from the point of view of the AI doing the decision-making, you’ll notice that some mathematical facts that are supposed to be “fixed” (like the return value of your algorithm) become kind of vague and “controllable”. The deterministic operation of the AI can make mathematical facts come out a certain way which maximizes the AI’s utility. For example, an AI could have a tiny bit of control over the probabilities of certain bitstrings under the universal prior (which is after all a mixture of all possible programs including the AI), or the bits of Chaitin’s omega, or the truth values of certain statements in the arithmetical hierarchy. That last part freaked Will out, then Paul came along and said there’s nothing to worry about, exerting control over math is business as usual.