edit: okay, that’s just argument over the definition of NDT.
Then don’t refer to Halting problem and Godel’s incompleteness theorem. It is hard to object on the very informal things; all I can say, ‘you didn’t prove this’, but it is a fully general counter argument to informal things.
The other misinterpretation is that NDT does not make assumption (outputs X) = xi in this ‘transparent’ way with source code on the left side, and take this as an axiom. It has a black box for itself, denoted by symbol A, which it substitutes for own source code whenever it spots it’s own source code. It substitutes different xi as A into resulting utility equations, solving for utility, and finding the maximum utility. It does not analyze itself.
I do agree that for ONDT (orthonormal’s interpretation of NDT) which you present here, what you say, is correct. I do not agree that your NDT is the NDT that applied mathematicians use. There are substantial differences.
If you were substituting in variables for the output of X rather than analyzing the round as a whole, then you’re not talking about Naive Decision Theory, you’re talking about something in the family of CDT and TDT.
EDIT: Maybe you assumed that I was denigrating your intuitions on decision theory for departing from NDT? If so, that’s not the case- substituting for the output of X in various spots turns out to be a good way to avoid the problem of spurious counterfactuals. CDT does this in a very basic way, TDT in a better way.
What I am saying is that the decision theory which applied mathematicians follow, operates under practical constraints (limited computing time), and this prevents introduction of very fancy things like your (output X) simply because they are computationally heavy. The theory that always substitutes is what I originally noted in the other thread, regarding the newcomb’s. Due to substitution, that theory doesn’t ‘pollute’ it’s own proof checker with a proposition that may be untrue and may break the proof checker (but practically, would make proof checker very slow).
edit: That being said, the decision theories operate under notion of perfect accuracy and unlimited computing time; the applied mathematicians see this field as not very relevant to any practical software (e.g. AI). The practical AI is best off substituting for X everywhere. The practical AI is never a guaranteed utility maximizer; there will always be problems that it won’t be able to solve correctly in the given time, and the important consideration is to try to be able to solve as large set of problems in limited time, as possible. That is why i have very little inclination to write some semi-formalization of what i use when i am writing code to decide on things. I would rather write specifications for making AI that actually does it; the specifications only need to be as formal as for the programmer who implements them to understand the intent. (I can alternatively implement it myself, and then it becomes completely formalized to the point that computer can run it)
It’s just an argument over word definitions, in any case. You can call what i propose TDT if you wish. I’m just pointing out that Eliezer, being a bright guy outside the field, is precisely the kind of person to make a naive but not stupid decision theory which matches what many people actually use in practice.
edit: okay, that’s just argument over the definition of NDT.
Then don’t refer to Halting problem and Godel’s incompleteness theorem. It is hard to object on the very informal things; all I can say, ‘you didn’t prove this’, but it is a fully general counter argument to informal things.
The other misinterpretation is that NDT does not make assumption (outputs X) = xi in this ‘transparent’ way with source code on the left side, and take this as an axiom. It has a black box for itself, denoted by symbol A, which it substitutes for own source code whenever it spots it’s own source code. It substitutes different xi as A into resulting utility equations, solving for utility, and finding the maximum utility. It does not analyze itself.
I do agree that for ONDT (orthonormal’s interpretation of NDT) which you present here, what you say, is correct. I do not agree that your NDT is the NDT that applied mathematicians use. There are substantial differences.
If you were substituting in variables for the output of X rather than analyzing the round as a whole, then you’re not talking about Naive Decision Theory, you’re talking about something in the family of CDT and TDT.
TDT was made by Eliezer, right? Here, a bright guy (maybe naive, but certainly not stupid) outside the field’s theory.
I don’t understand your comment.
EDIT: Maybe you assumed that I was denigrating your intuitions on decision theory for departing from NDT? If so, that’s not the case- substituting for the output of X in various spots turns out to be a good way to avoid the problem of spurious counterfactuals. CDT does this in a very basic way, TDT in a better way.
re: EDIT
Ahh, I see.
What I am saying is that the decision theory which applied mathematicians follow, operates under practical constraints (limited computing time), and this prevents introduction of very fancy things like your (output X) simply because they are computationally heavy. The theory that always substitutes is what I originally noted in the other thread, regarding the newcomb’s. Due to substitution, that theory doesn’t ‘pollute’ it’s own proof checker with a proposition that may be untrue and may break the proof checker (but practically, would make proof checker very slow).
edit: That being said, the decision theories operate under notion of perfect accuracy and unlimited computing time; the applied mathematicians see this field as not very relevant to any practical software (e.g. AI). The practical AI is best off substituting for X everywhere. The practical AI is never a guaranteed utility maximizer; there will always be problems that it won’t be able to solve correctly in the given time, and the important consideration is to try to be able to solve as large set of problems in limited time, as possible. That is why i have very little inclination to write some semi-formalization of what i use when i am writing code to decide on things. I would rather write specifications for making AI that actually does it; the specifications only need to be as formal as for the programmer who implements them to understand the intent. (I can alternatively implement it myself, and then it becomes completely formalized to the point that computer can run it)
It’s just an argument over word definitions, in any case. You can call what i propose TDT if you wish. I’m just pointing out that Eliezer, being a bright guy outside the field, is precisely the kind of person to make a naive but not stupid decision theory which matches what many people actually use in practice.
I edited my comment while you were replying to it. I don’t think we’re actually disagreeing on this particular point.