Eliezer invented Timeless Decision Theory. Getting a decision theory that works for self-modifying or self-copying agents is in his view an important step in developing AGI.
He hasn’t finished it. I hope he does and I will be impressed. But I don’t think that answers what Raw_Power asks for. Humans are the weak spot when it comes to solving friendly AI. In my opinion it is justified to ask if Eliezer Yudkowsky (but also other people within the SIAI), are the right people for the job.
If the SIAI openly admits that it doesn’t have the horse power yet to attempt some hard problems, that would raise my confidence in their capability. That’s no contradiction, because it would pose a solvable short-term goal that can be supported by contributing money and finding experts who can judge the mathematical talent of job candidates.
Yudkowsky gave a detailed answer the last time you asked. Also, Drescher points out a particular error that DBDT makes: in Newcomb’s problem, if Omega chooses the contents of the box before the agent is born, the agent will two-box.
Also, Drescher points out a particular error that DBDT makes: in Newcomb’s problem, if Omega chooses the contents of the box before the agent is born, the agent will two-box.
The actual objection was:
I don’t think DBDT gives the right answer if the predictor’s snapshot of the local universe-state was taken before the agent was born (or before humans evolved, or whatever), because the “critical point”, as Fisher defines it, occurs too late.
Surely, as I pointed out at the time, the author already covered that in the paper. See this bit:
For now, let us take it for granted that, in short-duration scenarios like Newcomb’s problem and the psychologically-similar prisoners’ dilemma, the critical point comes prior to the first events mentioned in standard descriptions of these scenarios. (See Figure 1.)
...and this bit:
The critical point in Newcomb’s problem comes prior to the visit to the predictor.
Yudkowsky’s objection is based on the same mistake. He says:
there’s still a possibility that DBDT will end up two-boxing if Omega takes a snapshot of the (classical) universe a billion years ago before DBDT places the “critical point”
...but this directly contradicts what it says in the paper about where that point is located:
the critical point must come before the contents of the opaque box are determined.
...and...
the critical point comes prior to the first events mentioned in standard descriptions of these scenarios.
Again, what is DBDT to do in Drescher’s counterexample? All the author says is that he doesn’t consider that case in the paper, or possibly considers it lying outside the scope of his decision theory. TDT and UDT can deal with that case, and give the right answer, whereas DBDT, if applied in that (perhaps unintended) case, gives the wrong answer.
What I think is that cases where such situations would arise are corner cases of rather low practical significance...
...but yes, if you really believed that an all powerful agent took a snapshot of the universe before you were born, successfully predicted your dispositions from it and made important decisions based on the results, then the obvious way to deal with that within DBDT would be to put the “critical point” early on (the paper is pretty clear about the need to do this), and consider that the dynamical system before your creation had dispositions that must have causally led to your own dispositions. A “disposition” is treated as just a propensity to behave in a particular way in particular circumstances—so is quite a general concept.
On a first glance, the two should cash out the same as a decision theory for humans, but TDT seems more amenable to programming an AI; a disposition is a fuzzy intuitive category compared to the hypothesis “this algorithm outputs X”.
It doesn’t make decisions, since the process of selecting a “critical point” is not specified, only some informal heuristics for doing so.
Uh huh—well that seems kind-of appropriate for a resource-limited agent. The more of the universe you consider, the harder that becomes—so the more powerful the agent has to be to be able to do it.
Yudkowsky’s idea has agents hunting through all spacetime for decision processes which are correlated with theirs—which is enormously-more expensive—and seems much less likely to lead to any decisions actually being made in real time. The DBDT version of that would be to put the “critical point” at the beginning of time.
However, a means of cutting down the work required to make a decision seems to be an interesting and potentially-useful idea to me. If an agent can ignore much of the universe when making a decision, it is interesting to be aware of that—and indeed necessary if we want to build a practical system.
Eliezer invented Timeless Decision Theory. Getting a decision theory that works for self-modifying or self-copying agents is in his view an important step in developing AGI.
He hasn’t finished it. I hope he does and I will be impressed. But I don’t think that answers what Raw_Power asks for. Humans are the weak spot when it comes to solving friendly AI. In my opinion it is justified to ask if Eliezer Yudkowsky (but also other people within the SIAI), are the right people for the job.
If the SIAI openly admits that it doesn’t have the horse power yet to attempt some hard problems, that would raise my confidence in their capability. That’s no contradiction, because it would pose a solvable short-term goal that can be supported by contributing money and finding experts who can judge the mathematical talent of job candidates.
So: does that do anything that Disposition-Based Decision Theory doesn’t?
Yudkowsky gave a detailed answer the last time you asked. Also, Drescher points out a particular error that DBDT makes: in Newcomb’s problem, if Omega chooses the contents of the box before the agent is born, the agent will two-box.
The actual objection was:
Surely, as I pointed out at the time, the author already covered that in the paper. See this bit:
...and this bit:
Yudkowsky’s objection is based on the same mistake. He says:
...but this directly contradicts what it says in the paper about where that point is located:
...and...
Again, what is DBDT to do in Drescher’s counterexample? All the author says is that he doesn’t consider that case in the paper, or possibly considers it lying outside the scope of his decision theory. TDT and UDT can deal with that case, and give the right answer, whereas DBDT, if applied in that (perhaps unintended) case, gives the wrong answer.
You are not being very clear. Where does the author say either of those things?
In the passages you quoted.
AFAICS, the author does not say anything like: “that he doesn’t consider that case in the paper”.
He doesn’t say anything like that he: “possibly considers it lying outside the scope of his decision theory” either.
Do you believe that DBDT can place a critical point at the time/situation where the agent doesn’t exist?
What I think is that cases where such situations would arise are corner cases of rather low practical significance...
...but yes, if you really believed that an all powerful agent took a snapshot of the universe before you were born, successfully predicted your dispositions from it and made important decisions based on the results, then the obvious way to deal with that within DBDT would be to put the “critical point” early on (the paper is pretty clear about the need to do this), and consider that the dynamical system before your creation had dispositions that must have causally led to your own dispositions. A “disposition” is treated as just a propensity to behave in a particular way in particular circumstances—so is quite a general concept.
Interesting philosopher- thanks for the link!
On a first glance, the two should cash out the same as a decision theory for humans, but TDT seems more amenable to programming an AI; a disposition is a fuzzy intuitive category compared to the hypothesis “this algorithm outputs X”.
TDT is (more) technical.
I meant more: does it make any decisions differently.
It doesn’t make decisions, since the process of selecting a “critical point” is not specified, only some informal heuristics for doing so.
Uh huh—well that seems kind-of appropriate for a resource-limited agent. The more of the universe you consider, the harder that becomes—so the more powerful the agent has to be to be able to do it.
Yudkowsky’s idea has agents hunting through all spacetime for decision processes which are correlated with theirs—which is enormously-more expensive—and seems much less likely to lead to any decisions actually being made in real time. The DBDT version of that would be to put the “critical point” at the beginning of time.
However, a means of cutting down the work required to make a decision seems to be an interesting and potentially-useful idea to me. If an agent can ignore much of the universe when making a decision, it is interesting to be aware of that—and indeed necessary if we want to build a practical system.
Huh, cool. Looks pretty much the same, though minus some arguments and analysis.
It certainly seems like a rather similar perspective. It was published back in 2002.