Yudkowsky (2010) proposes a new extension of causal decision theory to suit the needs of a self-modifying AI.
I wouldn’t describe TDT as “an extension of CDT”. Also, the AI being self-modifying is probably not the central issue actually addressed by TDT (as opposed to what possibly motivated development of the ideas).
“Timeless decision theory (TDT) is an extension of causal decision networks that compactly represents uncertainty about correlated computational processes and represents the decision-maker as such a process.”
The second point is made in paragraph #2:
“I show that an evidential or causal decision-maker capable of self-modifying actions, given a choice between remaining an evidential or causal decision-maker and modifying itself to imitate a timeless decision-maker, will choose to imitate a timeless decisionmaker on a large class of problems.”
“Timeless decision theory (TDT) is an extension of causal decision networks that compactly represents uncertainty about correlated computational processes and represents the decision-maker as such a process.”
After a fashion, since causal networks are not exactly CDT, modeling correlated computations with causal networks makes them less “causal” (i.e. related to physical causality), and the paper doesn’t achieve clear specification of how to do that (it’s an open problem, but I can say that any nontrivial causal network relating computations may need to be revised in face of new logical evidence, which makes the decision procedure that itself works with resolution of logical uncertainty brittle).
“I show that an evidential or causal decision-maker capable of self-modifying actions, given a choice between remaining an evidential or causal decision-maker and modifying itself to imitate a timeless decision-maker, will choose to imitate a timeless decisionmaker on a large class of problems.”
That CDT/EDT agents with self-modification would become more TDT-like is somewhat different from saying that TDT “suits the needs of a self-modifying AI”. TDT is a more sane theory, and to the extent CDT/EDT agents would prefer to be more effective, they’d prefer to adopt TDT’s greater sanity. But TDT is not a fixed point, suiting the needs of a self-modifying AI is a tall order that probably can’t be met to any reasonable extent, since that would mean establishing some rules that AI itself would not have much opportunity to rebuild. Perhaps laws of physics or logic can quality, with appropriate framing.
(I agree with your description more now than when I posted the comment, my perception of the paper having clouded the memory of its wording.)
“Timeless decision theory (TDT) is an extension of causal decision networks that compactly represents uncertainty about correlated computational processes and represents the decision-maker as such a process.”
Warning: my comment here is based on dim memories and vague understandings. That said...
Are you sure that being an extension of causal decision networks is the same as being an extension of causal decision theory? I took “causal decision networks” to be referring to the Judea Pearl-inspired stuff. I thought that Eliezer said somewhere that CDT just assumes causal dependencies wherever it wants, while Eliezer derives them with causal networks.
I’m familiar with the link. I suggested adding the page number because that affords significant ease to finding the passage you claim is present, even given the pdf file.
Edit: To avoid this passive-aggressive beating around the bush: the part you’ve posted so far gives off the vibe of bibliography padding. You’ve done a great service to LW in your articles reminding everyone of the importance of building off the work of others. But if there’s a specific claim the author makes that you’re relying on, it helps establish relevance and make things easy on the reader if you are more specific than just citing the whole work
That is not standard practice when the point I’m citing the work for is in the abstract of the paper. Moreover, it’s just not true that it’s always standard practice to cite the page number of the specific work. I can point you to thousands of examples. That said, for books it is indeed helpful to cite page numbers.
Separate issues going on here: yes, it is common not to cite the specific page number, perhaps because one’s referring to the work “as a whole”. But my criticism of your paper having a vibe of bib-padding was not because of you saying “Yudkowsky (2010)” here, or any particular time in your excerpt, but rather, because taken as a whole, the excerpt looks that way. For example, there is much more citation than explanation of what the cited works are specifically there to substantiate.
Certainly, doing that a few times is fine, but here it looks more like what would result from someone trying to game the “bibliography length metric”.
Independently of that, your comment on this thread was supposedly to help someone find where a paper makes a claim that others didn’t remember the paper making. Regardless of the standards for academic papers, it is general etiquette that would suggest you give the page number in such a reply. And that shouldn’t be hard if you know the paper well enough to be using it for that claim.
So while “Yudkowsky (2010)” might be fine for the paper, when someone questions whether it makes a specific claim, you should do a little more than just give the naked citation a second time within the same thread—then you should give the page number.
I just disagree with all of this. The “explanation” stuff comes later; this is just the intro. If you read major works in Anglophone philosophy, you’ll see that what I have here is very much the same style. You can even compare to, for example, the extended abstract submitted by Salamon & Bostrom for an edited volume on the technological singularity.
And no, you don’t usually give the page number when the claims you’re saying a cited work covers are in the abstract of the paper. The hope is that someone will bother to at least read the abstract of the cited paper before bothering the original author with questions about “Which page number is that on?”
I would ask you to reconsider. As SilasBarta says, “Yudkowsky (2010)” is fine in the paper, but you used it in the comments here in response to someone’s question in this forum.
And no, you don’t usually give the page number when the claims you’re saying a cited work covers are in the abstract of the paper. The hope is that someone will bother to at least read the abstract of the cited paper before bothering the original author with questions about “Which page number is that on?”
“Um… page one?”
You seem to assume that the only way someone could have asked that question is if they hadn’t read even the abstract. But it is easy for me to imagine someone who read the whole paper, or some significant fraction, and just have missed or forgotten the claim that you attribute to Yudkowsky . In which case, saying “It’s on page one” would be helpful.
In fact, having read that significant fraction, I would be moderately surprised to hear Yudkowsky characterize TDT as an extension of CDT. He gave me the strong impression of offering an alternative to CDT, one which gets right answers where CDT is wrong. To me, calling TDT “an extension of CDT” implies that it applies to a wider range of problems than CDT, while agreeing with CDT where CDT gives a well-defined answer.
To me, calling TDT “an extension of CDT” implies that it applies to a wider range of problems than CDT, while agreeing with CDT where CDT gives a well-defined answer.
But this is a correct characterization of what TDT does. It extends applicability of CDT from action-determined to decision-determined problems.
Again, separate issues here that I think you’re blurring (of I’m just being unclear): I’m not criticizing you for lacking page numbers in the paper excerpt, but for piling on whole-work citations without clarifying what specific insight it adds. I will have to retract that in light of viewing this as an introduction, since you say your paper will cover that later.
WRT page numbers, my criticism was that when someone says, “hey, that claim about [citation of X which I have already read] doesn’t sound right”, then you should give a more helpful answer than “oh, that’s in [citation of X]”—you should point to a more specific passage.
And yes, it would have indeed been helpful if you had said “see the abstract”, because that would have told the questioner (and the onlooker, me, who was wondering the same thing) what you are basing that claim on. In this matter, for the reasons Vladimir gave, TDT isn’t best regarded as an extension of CDT. So a reply that you just got it from the abstract would show that (as turned out to be the case) your claim was based on a summary rather than on a specific analysis of the mechanics of TDT and its relationship to other decision theories.
Edit: I apologize for any abrasiveness in my comments here. I sensed a kind of stubbornness and condescension in your replies and overreacted to that.
WRT page numbers, my criticism was that when someone says, “hey, that claim about [citation of X which I have already read] doesn’t sound right”, then you should give a more helpful answer than “oh, that’s in [citation of X]”—you should point to a more specific passage.
Luke quoted the source and gave a link to it in digital form. If you want to find the context, open it up in Acrobat and copy the quote into the search bar.
It wasn’t in quote form, and like I just said, it matters which instance of the “TDT = extension of CDT” claim lukeprog had in mind. As it turns out, he was relying on a qualification-free summary, rather than the meat of the paper. If he had said so the first time Vladimir asked about it (rather than repeat the exact citation he already gave, and which the questioner had already read and already had the URL for), then he could have saved himself, me, and Vladimir the time it took to unravel this oversimplification (as it turned out to be).
“Find the passage that helps me the most” just isn’t good enough.
Luke’s comment has two paragraphs in quotes, and a link to Eliezer’s TDT paper. If you follow the link, you can copy either of the quoted paragraphs into Acrobats search bar, and it will show where in the document that paragraph appears.
Your first request for clarification that led to this comment was justified. But that should have satisfied your desire for a citation.
That wasn’t until ~3 rounds of back-and-forth! And I didn’t ask further after that; I just said why the previous answers weren’t as helpful. And while the desire for a citation may have been satisfied, the entire point was to reveal the level of understanding which led to his claim about the TDT paper, and that issue was not satisfied by lukeprog’s responses. His exchange with Vladimir, however, did show my concerns to be justified.
Again, if he had just simply said from the beginning, “Oh, I was just copying what the abstract said”, it would have saved everyone a lot of time. But instead, he decided to unhelpfully repeat a citation everyone already knew about, thus hiding his level of understanding for a few more rounds.
You asked for clarification once, and Luke gave a satisfactory response. How do you get “~3 rounds” from that?
I just said why the previous answers weren’t as helpful.
There was no reason for you to do that, you got an answer that addressed those concerns.
And while the desire for a citation may have been satisfied, the entire point was to reveal the level of understanding which led to his claim about the TDT paper, and that issue was not satisfied by lukeprog’s responses.
Notice how Nesov, by focusing back on the object level after the issue of communicating citations was resolved, was able to deal with in this comment.
I wouldn’t describe TDT as “an extension of CDT”. Also, the AI being self-modifying is probably not the central issue actually addressed by TDT (as opposed to what possibly motivated development of the ideas).
“An extension of CDT” is how Yudkowsky (2010) presents TDT. The paper also says one of its central goals is to handle self-modifying AI.
A page number, rather than a reminder of the publication year, would be a more helpful parenthetical there.
The first point is made in paragraph #1:
“Timeless decision theory (TDT) is an extension of causal decision networks that compactly represents uncertainty about correlated computational processes and represents the decision-maker as such a process.”
The second point is made in paragraph #2:
“I show that an evidential or causal decision-maker capable of self-modifying actions, given a choice between remaining an evidential or causal decision-maker and modifying itself to imitate a timeless decision-maker, will choose to imitate a timeless decisionmaker on a large class of problems.”
http://singinst.org/upload/TDT-v01o.pdf
After a fashion, since causal networks are not exactly CDT, modeling correlated computations with causal networks makes them less “causal” (i.e. related to physical causality), and the paper doesn’t achieve clear specification of how to do that (it’s an open problem, but I can say that any nontrivial causal network relating computations may need to be revised in face of new logical evidence, which makes the decision procedure that itself works with resolution of logical uncertainty brittle).
That CDT/EDT agents with self-modification would become more TDT-like is somewhat different from saying that TDT “suits the needs of a self-modifying AI”. TDT is a more sane theory, and to the extent CDT/EDT agents would prefer to be more effective, they’d prefer to adopt TDT’s greater sanity. But TDT is not a fixed point, suiting the needs of a self-modifying AI is a tall order that probably can’t be met to any reasonable extent, since that would mean establishing some rules that AI itself would not have much opportunity to rebuild. Perhaps laws of physics or logic can quality, with appropriate framing.
(I agree with your description more now than when I posted the comment, my perception of the paper having clouded the memory of its wording.)
Fair enough. Thanks for this. I’ve clarified the wording in my copy of this intro.
Warning: my comment here is based on dim memories and vague understandings. That said...
Are you sure that being an extension of causal decision networks is the same as being an extension of causal decision theory? I took “causal decision networks” to be referring to the Judea Pearl-inspired stuff. I thought that Eliezer said somewhere that CDT just assumes causal dependencies wherever it wants, while Eliezer derives them with causal networks.
I’m familiar with the link. I suggested adding the page number because that affords significant ease to finding the passage you claim is present, even given the pdf file.
Edit: To avoid this passive-aggressive beating around the bush: the part you’ve posted so far gives off the vibe of bibliography padding. You’ve done a great service to LW in your articles reminding everyone of the importance of building off the work of others. But if there’s a specific claim the author makes that you’re relying on, it helps establish relevance and make things easy on the reader if you are more specific than just citing the whole work
Re: your edit.
That is not standard practice when the point I’m citing the work for is in the abstract of the paper. Moreover, it’s just not true that it’s always standard practice to cite the page number of the specific work. I can point you to thousands of examples. That said, for books it is indeed helpful to cite page numbers.
Separate issues going on here: yes, it is common not to cite the specific page number, perhaps because one’s referring to the work “as a whole”. But my criticism of your paper having a vibe of bib-padding was not because of you saying “Yudkowsky (2010)” here, or any particular time in your excerpt, but rather, because taken as a whole, the excerpt looks that way. For example, there is much more citation than explanation of what the cited works are specifically there to substantiate.
Certainly, doing that a few times is fine, but here it looks more like what would result from someone trying to game the “bibliography length metric”.
Independently of that, your comment on this thread was supposedly to help someone find where a paper makes a claim that others didn’t remember the paper making. Regardless of the standards for academic papers, it is general etiquette that would suggest you give the page number in such a reply. And that shouldn’t be hard if you know the paper well enough to be using it for that claim.
So while “Yudkowsky (2010)” might be fine for the paper, when someone questions whether it makes a specific claim, you should do a little more than just give the naked citation a second time within the same thread—then you should give the page number.
I just disagree with all of this. The “explanation” stuff comes later; this is just the intro. If you read major works in Anglophone philosophy, you’ll see that what I have here is very much the same style. You can even compare to, for example, the extended abstract submitted by Salamon & Bostrom for an edited volume on the technological singularity.
And no, you don’t usually give the page number when the claims you’re saying a cited work covers are in the abstract of the paper. The hope is that someone will bother to at least read the abstract of the cited paper before bothering the original author with questions about “Which page number is that on?”
“Um… page one?”
I would ask you to reconsider. As SilasBarta says, “Yudkowsky (2010)” is fine in the paper, but you used it in the comments here in response to someone’s question in this forum.
You seem to assume that the only way someone could have asked that question is if they hadn’t read even the abstract. But it is easy for me to imagine someone who read the whole paper, or some significant fraction, and just have missed or forgotten the claim that you attribute to Yudkowsky . In which case, saying “It’s on page one” would be helpful.
In fact, having read that significant fraction, I would be moderately surprised to hear Yudkowsky characterize TDT as an extension of CDT. He gave me the strong impression of offering an alternative to CDT, one which gets right answers where CDT is wrong. To me, calling TDT “an extension of CDT” implies that it applies to a wider range of problems than CDT, while agreeing with CDT where CDT gives a well-defined answer.
But this is a correct characterization of what TDT does. It extends applicability of CDT from action-determined to decision-determined problems.
But CDT already gives well-defined answers to decision-determined problems such as Newcomb’s problem. They’re just not necessarily the right answers.
By “applies”, I mean “yields an output, which supporters claim is correct”, not “yields a correct output”.
Again, separate issues here that I think you’re blurring (of I’m just being unclear): I’m not criticizing you for lacking page numbers in the paper excerpt, but for piling on whole-work citations without clarifying what specific insight it adds. I will have to retract that in light of viewing this as an introduction, since you say your paper will cover that later.
WRT page numbers, my criticism was that when someone says, “hey, that claim about [citation of X which I have already read] doesn’t sound right”, then you should give a more helpful answer than “oh, that’s in [citation of X]”—you should point to a more specific passage.
And yes, it would have indeed been helpful if you had said “see the abstract”, because that would have told the questioner (and the onlooker, me, who was wondering the same thing) what you are basing that claim on. In this matter, for the reasons Vladimir gave, TDT isn’t best regarded as an extension of CDT. So a reply that you just got it from the abstract would show that (as turned out to be the case) your claim was based on a summary rather than on a specific analysis of the mechanics of TDT and its relationship to other decision theories.
Edit: I apologize for any abrasiveness in my comments here. I sensed a kind of stubbornness and condescension in your replies and overreacted to that.
Luke quoted the source and gave a link to it in digital form. If you want to find the context, open it up in Acrobat and copy the quote into the search bar.
It wasn’t in quote form, and like I just said, it matters which instance of the “TDT = extension of CDT” claim lukeprog had in mind. As it turns out, he was relying on a qualification-free summary, rather than the meat of the paper. If he had said so the first time Vladimir asked about it (rather than repeat the exact citation he already gave, and which the questioner had already read and already had the URL for), then he could have saved himself, me, and Vladimir the time it took to unravel this oversimplification (as it turned out to be).
“Find the passage that helps me the most” just isn’t good enough.
Luke’s comment has two paragraphs in quotes, and a link to Eliezer’s TDT paper. If you follow the link, you can copy either of the quoted paragraphs into Acrobats search bar, and it will show where in the document that paragraph appears.
Your first request for clarification that led to this comment was justified. But that should have satisfied your desire for a citation.
That wasn’t until ~3 rounds of back-and-forth! And I didn’t ask further after that; I just said why the previous answers weren’t as helpful. And while the desire for a citation may have been satisfied, the entire point was to reveal the level of understanding which led to his claim about the TDT paper, and that issue was not satisfied by lukeprog’s responses. His exchange with Vladimir, however, did show my concerns to be justified.
Again, if he had just simply said from the beginning, “Oh, I was just copying what the abstract said”, it would have saved everyone a lot of time. But instead, he decided to unhelpfully repeat a citation everyone already knew about, thus hiding his level of understanding for a few more rounds.
I don’t know why you’d want to defend that.
You asked for clarification once, and Luke gave a satisfactory response. How do you get “~3 rounds” from that?
There was no reason for you to do that, you got an answer that addressed those concerns.
Notice how Nesov, by focusing back on the object level after the issue of communicating citations was resolved, was able to deal with in this comment.
Pages 2 and 3.