The difference between Hofstadter and Eliezer is that Hofstadter couldn’t make a convincing enough case for his assumptions, because he was talking about humans instead of AIs, and it’s just not clear that human decision procedures are similar enough to each other for his assumptions to hold. Eliezer also thought his ideas applied to humans, but he had a backup argument to the effect “even if you don’t think this applies to humans, at least it applies to AIs who know each others’ source code, so it’s still important to to work on” and that’s what convinced me.
BTW, for historical interest, I found a 2002 post by Hal Finney that came pretty close to some of the ideas behind TDT:
I have a problem with this application of game theory to a situation where
A and B both know that they are going to choose the same thing, which I
believe is the case here. [...]
[...] They are two instances
of the same deterministic calculation, with exactly the same steps being
executed for both.
[...] And the best of the two possible outcomes is when
both parties cooperate rather than defect.
I responded to Hal, and stated my agreement, but neither of us followed it up at the time. I even forgot about the post until I found it again yesterday, but I guess it must have influenced my thinking once Eliezer started talking about similar ideas.
The difference between Hofstadter and Eliezer is that Hofstadter couldn’t make a convincing enough case for his assumptions, because he was talking about humans instead of AIs, and it’s just not clear that human decision procedures are similar enough to each other for his assumptions to hold. Eliezer also thought his ideas applied to humans, but he had a backup argument to the effect “even if you don’t think this applies to humans, at least it applies to AIs who know each others’ source code, so it’s still important to to work on” and that’s what convinced me.
BTW, for historical interest, I found a 2002 post by Hal Finney that came pretty close to some of the ideas behind TDT:
I responded to Hal, and stated my agreement, but neither of us followed it up at the time. I even forgot about the post until I found it again yesterday, but I guess it must have influenced my thinking once Eliezer started talking about similar ideas.