I ask this more as a question, not as a statement, because I’m not terribly familiar with TDT or AI (I’m just an engineering student who’s maybe 1⁄3 of the way through the sequences) but is there any conflict between TDT and self modification? Suppose we modify Newcomb’s problem somewhat, and say that Omega is predicting whether or not you were a one boxer a year ago. Suppose an AI was, in fact, a two boxer a year ago and self modified to be a one boxer now. Since Omega could simply read the code, it would know this. But by my understanding, a TDT AI would still one box in this situation, which would lose.
Supposedly we want a new, reflective decision theory because you can’t have “a CDT agent who can self-modify.”
I ask this more as a question, not as a statement, because I’m not terribly familiar with TDT or AI (I’m just an engineering student who’s maybe 1⁄3 of the way through the sequences) but is there any conflict between TDT and self modification? Suppose we modify Newcomb’s problem somewhat, and say that Omega is predicting whether or not you were a one boxer a year ago. Suppose an AI was, in fact, a two boxer a year ago and self modified to be a one boxer now. Since Omega could simply read the code, it would know this. But by my understanding, a TDT AI would still one box in this situation, which would lose.