“What is the you” is part of the question. Consider it in terms of counterfactuals. Agent A is told via quining that it has source code S. We’re interested in how to implement A so that it outputs “yes” if S is really its source code, but would output “no” if S were changed to S’ while leaving the rest of the agent unchanged.
In this formulation the problem seems to be impossible to solve, unless the agent has access to an external “reader oracle” that can just read its source code back to it. Guess that answers my original question, then.
“What is the you” is part of the question. Consider it in terms of counterfactuals. Agent A is told via quining that it has source code S. We’re interested in how to implement A so that it outputs “yes” if S is really its source code, but would output “no” if S were changed to S’ while leaving the rest of the agent unchanged.
In this formulation the problem seems to be impossible to solve, unless the agent has access to an external “reader oracle” that can just read its source code back to it. Guess that answers my original question, then.