I think you’re right. Parameterizing on PA(n) doesn’t really seem to do anything except perhaps give an illusion of self-improvement when there’s really a hard limit of PA(omega) that the AI can’t go beyond. I guess I choose it mostly because the OP was doing something similar.
If you just have PA(omega) and, say, parameterize on the proof limit instead, will your program double down on versions of itself that use weaker formal theories, not mentioned in your program?
Yes. If you call it with the argument AIQ2_T (i.e. the same algorithm but with T instead of PA(omega)), it will accept if it can find a proof that forall q,
AIQ2_T(q) != ‘self-destruct’
(which is easy) and
AIQ2_T(q) == ‘double’ ==> AIQ2(q)==‘double’
The latter condition expands to
PA(omega) |- someFormula ==> T |- someFormula
which may be quite easy to show, depending on T.
Basically, the Quining approach avoids trouble from the 2nd incompleteness theorem by using relative consistency instead of consistency.
It seems to me that the two programs don’t have the same “someFormula”, because each program’s formula uses a quined description of that program, but not the other one. Or maybe I’m wrong. Can you expand what you mean by “someFormula”?
I think you’re right. Parameterizing on PA(n) doesn’t really seem to do anything except perhaps give an illusion of self-improvement when there’s really a hard limit of PA(omega) that the AI can’t go beyond. I guess I choose it mostly because the OP was doing something similar.
If you just have PA(omega) and, say, parameterize on the proof limit instead, will your program double down on versions of itself that use weaker formal theories, not mentioned in your program?
Yes. If you call it with the argument AIQ2_T (i.e. the same algorithm but with T instead of PA(omega)), it will accept if it can find a proof that forall q,
AIQ2_T(q) != ‘self-destruct’
(which is easy) and
AIQ2_T(q) == ‘double’ ==> AIQ2(q)==‘double’
The latter condition expands to
PA(omega) |- someFormula ==> T |- someFormula
which may be quite easy to show, depending on T.
Basically, the Quining approach avoids trouble from the 2nd incompleteness theorem by using relative consistency instead of consistency.
It seems to me that the two programs don’t have the same “someFormula”, because each program’s formula uses a quined description of that program, but not the other one. Or maybe I’m wrong. Can you expand what you mean by “someFormula”?
Oops.