“Stuart Armstrong does not believe this sentence.”
Aw, I happen to have a bit of difficulty in figuring out what proposition that desugars to in the language of Peano Arithmetic, could you help me out? :-)
(The serious point being, we know that you can write self-contradictory statements in English and we don’t expect to be able to assign consistent truth-values to them, but the statements of PA or the question whether a given Turing machine halts seem to us to have well-defined meaning, and if human-level intelligence is computable, it seems at least at first as if we should be able to encode “Stuart Armstrong believes proposition A” as a statement of PA. But the result won’t be anywhere as easily recognizable to him as what you wrote.)
But that sentence isn’t self-contradictory like “This is a lie”, it is just self-referential, like “This sentence has five words”. It does have a well-defined meaning and is decidable for all hypothetical consistent people other than hypothetical consitentified Stuart Armstrong.
You’re right, I didn’t think that one through, thanks!
I still think the interesting thing is the potential for writing down a mathematical statement humanity can’t decide, not an English one that we can’t decide even though it is meaningful, but I’ll shut up about the question for now.
Aw, I happen to have a bit of difficulty in figuring out what proposition that desugars to in the language of Peano Arithmetic, could you help me out? :-)
(The serious point being, we know that you can write self-contradictory statements in English and we don’t expect to be able to assign consistent truth-values to them, but the statements of PA or the question whether a given Turing machine halts seem to us to have well-defined meaning, and if human-level intelligence is computable, it seems at least at first as if we should be able to encode “Stuart Armstrong believes proposition A” as a statement of PA. But the result won’t be anywhere as easily recognizable to him as what you wrote.)
But that sentence isn’t self-contradictory like “This is a lie”, it is just self-referential, like “This sentence has five words”. It does have a well-defined meaning and is decidable for all hypothetical consistent people other than hypothetical consitentified Stuart Armstrong.
You’re right, I didn’t think that one through, thanks!
I still think the interesting thing is the potential for writing down a mathematical statement humanity can’t decide, not an English one that we can’t decide even though it is meaningful, but I’ll shut up about the question for now.