Yeah, I think that you got it right. The Gödel sentence of a person must be a thought that that person can think, and it will refer to that person’s thought processes. So something like “You don’t believe this” or “You don’t understand this idea”.
“Stuart Armstrong does not believe this sentence.”
Aw, I happen to have a bit of difficulty in figuring out what proposition that desugars to in the language of Peano Arithmetic, could you help me out? :-)
(The serious point being, we know that you can write self-contradictory statements in English and we don’t expect to be able to assign consistent truth-values to them, but the statements of PA or the question whether a given Turing machine halts seem to us to have well-defined meaning, and if human-level intelligence is computable, it seems at least at first as if we should be able to encode “Stuart Armstrong believes proposition A” as a statement of PA. But the result won’t be anywhere as easily recognizable to him as what you wrote.)
But that sentence isn’t self-contradictory like “This is a lie”, it is just self-referential, like “This sentence has five words”. It does have a well-defined meaning and is decidable for all hypothetical consistent people other than hypothetical consitentified Stuart Armstrong.
You’re right, I didn’t think that one through, thanks!
I still think the interesting thing is the potential for writing down a mathematical statement humanity can’t decide, not an English one that we can’t decide even though it is meaningful, but I’ll shut up about the question for now.
Doesn’t that allow for Stuart Armstrong to have a false belief?
“Stuart Armstrong cannot justify a belief in this sentence” might be a stronger form since it implicitly refers to any logical or rational reasoning that might be used to justify the belief.
There are some obvious weaknesses to human Godel sentences. Stuart Armstrong is not a permanent name. Additionally our biology changes over time so we probably can’t have permanent personal Godel sentences.
“Stuart Armstrong does not believe this sentence.”
Yeah, I think that you got it right. The Gödel sentence of a person must be a thought that that person can think, and it will refer to that person’s thought processes. So something like “You don’t believe this” or “You don’t understand this idea”.
Aw, I happen to have a bit of difficulty in figuring out what proposition that desugars to in the language of Peano Arithmetic, could you help me out? :-)
(The serious point being, we know that you can write self-contradictory statements in English and we don’t expect to be able to assign consistent truth-values to them, but the statements of PA or the question whether a given Turing machine halts seem to us to have well-defined meaning, and if human-level intelligence is computable, it seems at least at first as if we should be able to encode “Stuart Armstrong believes proposition A” as a statement of PA. But the result won’t be anywhere as easily recognizable to him as what you wrote.)
But that sentence isn’t self-contradictory like “This is a lie”, it is just self-referential, like “This sentence has five words”. It does have a well-defined meaning and is decidable for all hypothetical consistent people other than hypothetical consitentified Stuart Armstrong.
You’re right, I didn’t think that one through, thanks!
I still think the interesting thing is the potential for writing down a mathematical statement humanity can’t decide, not an English one that we can’t decide even though it is meaningful, but I’ll shut up about the question for now.
Doesn’t that allow for Stuart Armstrong to have a false belief?
“Stuart Armstrong cannot justify a belief in this sentence” might be a stronger form since it implicitly refers to any logical or rational reasoning that might be used to justify the belief.
There are some obvious weaknesses to human Godel sentences. Stuart Armstrong is not a permanent name. Additionally our biology changes over time so we probably can’t have permanent personal Godel sentences.