The problem with your “in practice” argument is that it would similarly imply that we can never know if someone is bald, since it is impossible to give a definition of baldness that rigidly separate bald people from non-bald people while respecting what we mean by the word. But in practice we can know that a particular person is bald regardless of the absence of that rigid definition. In the same way a particular person can know that he went to the store to buy milk, even if it is theoretically possible to explain what he did by saying that he has an abhorrence of milk and did it for totally different reasons.
Likewise, in practice we can avoid money pumps by avoiding them when they come up in practice. We don’t need to formulate principles which will guarantee that we will avoid them.
A person with less than 6% hair is bald, a person with 6% − 15% hair might be bald, but it is unknowable due to the nature of natural language. A person with 15% − 100% hair is not bald.
We can’t always say whether someone is bald, but more often, we can. Baldness remains applicable.
In the same way a particular person can know that he went to the store to buy milk
Yes. Isn’t this fascinating? What is going on in human minds that, not only can we say stuff about our own values and rationality, but about those of other humans? And can we copy that into an AI somehow?
What about a situation when a person says and thinks that he is going to buy a milk, but actually buy milk plus some sweets? And do it often, but do not acknowledge compulsive-obsessive behaviour towards sweets?
They don’t have to acknowledge compulsive-obsessive behavior. Obviously they want both milk and sweets, even if they don’t notice wanting the sweets. That doesn’t prevent other people from noticing it.
Also, they may be lying, since they might think that liking sweets is low status.
The problem with your “in practice” argument is that it would similarly imply that we can never know if someone is bald, since it is impossible to give a definition of baldness that rigidly separate bald people from non-bald people while respecting what we mean by the word. But in practice we can know that a particular person is bald regardless of the absence of that rigid definition. In the same way a particular person can know that he went to the store to buy milk, even if it is theoretically possible to explain what he did by saying that he has an abhorrence of milk and did it for totally different reasons.
Likewise, in practice we can avoid money pumps by avoiding them when they come up in practice. We don’t need to formulate principles which will guarantee that we will avoid them.
A person with less than 6% hair is bald, a person with 6% − 15% hair might be bald, but it is unknowable due to the nature of natural language. A person with 15% − 100% hair is not bald.
We can’t always say whether someone is bald, but more often, we can. Baldness remains applicable.
We can make similar answers about people’s intentions.
Yes. Isn’t this fascinating? What is going on in human minds that, not only can we say stuff about our own values and rationality, but about those of other humans? And can we copy that into an AI somehow?
That will be the subject of subsequent posts.
What about a situation when a person says and thinks that he is going to buy a milk, but actually buy milk plus some sweets? And do it often, but do not acknowledge compulsive-obsessive behaviour towards sweets?
They don’t have to acknowledge compulsive-obsessive behavior. Obviously they want both milk and sweets, even if they don’t notice wanting the sweets. That doesn’t prevent other people from noticing it.
Also, they may be lying, since they might think that liking sweets is low status.