Having a word [...] is a more compact code precisely in those cases where we can infer some of those properties from the other properties. (With the exception perhaps of very primitive words, like “red” [...]).
Remember that mutual information is symmetric. If some things have the property of being red, then “red” has the property of being a property of those things. Saying “blood is red” is really saying “remember that visual experience that you get when you look at certain roses, apples, peppers, lipsticks and English buses and phone booths? The same happens with blood.” If I give you the list above, can you find (“infer”) more red things? Then “red” is a good word.
But do note that this is a dual sense to the one in which “human” is a good word. Most of the properties of humans are statistically necessary for being human: remove any one of them, and the thing is much less likely to be human. “Human” is a good word because these properties are positively correlated. On the other hand, most of the red things are statistically sufficient for being red: take any one of them, and the thing is much more likely to be red. “Red” is a good word because these things are negatively correlated—they are a bunch of distinct things with a shared aspect.
Remember that mutual information is symmetric. If some things have the property of being red, then “red” has the property of being a property of those things. Saying “blood is red” is really saying “remember that visual experience that you get when you look at certain roses, apples, peppers, lipsticks and English buses and phone booths? The same happens with blood.” If I give you the list above, can you find (“infer”) more red things? Then “red” is a good word.
But do note that this is a dual sense to the one in which “human” is a good word. Most of the properties of humans are statistically necessary for being human: remove any one of them, and the thing is much less likely to be human. “Human” is a good word because these properties are positively correlated. On the other hand, most of the red things are statistically sufficient for being red: take any one of them, and the thing is much more likely to be red. “Red” is a good word because these things are negatively correlated—they are a bunch of distinct things with a shared aspect.