Shane: I mean differentiation in the sense of differentiating between the abstract categories.
The abstract categories? This sounds like a unique categorization that the AI just has to find-in-the-world. You keep speaking of “good” abstractions as if this were a property of the categories themselves, rather than a ranking in your preference ordering relative to some decision task that makes use of the categories.
Shane: I mean differentiation in the sense of differentiating between the abstract categories.
The abstract categories? This sounds like a unique categorization that the AI just has to find-in-the-world. You keep speaking of “good” abstractions as if this were a property of the categories themselves, rather than a ranking in your preference ordering relative to some decision task that makes use of the categories.