Staring into the abyss means thinking reasonably about things that are uncomfortable to contemplate, like arguments against your religious beliefs, or in favor of breaking up with your partner. It’s common to procrastinate on thinking hard about these things because it might require you to acknowledge that you were very wrong about something in the past, and perhaps wasted a bunch of time based on that (e.g. dating the wrong person or praying to the wrong god).
I strongly disagree with this, for the reason that words are not reality. For most people, their relationships and their gods are load bearing for unarticulated reasons. arguments are going to outcompete your sacred pillars in the world of words and arguments. Challenging yourself to stare into the abyss (aka reduce your unarticulated beliefs) is more likely than not to conclude that your sacred beliefs are somehow wrong. in high dimensional space odds are that there is some suboptimal dimension.
This is great, and I love the specific example of trees as a failure to classify a large set into subsets.
Something that’s not exactly the same problem, but rhymes, is that of genre classification for content discovery. Consider Spotify playlists. There are millions of songs, and hundreds of classified genres. Genres are classified much like species/genus taxonomies— two songs share a genre if they share a common ancestor of music. Led Zeppelin and the Beatles are different, but they both derive from traditions of electric guitar which grew out of the blues which… and so on. So we say Led Zeppelin and the Beatles are the same genre, “Rock”. You can do this kind of classification in much greater detail to carve out new genres and subgenres.
However when it comes to utility and discovery, genres underperform. Despite being the same genre, there are few parties which shuffle between the melancholic “Yesterday” and the screaming “Ramble On”. People seek songs which are similar to others in strategy, NOT in tradition. As you said:
Successful user created playlists on Spotify (ones which are public with lots of likes) tend NOT to use genre. The tend to be called something line “rainy tuesday drive home from work”, or “music school nerds playing ALL the notes”. Rather than carve out a subset using genre (a playlist is a subset of music), they define it by strategy.
A failure in taxonomy.