Any “categories” you introduce here are at best helpful heuristics, with no deep philosophical significance.
I mean, yes, but I was imagining that there would be some deep philosophy about how computationally bounded agents should construct optimally helpful heuristics.
Alright, but then you need some (at least informal) model of why computationally bounded agents need categories. Instead, your argument seems to rely purely on the intuition of your fictional character (“you notice that… they seem to occupy a third category in your ontology of sortable objects”).
Also, you seem to assume that categories are non-overlapping. You write “you don’t really put them in the same mental category as bleggs”. What does it even mean, to put two objects in the same or not the same category? Consider a horse and a cow. Are they in the same mental category? Both are in the categories “living organisms”, “animals”, “mammals”, “domesticated mammals”. But, they are different species. So, sometimes you put them in the same category, sometimes you put them in different categories. Are “raven” and “F16 aircraft” in the same category? They are if your categories are “flying objects” vs. “non-flying objects”, but they aren’t if your categories are “animate” vs. “non-animate”.
Moreover, you seem to assume that categories are crisp rather than fuzzy, which is almost never the case for categories that people actually use. How many coins does it take to make a “pile” of coins? Is there an exact number? Is there an exact age when a person gets to be called “old”? If you take a table made out of a block of wood, and start to gradually deform its shape until it becomes perfectly spherical, is there an exact point when it is no longer called a “table”? So, “rubes” and “bleggs” can be fuzzy categories, and the anomalous objects are in the gray area that defies categorization. There’s nothing wrong with that.
If we take this rube/blegg factory thought experiment seriously, then what we need to imagine is the algorithm (instructions) that the worker in the factory executes. Then you can say that the relevant “categories” (in the context of the factory, and in that context only) are the vertices in the flow graph of the algorithm. For example, the algorithm might be, a table that specifies how to score each object (blue +5 points, egg-shaped +10 points, furry +1 point...) and a threshold which says what the score should to be to put it in a given bin. Then there are essentially only two categories. Another algorithm might be “if object passes test X, put in the rube bin, if object passes test Y, put it in the blegg bin, if object passes neither test, put in in the Palladium scanner and sort according to that”. Then, you have approximately seven categories: “regular rube” (passed test X), “regular blegg” (passed test Y), “irregular object” (failed both tests), “irregular rube” (failed both tests and found to contain enough Palladium), “irregular blegg” (failed both tests and found to contain not enough Palladium), “rube” (anything put in the rube bin) and “blegg” (anything put in the blegg bin). But in any case, the categorization would depend on the particular trade-offs that the designers of the production line made (depending on things like, how expensive is it to run the palladium scanner), rather than immutable Platonic truths about the nature of the objects themselves.
Then again, I’m not entirely sure whether we are really disagreeing or just formulating the same thing in different ways?
your argument seems to rely purely on the intuition of your fictional character
Yes, the dependence on intuition is definitely a weakness of this particular post. (I wish I knew as much math as Jessica Taylor! If I want to become stronger, I’ll have to figure out how fit more studying into my schedule!)
you seem to assume that categories are non-overlapping.
you seem to assume that categories are crisp rather than fuzzy
I don’t believe either of those things. If you have any specific wording suggestions on how I can write more clearly so as to better communicate to my readers that I don’t believe either of those things, I’m listening.
If you take a table made out of a block of wood, and start to gradually deform its shape until it becomes perfectly spherical, is there an exact point when it is no longer called a “table”?
But in any case, the categorization would depend on the particular trade-offs that the designers of the production line made (depending on things like, how expensive is it to run the palladium scanner)
Right. Another example of one of the things the particular algorithm-design trade-offs will depend on is the distribution of objects.
We could imagine a slightly altered parable in which the frequency distribution of objects is much more evenly spread out in color–shape–metal-content space: while cubeness has a reasonably strong correlation with redness and palladium yield, and eggness with blueness and vanadium yield, you still have a substantial fraction of non-modal objects: bluish-purple rounded cubes, reddish-purple squarish eggs, &c.
In that scenario, a natural-language summary of the optimal decision algorithm wouldn’t talk about discrete categories: you’d probably want some kind of scoring algorithm with thresholds for various tests and decisions as you describe, and no matter where you set the threshold for each decision, you’d still see a lot of objects just on either side of the boundary, with no good “joint” to anchor the placement of a category boundary.
In contrast, my reading of Yudkowsky’s original parable posits a much sparser, more tightly-clustered distribution of objects in configuration space. The objects do vary somewhat (some bleggs are purple, some rubes contain vanadium), but there’s a very clear cluster-structure: virtually all objects are close to the center of—and could be said to “belong to”—either the “rube” cluster or the “blegg” cluster, with a lot of empty space in between.
In this scenario, I think it does make sense for a natural-language summary of the optimal decision algorithm to talk about two distinct “categories” where the density in the configuration space is concentrated. Platonic essences are just the limiting case as the overlap between clusters goes to zero.
In my fanfiction, I imagine that some unknown entity has taken objects that were originally in the “rube” cluster, and modified them so that they appear, at first glance but not on closer inspection, to be members of the “blegg” cluster. At first, the protagonist wishes to respect the apparent intent of the unknown entity by considering the modified objects to be bleggs. But in the process of her sorting work, the protagonist finds herself wanting to mentally distinguish adapted bleggs from regular bleggs, because she can’t make the same job-relevant probabilistic inferences with the new “bleggs (either regular or adapted)” concept as she could with the old “bleggs (only standard bleggs)” concept.
To see why, forget about the category labels for a moment and just consider the clusters in the six-dimensional color–shape–texture–firmness–luminesence–metal-content configuration space.
Before the unknown entity’s intervention, we had two distinct clusters: one centered at {blue, egg, furry, flexible, luminescent, vanadium}, and another centered at {red, cube, smooth, hard, non-luminescent, palladium}.
After the unknown entity’s intervention, we have three distinct clusters: the two previously-existing clusters, and a new cluster centered at {blue, egg, furry, hard, non-luminescent, palladium}. This is a different situation! Workers on the sorting line might want different language in order to describe this new reality!
Now, if we were to project into the three-dimensional color–shape–texture subspace, then we would have two clusters again: with just these attributes, we can’t distinguish between bleggs and adapted bleggs. But since workers on the sorting line can observe hardness, and care about metal content, they probably want to use the three-cluster representation, even if they suspect the unknown entity might thereby feel disrespected.
Hmm. Why would the entity feel disrespected by how many clusters the workers use? I actually am aware that this is an allegory for something else. Moreover, I think that I disagree you with about the something else (although I am not sure since I am not entirely sure what’s your position about the something else is). Which is to say, I think that this allegory misses crucial aspects of the original situation and loses the crux of the debate.
I think that this allegory misses crucial aspects of the original situation
That makes sense! As gjm noted, sometimes unscrupulous authors sneakily construct an allegory with the intent of leading the reader to a particular conclusion within the context of the allegory with the hope that the reader will map that conclusion back onto the real-world situation in a particular way, without doing the work of actually showing that the allegory and the real-world situation are actually analogous in the relevant aspects.
I don’t want to be guilty of that! This is a story about bleggs and rubes that I happened to come up with in the context of trying to think about something else (and I don’t want to be deceptive about that historical fact), but I definitely agree that people shouldn’t map the story onto some other situation unless they actually have a good argument for why that mapping makes sense. If we wanted to discuss the something else rather than the bleggs and rubes, we should do that on someone else’s website. Not here.
I mean, yes, there’s the allusion in the title! (The post wasn’t originally written for being shared on Less Wrong, it just seemed sufficiently sanitized to be shareable-here-without-running-too-afoul-of-anti-politics-norms after the fact.)
I read the title as just an allusion to Eliezer’s OP on bleggs and rubes. (Otoh, without having read the article just linked, I’m familiar with “egg” as transsexual jargon for someone exploring TS feelings, who (the ideology has it) will inevitably in the end “hatch” into a full-on TS.)
I mean, yes, but I was imagining that there would be some deep philosophy about how computationally bounded agents should construct optimally helpful heuristics.
Alright, but then you need some (at least informal) model of why computationally bounded agents need categories. Instead, your argument seems to rely purely on the intuition of your fictional character (“you notice that… they seem to occupy a third category in your ontology of sortable objects”).
Also, you seem to assume that categories are non-overlapping. You write “you don’t really put them in the same mental category as bleggs”. What does it even mean, to put two objects in the same or not the same category? Consider a horse and a cow. Are they in the same mental category? Both are in the categories “living organisms”, “animals”, “mammals”, “domesticated mammals”. But, they are different species. So, sometimes you put them in the same category, sometimes you put them in different categories. Are “raven” and “F16 aircraft” in the same category? They are if your categories are “flying objects” vs. “non-flying objects”, but they aren’t if your categories are “animate” vs. “non-animate”.
Moreover, you seem to assume that categories are crisp rather than fuzzy, which is almost never the case for categories that people actually use. How many coins does it take to make a “pile” of coins? Is there an exact number? Is there an exact age when a person gets to be called “old”? If you take a table made out of a block of wood, and start to gradually deform its shape until it becomes perfectly spherical, is there an exact point when it is no longer called a “table”? So, “rubes” and “bleggs” can be fuzzy categories, and the anomalous objects are in the gray area that defies categorization. There’s nothing wrong with that.
If we take this rube/blegg factory thought experiment seriously, then what we need to imagine is the algorithm (instructions) that the worker in the factory executes. Then you can say that the relevant “categories” (in the context of the factory, and in that context only) are the vertices in the flow graph of the algorithm. For example, the algorithm might be, a table that specifies how to score each object (blue +5 points, egg-shaped +10 points, furry +1 point...) and a threshold which says what the score should to be to put it in a given bin. Then there are essentially only two categories. Another algorithm might be “if object passes test X, put in the rube bin, if object passes test Y, put it in the blegg bin, if object passes neither test, put in in the Palladium scanner and sort according to that”. Then, you have approximately seven categories: “regular rube” (passed test X), “regular blegg” (passed test Y), “irregular object” (failed both tests), “irregular rube” (failed both tests and found to contain enough Palladium), “irregular blegg” (failed both tests and found to contain not enough Palladium), “rube” (anything put in the rube bin) and “blegg” (anything put in the blegg bin). But in any case, the categorization would depend on the particular trade-offs that the designers of the production line made (depending on things like, how expensive is it to run the palladium scanner), rather than immutable Platonic truths about the nature of the objects themselves.
Then again, I’m not entirely sure whether we are really disagreeing or just formulating the same thing in different ways?
Yes, the dependence on intuition is definitely a weakness of this particular post. (I wish I knew as much math as Jessica Taylor! If I want to become stronger, I’ll have to figure out how fit more studying into my schedule!)
I don’t believe either of those things. If you have any specific wording suggestions on how I can write more clearly so as to better communicate to my readers that I don’t believe either of those things, I’m listening.
No, there is no such exact point; like many longtime Less Wrong readers, I, too, am familiar with the Sorities paradox.
Right. Another example of one of the things the particular algorithm-design trade-offs will depend on is the distribution of objects.
We could imagine a slightly altered parable in which the frequency distribution of objects is much more evenly spread out in color–shape–metal-content space: while cubeness has a reasonably strong correlation with redness and palladium yield, and eggness with blueness and vanadium yield, you still have a substantial fraction of non-modal objects: bluish-purple rounded cubes, reddish-purple squarish eggs, &c.
In that scenario, a natural-language summary of the optimal decision algorithm wouldn’t talk about discrete categories: you’d probably want some kind of scoring algorithm with thresholds for various tests and decisions as you describe, and no matter where you set the threshold for each decision, you’d still see a lot of objects just on either side of the boundary, with no good “joint” to anchor the placement of a category boundary.
In contrast, my reading of Yudkowsky’s original parable posits a much sparser, more tightly-clustered distribution of objects in configuration space. The objects do vary somewhat (some bleggs are purple, some rubes contain vanadium), but there’s a very clear cluster-structure: virtually all objects are close to the center of—and could be said to “belong to”—either the “rube” cluster or the “blegg” cluster, with a lot of empty space in between.
In this scenario, I think it does make sense for a natural-language summary of the optimal decision algorithm to talk about two distinct “categories” where the density in the configuration space is concentrated. Platonic essences are just the limiting case as the overlap between clusters goes to zero.
In my fanfiction, I imagine that some unknown entity has taken objects that were originally in the “rube” cluster, and modified them so that they appear, at first glance but not on closer inspection, to be members of the “blegg” cluster. At first, the protagonist wishes to respect the apparent intent of the unknown entity by considering the modified objects to be bleggs. But in the process of her sorting work, the protagonist finds herself wanting to mentally distinguish adapted bleggs from regular bleggs, because she can’t make the same job-relevant probabilistic inferences with the new “bleggs (either regular or adapted)” concept as she could with the old “bleggs (only standard bleggs)” concept.
To see why, forget about the category labels for a moment and just consider the clusters in the six-dimensional color–shape–texture–firmness–luminesence–metal-content configuration space.
Before the unknown entity’s intervention, we had two distinct clusters: one centered at {blue, egg, furry, flexible, luminescent, vanadium}, and another centered at {red, cube, smooth, hard, non-luminescent, palladium}.
After the unknown entity’s intervention, we have three distinct clusters: the two previously-existing clusters, and a new cluster centered at {blue, egg, furry, hard, non-luminescent, palladium}. This is a different situation! Workers on the sorting line might want different language in order to describe this new reality!
Now, if we were to project into the three-dimensional color–shape–texture subspace, then we would have two clusters again: with just these attributes, we can’t distinguish between bleggs and adapted bleggs. But since workers on the sorting line can observe hardness, and care about metal content, they probably want to use the three-cluster representation, even if they suspect the unknown entity might thereby feel disrespected.
Hmm. Why would the entity feel disrespected by how many clusters the workers use? I actually am aware that this is an allegory for something else. Moreover, I think that I disagree you with about the something else (although I am not sure since I am not entirely sure what’s your position about the something else is). Which is to say, I think that this allegory misses crucial aspects of the original situation and loses the crux of the debate.
That makes sense! As gjm noted, sometimes unscrupulous authors sneakily construct an allegory with the intent of leading the reader to a particular conclusion within the context of the allegory with the hope that the reader will map that conclusion back onto the real-world situation in a particular way, without doing the work of actually showing that the allegory and the real-world situation are actually analogous in the relevant aspects.
I don’t want to be guilty of that! This is a story about bleggs and rubes that I happened to come up with in the context of trying to think about something else (and I don’t want to be deceptive about that historical fact), but I definitely agree that people shouldn’t map the story onto some other situation unless they actually have a good argument for why that mapping makes sense. If we wanted to discuss the something else rather than the bleggs and rubes, we should do that on someone else’s website. Not here.
FWIW, I predicted it would be an allegory of transsexuality even before I read it or any of the comments.
I mean, yes, there’s the allusion in the title! (The post wasn’t originally written for being shared on Less Wrong, it just seemed sufficiently sanitized to be shareable-here-without-running-too-afoul-of-anti-politics-norms after the fact.)
I read the title as just an allusion to Eliezer’s OP on bleggs and rubes. (Otoh, without having read the article just linked, I’m familiar with “egg” as transsexual jargon for someone exploring TS feelings, who (the ideology has it) will inevitably in the end “hatch” into a full-on TS.)