Your overall model isn’t far off, but your terminal value list needs some serious work. Also, human behavior is generally a better match for models that include a time parameter (such as Ainslie’s appetites model or PCT’s model of time-averaged perceptions) than simple utility-maximization models.
But these are relative quibbles; people do behave sort-of-as-if they were built according to your model. The biggest drawbacks to your model are:
The anthropomorphizing (neither the master nor the slave can truly be considered agents in their own right), and
You’ve drawn the dividing lines in the wrong place: the entire mechanism of reinforcement is part of the master, not the slave. The slave is largely a passive observer, abstract reasoner, and spokesperson, not an enslaved agent. To be the sort of slave you envision, we’d have to be actually capable of running the show without the “master”.
A better analogy would be to think of the “slave” as being a kind of specialized adjunct processor to the master, like a GPU chip on a computer, whose job is just to draw pretty pictures on the screen. (That’s what a big chunk of the slave is for, in fact: drawing pretty pictures to distract others from whatever the master is really up to.)
The slave also has a nasty tendency to attribute the master’s accomplishments, abilities, and choices to being its own doing… as can be seen in your depiction of the model, where you gave credit to the slave for huge chunks of what the master actually does. (The tendency to do this is—of course—another useful self/other-deception function, though!)
This isn’t the posted model at all but a confusing description of a different (not entirely incompatible except in some detail noted above) model using the post’s terminology.
It’s not a model. It’s a moral question about a simplified agent.
Um, the first sentence says:
In this post, I’ll describe a simple agent, a toy model,
I’m trying to point out that because of the model’s anthropomorphic (man, I hate trying to spell that word) tendencies, it would be a bad idea to try to draw moral conclusions from it.
It’d be an argument from confusion, because it just substitutes two homunculi (yay, a word I hate spelling even worse) for a human being, instead of actually reducing anything.
A correctly reductive model of human behavior needs to take into account that there is very little besides language in human behavior that is unique to humans… and that means that most of what we’re doing can be done by animals lacking in sentience. It would be a grave error to therefore conceive of the “slave” as being an individual, rather than a relatively minor set of add-on modules.
The question of whose preferences are “real” in that case is a confusion akin to asking how we can have free will if the universe is deterministic. That is, it’s yet another projection of our native/naive anthropomorphism—the inclination to label things as agents.
You can see this in the other part of the thread where you’re talking about what master and slave aliens would “want”—modeling these things as “wanting” is where the anthropomorphic injection is occurring. (E.g., in a human-accurate model, abstract wanting is not something the “master” would be capable of, as symbolic abstraction is the near-exclusive domain of the “slave”.)
I agree about substituting two homunculi for one without reducing anything being something to avoid, that the model in this post does it, and that pjeby’s model does not.
Your overall model isn’t far off, but your terminal value list needs some serious work. Also, human behavior is generally a better match for models that include a time parameter (such as Ainslie’s appetites model or PCT’s model of time-averaged perceptions) than simple utility-maximization models.
But these are relative quibbles; people do behave sort-of-as-if they were built according to your model. The biggest drawbacks to your model are:
The anthropomorphizing (neither the master nor the slave can truly be considered agents in their own right), and
You’ve drawn the dividing lines in the wrong place: the entire mechanism of reinforcement is part of the master, not the slave. The slave is largely a passive observer, abstract reasoner, and spokesperson, not an enslaved agent. To be the sort of slave you envision, we’d have to be actually capable of running the show without the “master”.
A better analogy would be to think of the “slave” as being a kind of specialized adjunct processor to the master, like a GPU chip on a computer, whose job is just to draw pretty pictures on the screen. (That’s what a big chunk of the slave is for, in fact: drawing pretty pictures to distract others from whatever the master is really up to.)
The slave also has a nasty tendency to attribute the master’s accomplishments, abilities, and choices to being its own doing… as can be seen in your depiction of the model, where you gave credit to the slave for huge chunks of what the master actually does. (The tendency to do this is—of course—another useful self/other-deception function, though!)
Your “drawbacks” point out ways in which Wei Dai’s model might differ from a human. But Wei Dai wasn’t trying to model a human.
This isn’t the posted model at all but a confusing description of a different (not entirely incompatible except in some detail noted above) model using the post’s terminology.
It’s not a model. It’s a moral question about a simplified agent.
Um, the first sentence says:
I’m trying to point out that because of the model’s anthropomorphic (man, I hate trying to spell that word) tendencies, it would be a bad idea to try to draw moral conclusions from it.
It’d be an argument from confusion, because it just substitutes two homunculi (yay, a word I hate spelling even worse) for a human being, instead of actually reducing anything.
A correctly reductive model of human behavior needs to take into account that there is very little besides language in human behavior that is unique to humans… and that means that most of what we’re doing can be done by animals lacking in sentience. It would be a grave error to therefore conceive of the “slave” as being an individual, rather than a relatively minor set of add-on modules.
The question of whose preferences are “real” in that case is a confusion akin to asking how we can have free will if the universe is deterministic. That is, it’s yet another projection of our native/naive anthropomorphism—the inclination to label things as agents.
You can see this in the other part of the thread where you’re talking about what master and slave aliens would “want”—modeling these things as “wanting” is where the anthropomorphic injection is occurring. (E.g., in a human-accurate model, abstract wanting is not something the “master” would be capable of, as symbolic abstraction is the near-exclusive domain of the “slave”.)
I agree about substituting two homunculi for one without reducing anything being something to avoid, that the model in this post does it, and that pjeby’s model does not.