This one came up at the recent London meetup and I’m curious what everyone here thinks:
What would happen if CEV was applied to the Baby Eaters?
My thoughts are that if you applied it to all baby eaters, including the living babies and the ones being digested, it would end up in a place that adult baby eaters would not be happy. If you expanded it to include all babyeaters that ever existed, or that would ever exist, knowing the fate of 99% of them, it would be a much more pronounced effect. So what I make of all this is that either CEV is not utility-function-neutral, or that the babyeater morality is objectively unstable when aggregated.
What would happen if CEV was applied to the Baby Eaters?
My intuitions of CEV are informed by the Rawlsian Veil of Ignorance, which effectively asks: “What rules would you want to prevail if you didn’t know in advance who you would turn out to be?”
Where CEV as I understand it adds more information—assumes our preferences are extrapolated as if we knew more, were more the kind of people we want to be—the Veil of Ignorance removes information: it strips people under a set of specific circumstances of the detailed information about what their preferences are, what their contignent histories brought them there, and so on. This includes things like what age you are, and even—conceivably—how many of you there are.
To this bunch of undifferentiated people you’d put the question, “All in favor of a 99% chance of dying horribly shortly after being born, in return for the 1% chance to partake in the crowning glory of babyeating cultural tradition, please raise your hands.”
I expect that not dying horribly takes lexical precedence over any kind of cultural tradition, for any sentient being whose kin has evolved to sentience (it may not be that way for constructed minds). So I would expect the Babyeaters to choose against cultural tradition.
The obvious caveat is that my intuitions about CEV may be wrong, but lacking a formal explanation of CEV it’s hard to check intuitions.
You’re correct. I’m using the term “people” loosely. However, I wrote the grand-parent while fully informed of what the Babyeaters are. Did you mean to rebut something in particular in the above?
“All in favor of a 99% chance of dying horribly shortly after being born, in return for the 1% chance to partake in the crowning glory of babyeating cultural tradition, please raise your hands.”
If we translate it to our cultural context, we will get something like “All in favor of 100% dying horribly of old age, in return for good lives of your babies, please rise your hands”. They ARE aliens.
Well, we would say “no” to that, if we had the means to abolish old age. We’d want to have our cake and eat it too.
The text stipulates that it is within the BE’s technological means to abolish the suffering of the babies, so I expect that they would choose to do so, behind the Veil.
Who will ask them? FAI have no idea, that a) baby eating is bad, b) it should generalize moral values past BE to all conscious beings.
Even if FAI will ask that question and it turns out that majority of population don’t want to do inherently good thing (it is for them), then FAI must undergo controlled shutdown.
EDIT: To disambiguate. I am talking about FAI, which is implemented by BEs.
As we should not allow FAI to generalize morals past conscious beings, just to be sure, that it will not take CEV of all bacterium, so BEs should not allow their FAI to generalize past BEs.
As we should built in automatic off switch into our FAI, to stop it if its goals is inherently wrong, so should BEs.
It doesn’t seem from the story like the babies are gladly sacrificing for the tribe...
“But...” said the Master. “But, my Lady, if they want to be eaten—”
“They don’t,” said the Xenopsychologist. “Of course they don’t. They run from their parents when the terrible winnowing comes. The Babyeater children aren’t emotionally mature—I mean they don’t have their adult emotional state yet. Evolution would take care of anyone who wanted to get eaten. And they’re still learning, still making mistakes, so they don’t yet have the instinct to exterminate violators of the group code. It’s a simpler time for them. They play, they explore, they try out new ideas. They’re...” and the Xenopsychologist stopped. “Damn,” she said, and turned her head away from the table, covering her face with her hands. “Excuse me.” Her voice was unsteady. “They’re a lot like human children, really.”
Yes. It’s horrible. For us. But why FAI should place any weight on removing that? How FAI can generalize past “Life of Baby Eater is sacred” to “Life of every conscious being is sacred”? FAI has all evidence that latter is plain wrong.
Do You want convince me or FAI that it’s bad? I know that it is, I just try to demonstrate that FAI as it is, is about preservation and not development to (universally) better ends.
Why? There must be very strong arguments for BEs to stop doing the Right Thing. And there’s only one source of objections—children. And their volitions will be selfish and unaggregatable.
EDIT: What does utility-function-neutral mean?
EDIT: Ok. Ok. CEV will be to make BE’s morale change and allow them to not eat children. So, FAI will undergo controlled shutdown. Objections, please?
EDIT: Here’s yet another arguments.
Guidelines of FAI as of may 2004.
Defend humans, the future of humankind, and humane nature.
BEs will formulate this as “Defend BEs (except for the ceremony of BEing), the future of BEkind, and BE’s nature.”
Encapsulate moral growth.
BEs never considered, that child eating is bad. And it is good for them to kill anyone who thinks otherwise.
There’s no trend in moral that can be encapsulated.
Humankind should not spend the rest of eternity desperately wishing that the programmers had done something differently.
If they stop being BE they will mourn their wrong doings to the death.
Avoid creating a motive for modern-day humans to fight over the initial dynamic
Every single notion that FAI will make in lines of “Let’s suppose that you are non-BE” will cause it to be destroyed.
Help people.
Help BEs everytime, but the ceremony of BEing.
How this will take FAI to the point that every conscious being must live?
This one came up at the recent London meetup and I’m curious what everyone here thinks:
What would happen if CEV was applied to the Baby Eaters?
My thoughts are that if you applied it to all baby eaters, including the living babies and the ones being digested, it would end up in a place that adult baby eaters would not be happy. If you expanded it to include all babyeaters that ever existed, or that would ever exist, knowing the fate of 99% of them, it would be a much more pronounced effect. So what I make of all this is that either CEV is not utility-function-neutral, or that the babyeater morality is objectively unstable when aggregated.
Thoughts?
My intuitions of CEV are informed by the Rawlsian Veil of Ignorance, which effectively asks: “What rules would you want to prevail if you didn’t know in advance who you would turn out to be?”
Where CEV as I understand it adds more information—assumes our preferences are extrapolated as if we knew more, were more the kind of people we want to be—the Veil of Ignorance removes information: it strips people under a set of specific circumstances of the detailed information about what their preferences are, what their contignent histories brought them there, and so on. This includes things like what age you are, and even—conceivably—how many of you there are.
To this bunch of undifferentiated people you’d put the question, “All in favor of a 99% chance of dying horribly shortly after being born, in return for the 1% chance to partake in the crowning glory of babyeating cultural tradition, please raise your hands.”
I expect that not dying horribly takes lexical precedence over any kind of cultural tradition, for any sentient being whose kin has evolved to sentience (it may not be that way for constructed minds). So I would expect the Babyeaters to choose against cultural tradition.
The obvious caveat is that my intuitions about CEV may be wrong, but lacking a formal explanation of CEV it’s hard to check intuitions.
BEs aren’t humans. They are Baby-Eating aliens
You’re correct. I’m using the term “people” loosely. However, I wrote the grand-parent while fully informed of what the Babyeaters are. Did you mean to rebut something in particular in the above?
If we translate it to our cultural context, we will get something like “All in favor of 100% dying horribly of old age, in return for good lives of your babies, please rise your hands”. They ARE aliens.
Well, we would say “no” to that, if we had the means to abolish old age. We’d want to have our cake and eat it too.
The text stipulates that it is within the BE’s technological means to abolish the suffering of the babies, so I expect that they would choose to do so, behind the Veil.
Yes, but a surprisingly large number of humans seem to react in horror when you talk about getting rid of aging.
Who will ask them? FAI have no idea, that a) baby eating is bad, b) it should generalize moral values past BE to all conscious beings.
Even if FAI will ask that question and it turns out that majority of population don’t want to do inherently good thing (it is for them), then FAI must undergo controlled shutdown.
EDIT: To disambiguate. I am talking about FAI, which is implemented by BEs.
As we should not allow FAI to generalize morals past conscious beings, just to be sure, that it will not take CEV of all bacterium, so BEs should not allow their FAI to generalize past BEs.
As we should built in automatic off switch into our FAI, to stop it if its goals is inherently wrong, so should BEs.
It doesn’t seem from the story like the babies are gladly sacrificing for the tribe...
Yes. It’s horrible. For us. But why FAI should place any weight on removing that? How FAI can generalize past “Life of Baby Eater is sacred” to “Life of every conscious being is sacred”? FAI has all evidence that latter is plain wrong.
Do You want convince me or FAI that it’s bad? I know that it is, I just try to demonstrate that FAI as it is, is about preservation and not development to (universally) better ends.
Correct. CEV is supposed to be a component of Friendliness, which is defined in reference to human values.
CEV will be to maintain existing order.
Why? There must be very strong arguments for BEs to stop doing the Right Thing. And there’s only one source of objections—children. And their volitions will be selfish and unaggregatable.
EDIT: What does utility-function-neutral mean?
EDIT: Ok. Ok. CEV will be to make BE’s morale change and allow them to not eat children. So, FAI will undergo controlled shutdown. Objections, please?
EDIT: Here’s yet another arguments.
Guidelines of FAI as of may 2004.
BEs will formulate this as “Defend BEs (except for the ceremony of BEing), the future of BEkind, and BE’s nature.”
BEs never considered, that child eating is bad. And it is good for them to kill anyone who thinks otherwise. There’s no trend in moral that can be encapsulated.
If they stop being BE they will mourn their wrong doings to the death.
Every single notion that FAI will make in lines of “Let’s suppose that you are non-BE” will cause it to be destroyed.
Help BEs everytime, but the ceremony of BEing.
How this will take FAI to the point that every conscious being must live?