@Eliezer: Can you expand on the “less ashamed of provincial values” part?
@Carl Shuman: I don’t know about him, but for myself, HELL YES I DO. Family—they’re just randomly selected by the birth lottery. Lovers—falling in love is some weird stuff that happens to you regardless of whether you want it, reaching into your brain to change your values: like, dude, ew—I want affection and tenderness and intimacy and most of the old interpersonal fun and much more new interaction, but romantic love can go right out of the window with me. Friends—I do value friendship; I’m confused; maybe I just value having friends, and it’d rock to be close friends with every existing mind; maybe I really value preferring some people to others; but I’m sure about this: I should not, and do not want to, worry more about a friend with the flu than about a stranger with cholera.
@Robin Hanson: HUH? You’d really expect natural selection to come up with minds who enjoy art, mourn dead strangers and prefer a flawed but sentient woman to a perfect catgirl on most planets?
This talk about “‘right’ means right” still makes me damn uneasy. I don’t have more to show for it than “still feels a little forced”—when I visualize a humane mind (say, a human) and a paperclipper (a sentient, moral one) looking at each other in horror and knowing there is no way they could agree about whether using atoms to feed babies or make paperclips, I feel wrong. I think about the paperclipper in exactly the same way it thinks about me! Sure, that’s also what happens when I talk to a creationist, but we’re trying to approximate external truth; and if our priors were too stupid, our genetic line would be extinct (or at least that’s what I think) - but morality doesn’t work like probability, it’s not trying to approximate anything external. So I don’t feel so happier about the moral miracle that made us than about the one that makes the paperclipper.
@Eliezer: Can you expand on the “less ashamed of provincial values” part?
@Carl Shuman: I don’t know about him, but for myself, HELL YES I DO. Family—they’re just randomly selected by the birth lottery. Lovers—falling in love is some weird stuff that happens to you regardless of whether you want it, reaching into your brain to change your values: like, dude, ew—I want affection and tenderness and intimacy and most of the old interpersonal fun and much more new interaction, but romantic love can go right out of the window with me. Friends—I do value friendship; I’m confused; maybe I just value having friends, and it’d rock to be close friends with every existing mind; maybe I really value preferring some people to others; but I’m sure about this: I should not, and do not want to, worry more about a friend with the flu than about a stranger with cholera.
@Robin Hanson: HUH? You’d really expect natural selection to come up with minds who enjoy art, mourn dead strangers and prefer a flawed but sentient woman to a perfect catgirl on most planets?
This talk about “‘right’ means right” still makes me damn uneasy. I don’t have more to show for it than “still feels a little forced”—when I visualize a humane mind (say, a human) and a paperclipper (a sentient, moral one) looking at each other in horror and knowing there is no way they could agree about whether using atoms to feed babies or make paperclips, I feel wrong. I think about the paperclipper in exactly the same way it thinks about me! Sure, that’s also what happens when I talk to a creationist, but we’re trying to approximate external truth; and if our priors were too stupid, our genetic line would be extinct (or at least that’s what I think) - but morality doesn’t work like probability, it’s not trying to approximate anything external. So I don’t feel so happier about the moral miracle that made us than about the one that makes the paperclipper.