I personally treat lives as valuable in and of themselves.
Why? What sort of life has value? Does the life of a bacterium have inherent value? How about a chicken? Does a life have finite inherent value? How do you compare the inherent value of different lives?
I must admit I am a tad confused here, but intelligence or whatever seems a good rule of thumb.
It’s why I don’t kill sad people, I try to make them happier.
Killing people makes them have 0 happiness (in practice, it actually reduces the total happiness in the world by quite a bit because killing someone has a lot of side effects.) Making people happy gives them positive happiness. Positive happiness is better than 0 happiness.
Oh, yes. Nevertheless, even if it would increase net happiness, I don’t kill people. Not for the sake of happiness alone and all that.
Most people would argue that animals are less capable of experiencing suffering and thus the same amount of pain is worth less in an animal than a human.
I don’t care what most people think. The majority is wrong about a lot of things. I believe that non-human animals [1] experience pain in roughly the same way that humans do because that’s where the evidence seems to point. What most people think about it does not come into the equation.
The same way, sure. But introspection suggests I don’t value it as much depending on how conscious they are (probably the same as intelligence.)
Do you also support tiling the universe with orgasmium?
Probably. I’m reluctant to make a change of that magnitude without considering it really, really carefully, no matter how sure I may be right now that it’s a good thing. If I found myself with the capacity to do this, I would probably recruit an army of the world’s best thinkers to decide if it’s worth doing. But right now I’m inclined to say that it is.
I must admit I am a tad confused here, but intelligence or whatever seems a good rule of thumb.
I was asking questions to try to better understand where you’re coming from. Do you mean the questions were confusing?
Are you saying that moral worth is directly proportional to intelligence? If so, why do you think this is true?
But introspection suggests I don’t value it as much depending on how conscious they are (probably the same as intelligence.)
Why not? Do you have a good reason, or are you just going off of intuition?
Have you read “Not for the Sake of Happiness (Alone)”?
Yes, I’ve read it. I’m not entirely convinced that all values reduce to happiness, but I’ve never seen any value that can’t be reduced to happiness. That’s one of the areas in ethics where I’m the most uncertain. In practice, it doesn’t come up much because in almost every situation, happiness and preference satisfaction amount to the same thing.
I’m inclined to believe that not all preferences reduce to happiness, but all CEV preferences do reduce to happiness. As I said before, I’m fairly uncertain about this and I don’t have much evidence.
Yes, I’ve read it. I’m not entirely convinced that all values reduce to happiness, but I’ve never seen any value that can’t be reduced to happiness. That’s one of the areas in ethics where I’m the most uncertain. In practice, it doesn’t come up much because in almost every situation, happiness and preference satisfaction amount to the same thing.
You can probably think of a happiness-based justification for any value someone throws at you. But that’s probably only because you’re coming from the privileged position of being a human who already knows those values are good, and hence wants to find a reason happiness justifies them. I suspect an AI designed only to maximise happiness would probably find a different way that would produce more happiness while disregarding almost all values we think we have.
It’s difficult for me to say because this sort of introspection is difficult, but I believe that I generally reject values when I find that they don’t promote happiness.
You can probably think of a happiness-based justification for any value someone throws at you.
But some justifications are legitimate and some are rationalizations. With the examples of discovery and creativity, I think it’s obvious that they increase happiness by a lot. It’s not like I came up with some ad hoc justification for why they maybe provide a little bit of happiness. It’s like discovery is responsible for almost all of the increases in quality of life that have taken place over the past several thousand years.
I suspect an AI designed only to maximise happiness would probably find a different way that would produce more happiness while disregarding almost all values we think we have.
I think a lot of our values do a very good job of increasing happiness, and I welcome an AI that can point out which values don’t.
With the examples of discovery and creativity, I think it’s obvious that they increase happiness by a lot.
The point is that’s not sufficient. Like saying “all good is complexity, because for example a mother’s love for her child is really complex”. Yes, it’s complex compared to some boring things like carving identical chair legs out of wood over and over for eternity, but compared to, say, tiling the universe with the digits of chaitin’s omega or something, it’s nothing. And tiling the universe with chaitin’s omega would be a very boring and stupid thing to do.
You need to show that the value in question is the best way of generating happiness. Not just that it results in more than the status quo. It has to generate more happiness, than, say, putting everyone on heroine forever. Because otherwise someone who really cared about happiness would just do that.
I think a lot of our values do a very good job of increasing happiness, and I welcome an AI that can point out which values don’t.
And they other point is that values aren’t supposed to do a job. They’re meant to describe what job you would like done! If you care about something that doesn’t increase happiness, then self-modifying to lose that so as to make more happiness would be a mistake.
You need to show that the value in question is the best way of generating happiness.
You’re absolutely correct. Discovery may not always be the best way of generating happiness; and if it’s not, you should do something else.
And the other point is that values aren’t supposed to do a job.
Not all values are terminal values. Some people value coffee because it wakes them up; they don’t value coffee in itself. If they discover that coffee in fact doesn’t wake them up, they should stop valuing coffee.
With the examples of discovery and creativity, I think it’s obvious that they increase happiness by a lot.
The point is that’s not sufficient.
What is sufficient is demonstrating that if discovery does not promote happiness then it is not valuable. As I explained in my sorting sand example, discovery that does not in any way promote happiness is not worthwhile.
must admit I am a tad confused here, but intelligence or whatever seems a good rule of thumb.
I was asking questions to try to better understand where you’re coming from. Do you mean the questions were confusing?
No, I mean I am unsure as to what my CEV would answer.
Are you saying that moral worth is directly proportional to intelligence? If so, why do you think this is true?
Because I’ll kill a bug to save a chicken, a chicken to save a cat, a cat to save an ape, and an ape to save a human. The part of me responsible for morality clearly has some sort of criteria for moral worth that seems roughly equivalent to intelligence.
But introspection suggests I don’t value it as much depending on how conscious they are (probably the same as intelligence.)
Why not? Do you have a good reason, or are you just going off of intuition?
… both?
Have you read “Not for the Sake of Happiness (Alone)”?
Yes, I’ve read it. I’m not entirely convinced that all values reduce to happiness, but I’ve never seen any value that can’t be reduced to happiness. That’s one of the areas in ethics where I’m the most uncertain. In practice, it doesn’t come up much because in almost every situation, happiness and preference satisfaction amount to the same thing.
Fair enough. Unfortunately, the area of ethics where I’m the most uncertain is weighting creatures with different intelligence levels.
Thing like discovery and creativity seem like good examples of preferences that don’t reduce to happiness IIRC, although it’s been a while since I thought everything reduced to happiness so I don’t recall very well.
I’m inclined to believe that not all preferences reduce to happiness, but all CEV preferences do reduce to happiness. As I said before, I’m fairly uncertain about this and I don’t have much evidence.
Are you saying that moral worth is directly proportional to intelligence? If so, why do you think this is true?
Because I’ll kill a bug to save a chicken, a chicken to save a cat, a cat to save an ape, and an ape to save a human. The part of me responsible for morality clearly has some sort of criteria for moral worth that seems roughly equivalent to intelligence.
But why is intelligence important? I don’t see its connection to morality. I know it’s commonly believed that intelligence is morally relevant, and my best guess as to why is that it conveniently places humans at the top and thus justifies mistreating non-human animals.
If intelligence is morally significant, then it’s not really that bad to torture a mentally handicapped person.
I believe this is false: a mentally handicapped person suffers physical pain to the same extent that I do, so his suffering is just as morally significant. The same reasoning applies to many species of non-human animal. What matters is not intelligence but the capacity to experience happiness and suffering.
… both?
So then what is your good reason that’s not directly based on intuition?
Thing like discovery and creativity seem like good examples of preferences that don’t reduce to happiness IIRC, although it’s been a while since I thought everything reduced to happiness so I don’t recall very well.
Discovery leads to the invention of new things. In general, new things lead to increased happiness. It also leads to a better understanding of the universe, which allows us to better increase happiness. If the process of discovery brought no pleasure in itself and also didn’t make it easier for us to increase happiness, I think it would be useless. The same reasoning applies to creativity.
Not sure what this means.
You mentioned CEV in your previous comment, so I assume you’re familiar with it. I mean that I think if you took people’s coherent extrapolated volitions, they would exclusively value happiness.
I’ll kill a bug to save a chicken, a chicken to save a cat, a cat to save an ape, and an ape to save a human. The part of me responsible for morality clearly has some sort of criteria for moral worth that seems roughly equivalent to intelligence.
But why is intelligence important? I don’t see its connection to morality. I know it’s commonly believed that intelligence is morally relevant, and my best guess as to why is that it conveniently places humans at the top and thus justifies mistreating non-human animals.
Well, why is pain important? I suspect empathy is mixed up here somewhere, but honestly, it doesn’t feel like it reduces—bugs just are worth less. Besides, where do you draw the line if you lack a sliding scale—I assume you don’t care about rocks, or sponges, or germs.
If intelligence is morally significant, then it’s not really that bad to torture a mentally handicapped person.
Well … not as bad as torturing, say, Bob, the Entirely Average Person, no. But it’s risky to distinguish between humans like this because it lets in all sorts of nasty biases, so I try not to except in exceptional cases.
I believe this is false: a mentally handicapped person suffers physical pain to the same extent that I do, so his suffering is just as morally significant. The same reasoning applies to many species of non-human animal. What matters is not intelligence but the capacity to experience happiness and suffering.
I know you do. Of course, unless they’re really handicapped, most animals are still much lower; and, of course there’s the worry that the intelligence is ther and they just can’t express it in everyday life (idiot savants and so on.)
So then what is your good reason that’s not directly based on intuition?
Well, it’s morality, it does ultimately come down to intuition no matter what. I can come up with all sorts of reasons, but remember that they aren’t my true rejection—my true rejection is the mental image of killing a man to save some cockroaches.
Discovery leads to the invention of new things. In general, new things lead to increased happiness. It also leads to a better understanding of the universe, which allows us to better increase happiness. If the process of discovery brought no pleasure in itself and also didn’t make it easier for us to increase happiness, I think it would be useless. The same reasoning applies to creativity.
And yet, a world without them sounds bleak and lacking in utility.
You mentioned CEV in your previous comment, so I assume you’re familiar with it. I mean that I think if you took people’s coherent extrapolated volitions, they would exclusively value happiness
Oh, right.
Ah … not sure what I can say to convince you if NFTSOH(A) didn’t.
It’s really abstract and difficult to explain, so I probably won’t do a very good job. Peter Singer explains it pretty well in “All Animals Are Equal.” Basically, we should give equal consideration to the interests of all beings. Any being capable of suffering has an interest in avoiding suffering. A more intelligent being does not have a greater interest in avoiding suffering [1]; hence, intelligence is not morally relevant.
Besides, where do you draw the line if you lack a sliding scale—I assume you don’t care about rocks, or sponges, or germs.
There is a sliding scale. More capacity to feel happiness and suffering = more moral worth. Rocks, sponges, and germs have no capacity to feel happiness and suffering.
And yet, a world without [discovery] sounds bleak and lacking in utility.
Well yeah. That’s because discovery tends to increase happiness. But if it didn’t, it would be pointless. For example, suppose you are tasked with sifting through a pile of sand to find which one is the whitest. When you finish, you will have discovered something new. But the process is really boring and it doesn’t benefit anyone, so what’s the point? Discovery is only worthwhile if it increases happiness in some way.
I’m not saying that it’s impossible to come up with an example of something that’s not reducible to happiness, but I don’t think discovery is such a thing.
[1] Unless it is capable of greater suffering, but that’s not a trait inherent to intelligence. I think it may be true in some respects that more intelligent beings are capable of greater suffering; but what matters is the capacity to suffer, not the intelligence itself.
There is a sliding scale. More capacity to feel happiness and suffering = more moral worth. Rocks, sponges, and germs have no capacity to feel happiness and suffering.
This sounds like a bad rule and could potentially create a sensitivity arms race. Assuming that people that practice Stoic or Buddhist techniques are successful in diminishing their capacity to suffer, does that mean they are worth less morally than before they started? This would be counter-intuitive, to say the least.
Assuming that people that practice Stoic or Buddhist techniques are successful in diminishing their capacity to suffer, does that mean they are worth less morally than before they started?
It means that inducing some typically-harmful action on a Stoic is less harmful than inducing it on a normal person. For example, suppose you have a Stoic who no longer feels negative reactions to insults. If you insult her, she doesn’t mind at all. It would be morally better to insult this person than to insult a typical person.
Let me put it this way: all suffering of equal degree is equally important, and the importance of suffering is proportional to its degree.
A lot of conclusions follow from this principle, including:
animal suffering is important;
if you have to do something to one of two beings and it will cause greater suffering to being A, then, all else being equal, you should do it to being B.
It’s really abstract and difficult to explain, so I probably won’t do a very good job. Peter Singer explains it pretty well in “All Animals Are Equal.” Basically, we should give equal consideration to the interests of all beings. Any being capable of suffering has an interest in avoiding suffering. A more intelligent being does not have a greater interest in avoiding suffering [1]; hence, intelligence is not morally relevant.
No, my point was that your valuing pain is itself a moral intuition. Picture a pebblesorter explaining that this pile is correct, while your pile is, obviously, incorrect.
There is a sliding scale. More capacity to feel happiness and suffering = more moral worth. Rocks, sponges, and germs have no capacity to feel happiness and suffering.
So, say, an emotionless AI? A human with damaged pain receptors? An alien with entirely different neurochemistry analogs?
Well yeah. That’s because discovery tends to increase happiness. But if it didn’t, it would be pointless. For example, suppose you are tasked with sifting through a pile of sand to find which one is the whitest. When you finish, you will have discovered something new. But the process is really boring and it doesn’t benefit anyone, so what’s the point? Discovery is only worthwhile if it increases happiness in some way.
No. I’m saying that I value exporation/discovery/whatever even when it serves no purpose, ultimately. Joe may be exploring a randomly-generated landscape, but it’s better than sitting in a whitewashed room wireheading nonetheless.
[1] Unless it is capable of greater suffering, but that’s not a trait inherent to intelligence. I think it may be true in some respects that more intelligent beings are capable of greater suffering; but what matters is the capacity to suffer, not the intelligence itself.
I’ve avoided using the word “suffering” or its synonyms in this comment, except in one instance where I believe it is appropriate.
No, my point was that your valuing pain is itself a moral intuition.
Yes, it’s an intuition. I can’t prove that suffering is important.
So, say, an emotionless AI?
If the AI does not consciously prefer any state to any other state, then it has no moral worth.
A human with damaged pain receptors?
Such a human could still experience emotions, so ey would still have moral worth.
An alien with entirely different neurochemistry analogs?
Difficult to say. If it can experience states about which it has an interest in promoting or avoiding, then it has moral worth.
No. I’m saying that I value exporation/discovery/whatever even when it serves no purpose, ultimately. Joe may be exploring a randomly-generated landscape, but it’s better than sitting in a whitewashed room wireheading nonetheless.
Okay. I don’t really get why, but I can’t dispute that you hold that value. This is why preference utilitarianism can be nice.
You were defining pain/suffering/whatever as generic disutility? That’s much more reasonable.
… so, is a hive of bees one mind of many or sort of both at once? Does evolution get a vote, here? If you aren’t discounting optimizers that lack consciousness you’re gonna get some damn strange results with this.
Deep Blue cannot experience disutility (i.e. negative states). Deep Blue can have a utility function to determine the state of the chess board, but that’s not the same as consciously experiencing positive or negative utility.
Okay, I see what you mean by “experience”… but that makes “A non-conscious being cannot experience disutility” a tautology, so following it with “therefore” and a non-tautological claim raises all kind of warning lights in my brain.
Unless you can taboo “conscious” in such a way that that made sense, I’m gonna substitute “intelligent” for “conscious” there (which is clearly what I meant, in context.)
The point with bees is that, as a “hive mind”, they act as an optimizer without any individual intention.
I’m gonna substitute “intelligent” for “conscious” there
I don’t see that you can substitute “intelligent” for “conscious”. Perhaps they are correlated, but they’re certainly not the same. I’m definitely more intelligent than my dog, but am I more conscious? Probably not. My dog seems to experience the world just as vividly as I do. (Knowing this for certain requires solving the hard problem of consciousness, but that’s where the evidence seems to point.)
(which is clearly what I meant, in context.)
It’s clear to you because you wrote it, but it wasn’t clear to me.
Well yes, that’s the illusion of transparency for you. I assure you, I was using conscious as a synonym for intelligent. Were you interpreting it as “able to experience qualia”? Because that is both a tad tautological and noticeably different from the argument I’ve been making here.
Whatever. We’re getting offtopic.
If you value optimizer’s goals regardless of intelligence—whether valuing a bugs desires as much as a human’s, a hivemind’s goals less than it’s individual members or an evolution’s goals anywhere—you get results that do not appear to correlate with anything you could call human morality. If I have misinterpreted your beliefs, I would like to know how. If I have interpreted them correctly, I would like to see how you reconcile this with saving orphans by tipping over the ant farm.
If ants experience qualia at all, which is highly uncertain, they probably don’t experience them to the same extent that humans do. Therefore, their desires are not as important. On the issue of the moral relevance of insects, the general consensus among utilitarians seems to be that we have no idea how vividly insects can experience the world, if at all, so we are in no position to rate their moral worth; and we should invest more into research on insect qualia.
I think it’s pretty obvious that (e.g.) dogs experience the world about as vividly as humans do, so all else being equal, kicking a dog is about as bad as kicking a human. (I won’t get into the question of killing because it’s massively more complicated.)
I would like to seehow you reconcile this with saving orphans by tipping over the ant farm.
I cannot say whether this is right or wrong because we don’t know enough about ant qualia, but I would guess that a single human’s experience is worth the experience of at least hundreds of ants, possibly a lot more.
you get results that do not appear to correlate with anything you could call human morality.
Like what, besides the orphans-ants thing? I don’t know if you’ve misinterpreted my beliefs unless I have a better idea of what you think I believe. That said, I do believe that a lot of “human morality” is horrendously incorrect.
I think it’s pretty obvious that (e.g.) dogs experience the world about as vividly as humans do, so all else being equal, kicking a dog is about as bad as kicking a human.
This isn’t obvious to me. And it is especially not obvious given that dogs are a species where one of the primary selection effects has been human sympathy.
You make a good point about human sympathy. Still, if you look at biological and neurological evidence, it appears that dogs are built in pretty much the same ways we are. They have the same senses—in fact, their senses are stronger in some cases. They have the same evolutionary reasons to react to pain. The parts of their brains responsible for pain look the same as ours. The biggest difference is probably that we have cerebral cortexes and they don’t, but that part of the brain isn’t especially important in responding to physical pain. Other forms of pain, yes; and I would agree that humans can feel some negative states more strongly than dogs can. But it doesn’t look like physical pain is one of those states.
If ants experience qualia at all, which is highly uncertain, they probably don’t experience them to the same extent that humans do. Therefore, their desires are not as important.
GOSH REALLY.
I think it’s pretty obvious that (e.g.) dogs experience the world about as vividly as humans do, so all else being equal, kicking a dog is about as bad as kicking a human. (I won’t get into the question of killing because it’s massively more complicated.)
Once again, you fail to provide the slightest justification for valuing dogs as much as humans; if this was “obvious” we wouldn’t be arguing, would we? Dogs are intelligent enough to be worth a non-negligable amount, but if we value all pain equally you should feel the same way about, say, mice, or … ants.
I would like to see how you reconcile this with saving orphans by tipping over the ant farm.
I cannot say whether this is right or wrong because we don’t know enough about ant qualia, but I would guess that a single human’s experience is worth the experience of at least hundreds of ants, possibly a lot more.
Like what, besides the orphans-ants thing? I don’t know if you’ve misinterpreted my beliefs unless I have a better idea of what you think I believe. That said, I do believe that a lot of “human morality” is horrendously incorrect.
How, exactly, can human morality be “incorrect”? What are you comparing it to?
if we value all pain equally you should feel the same way about, say, mice, or … ants.
Not if mice or ants don’t feel as much pain as humans do. Equal pain is equally valuable, no matter the species. But unequal pain is not equally valuable.
Huh? You value individualbees, yet not ants?
I worded my comment poorly. I didn’t mean to imply that bees are necessarily conscious. I’ve edited my comment to reflect this.
How, exactly, can human morality be “incorrect”? What are you comparing it to?
Well I’d have to get into metaethics to answer this, which I’m not very good at. I don’t think such a conversation would be fruitful.
GOSH REALLY.
Yes, really. You seemed to think that I believe ants were worth as much as humans, so I explained why I don’t believe that.
Firstly, I thought you said we were discussing disutility, not pain?
Secondly, could we taboo consciousness? It seems to mean all things to all people in discussions like this.
Thirdly, you claimed human morality was incorrect; I was under the impression that we were analyzing human morality. If you are working to a different standard to humanity (which I doubt) then perhaps a change in terminology is in order? If you are, in fact, a human, and as such the “morality” under discussion here is that of humans, then your statement makes no sense.
Assuming the second possibility, you’re right; there is no need to get into metaethics as long as we focus on actual (human) ethics.
Not if mice or ants don’t feel as much pain as humans do. Equal pain is equally valuable, no matter the species. But unequal pain is not equally valuable.
What conceivable test would verify if one organism feels more pain than another organism?
Good question. I don’t know of any such test, although I’m reluctant to say that it doesn’t exist. That’s why it’s important to do research in this area.
Some kind of brain scans? Probably not very useful on insects, etc, but would probably work for, say, chickens vs. chimpanzees.
Okay, say you had some kind of nociceptor analysis machine (or, for that matter, whatever you think “pain” will eventually reduce to). Would it count the number of discrete cociceptors or would it measure cociceptor mass? What if we encountered extra-terrestrial life that didn’t have any (of whatever it is that we have reduced “pain” to)? Would they then count for nothing in your moral calculus?
To me, this whole things feels like we are trying to multiply apples by oranges and divide by zebras. Also, it seems problematic from an institutional design perspective, due to poor incentive structure. It would reward those persons that self-modify towards being more utility-monster-like on the margin.
Well, there’s neurologically sophisticated Earthly life with neural organization very different from mammals’, come to that.
I’m not neurologist enough to give an informed account of how an octopus’s brain differs from a rhesus monkey’s, but I’m almost sure its version of nociception would look quite different. Though they’ve got an opioid receptor system, so maybe this is more basal than I thought.
I remember reading that crustaceans don’t have the part of the brain that processes pain. I don’t feel bad about throwing live crabs into boiling water.
If true, that is interesting. On the other hand, whether or not something feels pain seems like a much easier problem to solve than how much pain something feels relative to something else.
I must admit I am a tad confused here, but intelligence or whatever seems a good rule of thumb.
Oh, yes. Nevertheless, even if it would increase net happiness, I don’t kill people. Not for the sake of happiness alone and all that.
The same way, sure. But introspection suggests I don’t value it as much depending on how conscious they are (probably the same as intelligence.)
Have you read “Not for the Sake of Happiness (Alone)”? Human values are complicated.
I was asking questions to try to better understand where you’re coming from. Do you mean the questions were confusing?
Are you saying that moral worth is directly proportional to intelligence? If so, why do you think this is true?
Why not? Do you have a good reason, or are you just going off of intuition?
Yes, I’ve read it. I’m not entirely convinced that all values reduce to happiness, but I’ve never seen any value that can’t be reduced to happiness. That’s one of the areas in ethics where I’m the most uncertain. In practice, it doesn’t come up much because in almost every situation, happiness and preference satisfaction amount to the same thing.
I’m inclined to believe that not all preferences reduce to happiness, but all CEV preferences do reduce to happiness. As I said before, I’m fairly uncertain about this and I don’t have much evidence.
You can probably think of a happiness-based justification for any value someone throws at you. But that’s probably only because you’re coming from the privileged position of being a human who already knows those values are good, and hence wants to find a reason happiness justifies them. I suspect an AI designed only to maximise happiness would probably find a different way that would produce more happiness while disregarding almost all values we think we have.
It’s difficult for me to say because this sort of introspection is difficult, but I believe that I generally reject values when I find that they don’t promote happiness.
But some justifications are legitimate and some are rationalizations. With the examples of discovery and creativity, I think it’s obvious that they increase happiness by a lot. It’s not like I came up with some ad hoc justification for why they maybe provide a little bit of happiness. It’s like discovery is responsible for almost all of the increases in quality of life that have taken place over the past several thousand years.
I think a lot of our values do a very good job of increasing happiness, and I welcome an AI that can point out which values don’t.
The point is that’s not sufficient. Like saying “all good is complexity, because for example a mother’s love for her child is really complex”. Yes, it’s complex compared to some boring things like carving identical chair legs out of wood over and over for eternity, but compared to, say, tiling the universe with the digits of chaitin’s omega or something, it’s nothing. And tiling the universe with chaitin’s omega would be a very boring and stupid thing to do.
You need to show that the value in question is the best way of generating happiness. Not just that it results in more than the status quo. It has to generate more happiness, than, say, putting everyone on heroine forever. Because otherwise someone who really cared about happiness would just do that.
And they other point is that values aren’t supposed to do a job. They’re meant to describe what job you would like done! If you care about something that doesn’t increase happiness, then self-modifying to lose that so as to make more happiness would be a mistake.
You’re absolutely correct. Discovery may not always be the best way of generating happiness; and if it’s not, you should do something else.
Not all values are terminal values. Some people value coffee because it wakes them up; they don’t value coffee in itself. If they discover that coffee in fact doesn’t wake them up, they should stop valuing coffee.
What is sufficient is demonstrating that if discovery does not promote happiness then it is not valuable. As I explained in my sorting sand example, discovery that does not in any way promote happiness is not worthwhile.
Well, orgasmium, for a start.
No, I mean I am unsure as to what my CEV would answer.
Because I’ll kill a bug to save a chicken, a chicken to save a cat, a cat to save an ape, and an ape to save a human. The part of me responsible for morality clearly has some sort of criteria for moral worth that seems roughly equivalent to intelligence.
… both?
Fair enough. Unfortunately, the area of ethics where I’m the most uncertain is weighting creatures with different intelligence levels.
Thing like discovery and creativity seem like good examples of preferences that don’t reduce to happiness IIRC, although it’s been a while since I thought everything reduced to happiness so I don’t recall very well.
Not sure what this means.
But why is intelligence important? I don’t see its connection to morality. I know it’s commonly believed that intelligence is morally relevant, and my best guess as to why is that it conveniently places humans at the top and thus justifies mistreating non-human animals.
If intelligence is morally significant, then it’s not really that bad to torture a mentally handicapped person.
I believe this is false: a mentally handicapped person suffers physical pain to the same extent that I do, so his suffering is just as morally significant. The same reasoning applies to many species of non-human animal. What matters is not intelligence but the capacity to experience happiness and suffering.
So then what is your good reason that’s not directly based on intuition?
Discovery leads to the invention of new things. In general, new things lead to increased happiness. It also leads to a better understanding of the universe, which allows us to better increase happiness. If the process of discovery brought no pleasure in itself and also didn’t make it easier for us to increase happiness, I think it would be useless. The same reasoning applies to creativity.
You mentioned CEV in your previous comment, so I assume you’re familiar with it. I mean that I think if you took people’s coherent extrapolated volitions, they would exclusively value happiness.
Well, why is pain important? I suspect empathy is mixed up here somewhere, but honestly, it doesn’t feel like it reduces—bugs just are worth less. Besides, where do you draw the line if you lack a sliding scale—I assume you don’t care about rocks, or sponges, or germs.
Well … not as bad as torturing, say, Bob, the Entirely Average Person, no. But it’s risky to distinguish between humans like this because it lets in all sorts of nasty biases, so I try not to except in exceptional cases.
I know you do. Of course, unless they’re really handicapped, most animals are still much lower; and, of course there’s the worry that the intelligence is ther and they just can’t express it in everyday life (idiot savants and so on.)
Well, it’s morality, it does ultimately come down to intuition no matter what. I can come up with all sorts of reasons, but remember that they aren’t my true rejection—my true rejection is the mental image of killing a man to save some cockroaches.
And yet, a world without them sounds bleak and lacking in utility.
Oh, right.
Ah … not sure what I can say to convince you if NFTSOH(A) didn’t.
It’s really abstract and difficult to explain, so I probably won’t do a very good job. Peter Singer explains it pretty well in “All Animals Are Equal.” Basically, we should give equal consideration to the interests of all beings. Any being capable of suffering has an interest in avoiding suffering. A more intelligent being does not have a greater interest in avoiding suffering [1]; hence, intelligence is not morally relevant.
There is a sliding scale. More capacity to feel happiness and suffering = more moral worth. Rocks, sponges, and germs have no capacity to feel happiness and suffering.
Well yeah. That’s because discovery tends to increase happiness. But if it didn’t, it would be pointless. For example, suppose you are tasked with sifting through a pile of sand to find which one is the whitest. When you finish, you will have discovered something new. But the process is really boring and it doesn’t benefit anyone, so what’s the point? Discovery is only worthwhile if it increases happiness in some way.
I’m not saying that it’s impossible to come up with an example of something that’s not reducible to happiness, but I don’t think discovery is such a thing.
[1] Unless it is capable of greater suffering, but that’s not a trait inherent to intelligence. I think it may be true in some respects that more intelligent beings are capable of greater suffering; but what matters is the capacity to suffer, not the intelligence itself.
This sounds like a bad rule and could potentially create a sensitivity arms race. Assuming that people that practice Stoic or Buddhist techniques are successful in diminishing their capacity to suffer, does that mean they are worth less morally than before they started? This would be counter-intuitive, to say the least.
It means that inducing some typically-harmful action on a Stoic is less harmful than inducing it on a normal person. For example, suppose you have a Stoic who no longer feels negative reactions to insults. If you insult her, she doesn’t mind at all. It would be morally better to insult this person than to insult a typical person.
Let me put it this way: all suffering of equal degree is equally important, and the importance of suffering is proportional to its degree.
A lot of conclusions follow from this principle, including:
animal suffering is important;
if you have to do something to one of two beings and it will cause greater suffering to being A, then, all else being equal, you should do it to being B.
No, my point was that your valuing pain is itself a moral intuition. Picture a pebblesorter explaining that this pile is correct, while your pile is, obviously, incorrect.
So, say, an emotionless AI? A human with damaged pain receptors? An alien with entirely different neurochemistry analogs?
No. I’m saying that I value exporation/discovery/whatever even when it serves no purpose, ultimately. Joe may be exploring a randomly-generated landscape, but it’s better than sitting in a whitewashed room wireheading nonetheless.
Can you taboo “suffering” for me?
I’ve avoided using the word “suffering” or its synonyms in this comment, except in one instance where I believe it is appropriate.
Yes, it’s an intuition. I can’t prove that suffering is important.
If the AI does not consciously prefer any state to any other state, then it has no moral worth.
Such a human could still experience emotions, so ey would still have moral worth.
Difficult to say. If it can experience states about which it has an interest in promoting or avoiding, then it has moral worth.
Okay. I don’t really get why, but I can’t dispute that you hold that value. This is why preference utilitarianism can be nice.
… oh.
You were defining pain/suffering/whatever as generic disutility? That’s much more reasonable.
… so, is a hive of bees one mind of many or sort of both at once? Does evolution get a vote, here? If you aren’t discounting optimizers that lack consciousness you’re gonna get some damn strange results with this.
Many. The unit of moral significance is the conscious mind. A group of bees is not conscious; individual bees are conscious.
(Edit: It’s possible that bees are not conscious. What I meant was that if bees are conscious then they are conscious as individuals, not as a group.)
A non-conscious being cannot experience disutility, therefore it has no moral relevance.
Er… Deep Blue?
Deep Blue cannot experience disutility (i.e. negative states). Deep Blue can have a utility function to determine the state of the chess board, but that’s not the same as consciously experiencing positive or negative utility.
Okay, I see what you mean by “experience”… but that makes “A non-conscious being cannot experience disutility” a tautology, so following it with “therefore” and a non-tautological claim raises all kind of warning lights in my brain.
Unless you can taboo “conscious” in such a way that that made sense, I’m gonna substitute “intelligent” for “conscious” there (which is clearly what I meant, in context.)
The point with bees is that, as a “hive mind”, they act as an optimizer without any individual intention.
I don’t see that you can substitute “intelligent” for “conscious”. Perhaps they are correlated, but they’re certainly not the same. I’m definitely more intelligent than my dog, but am I more conscious? Probably not. My dog seems to experience the world just as vividly as I do. (Knowing this for certain requires solving the hard problem of consciousness, but that’s where the evidence seems to point.)
It’s clear to you because you wrote it, but it wasn’t clear to me.
Well yes, that’s the illusion of transparency for you. I assure you, I was using conscious as a synonym for intelligent. Were you interpreting it as “able to experience qualia”? Because that is both a tad tautological and noticeably different from the argument I’ve been making here.
Whatever. We’re getting offtopic.
If you value optimizer’s goals regardless of intelligence—whether valuing a bugs desires as much as a human’s, a hivemind’s goals less than it’s individual members or an evolution’s goals anywhere—you get results that do not appear to correlate with anything you could call human morality. If I have misinterpreted your beliefs, I would like to know how. If I have interpreted them correctly, I would like to see how you reconcile this with saving orphans by tipping over the ant farm.
If ants experience qualia at all, which is highly uncertain, they probably don’t experience them to the same extent that humans do. Therefore, their desires are not as important. On the issue of the moral relevance of insects, the general consensus among utilitarians seems to be that we have no idea how vividly insects can experience the world, if at all, so we are in no position to rate their moral worth; and we should invest more into research on insect qualia.
I think it’s pretty obvious that (e.g.) dogs experience the world about as vividly as humans do, so all else being equal, kicking a dog is about as bad as kicking a human. (I won’t get into the question of killing because it’s massively more complicated.)
I cannot say whether this is right or wrong because we don’t know enough about ant qualia, but I would guess that a single human’s experience is worth the experience of at least hundreds of ants, possibly a lot more.
Like what, besides the orphans-ants thing? I don’t know if you’ve misinterpreted my beliefs unless I have a better idea of what you think I believe. That said, I do believe that a lot of “human morality” is horrendously incorrect.
This isn’t obvious to me. And it is especially not obvious given that dogs are a species where one of the primary selection effects has been human sympathy.
You make a good point about human sympathy. Still, if you look at biological and neurological evidence, it appears that dogs are built in pretty much the same ways we are. They have the same senses—in fact, their senses are stronger in some cases. They have the same evolutionary reasons to react to pain. The parts of their brains responsible for pain look the same as ours. The biggest difference is probably that we have cerebral cortexes and they don’t, but that part of the brain isn’t especially important in responding to physical pain. Other forms of pain, yes; and I would agree that humans can feel some negative states more strongly than dogs can. But it doesn’t look like physical pain is one of those states.
GOSH REALLY.
Once again, you fail to provide the slightest justification for valuing dogs as much as humans; if this was “obvious” we wouldn’t be arguing, would we? Dogs are intelligent enough to be worth a non-negligable amount, but if we value all pain equally you should feel the same way about, say, mice, or … ants.
Huh? You value individual bees, yet not ants?
How, exactly, can human morality be “incorrect”? What are you comparing it to?
See my reply here.
Not if mice or ants don’t feel as much pain as humans do. Equal pain is equally valuable, no matter the species. But unequal pain is not equally valuable.
I worded my comment poorly. I didn’t mean to imply that bees are necessarily conscious. I’ve edited my comment to reflect this.
Well I’d have to get into metaethics to answer this, which I’m not very good at. I don’t think such a conversation would be fruitful.
Yes, really. You seemed to think that I believe ants were worth as much as humans, so I explained why I don’t believe that.
Firstly, I thought you said we were discussing disutility, not pain?
Secondly, could we taboo consciousness? It seems to mean all things to all people in discussions like this.
Thirdly, you claimed human morality was incorrect; I was under the impression that we were analyzing human morality. If you are working to a different standard to humanity (which I doubt) then perhaps a change in terminology is in order? If you are, in fact, a human, and as such the “morality” under discussion here is that of humans, then your statement makes no sense.
Assuming the second possibility, you’re right; there is no need to get into metaethics as long as we focus on actual (human) ethics.
What conceivable test would verify if one organism feels more pain than another organism?
Good question. I don’t know of any such test, although I’m reluctant to say that it doesn’t exist. That’s why it’s important to do research in this area.
Some kind of brain scans? Probably not very useful on insects, etc, but would probably work for, say, chickens vs. chimpanzees.
Okay, say you had some kind of nociceptor analysis machine (or, for that matter, whatever you think “pain” will eventually reduce to). Would it count the number of discrete cociceptors or would it measure cociceptor mass? What if we encountered extra-terrestrial life that didn’t have any (of whatever it is that we have reduced “pain” to)? Would they then count for nothing in your moral calculus?
To me, this whole things feels like we are trying to multiply apples by oranges and divide by zebras. Also, it seems problematic from an institutional design perspective, due to poor incentive structure. It would reward those persons that self-modify towards being more utility-monster-like on the margin.
Well, there’s neurologically sophisticated Earthly life with neural organization very different from mammals’, come to that.
I’m not neurologist enough to give an informed account of how an octopus’s brain differs from a rhesus monkey’s, but I’m almost sure its version of nociception would look quite different. Though they’ve got an opioid receptor system, so maybe this is more basal than I thought.
I remember reading that crustaceans don’t have the part of the brain that processes pain. I don’t feel bad about throwing live crabs into boiling water.
Really? I remember reading the opposite. Many times. If you’re regularly boiling them alive, have you considered researching this?
I’m not regularly boiling them alive, but I researched it a little anyway. Here’s a study often used to show that crustaceans DO feel pain: http://forms.mbl.edu/research/services/iacuc/pdf/pain_hermit_crabs.pdf
If true, that is interesting. On the other hand, whether or not something feels pain seems like a much easier problem to solve than how much pain something feels relative to something else.