Nothing which you have written appears to show that it’s impossible or even unlikely that people try to get
things they want (which sometimes include pleasure, and which sometimes include saving the world), and
that successful planning just feels good.
I’m not trying to show that. I agree that people try to get things they want, as long as with “things they want” we mean “things that they are tempted to go for because the thought of going for those things is so pleasurable”.
(something about X) --> (I like the thought of X) --> (I take action X), where (I like the thought of X) would seem
to be an unnecessary step where the same result would be obtained by eliminating it,
Why would you want to eliminate the pleasure involved in decision processes? Don’t you feel pleasure has intrinsic value? If you eliminate pleasure from decision processes, why not eliminate it altogether from life, for the same reasons that made you consider pleasure “unnecessary” in decision processes?
This, I think, is one thing that makes many people so reluctant to accept the idea of human-level and super-human AI: they notice that many advocates of the AI revolution seem to want to ignore the subjective part of being human and seem interested merely in how to give machines the objective abilities of humans (i.e. abilities to manipulate the outer environment rather than “intangibles” like love and happiness). This seems as backward as spending your whole life earning millions of dollars, having no fun doing it, and never doing anything fun or good with the money. For most people, at first at least, the purpose of earning money is to increase pleasure. So should the purpose of building human-level or super-human AI. If you start to think that step two (the pleasure) in decision processes is an unnecessary part of our decision processes and can be omitted, you are thinking like the money-hunter who has lost track of why money is important; by thinking that pleasure may as well be omitted in decision processes, you throw away the whole reason for having any decision processes at all.
It’s the second step (of your three steps above) - the step which is always “I like the though of...”, i.e. our striving to maximize pleasure—that determines our values and choices about whatever there is in the first step (“X” or “something about X”, the thing we happen to like the thought of). So, to the extent that the first step (“something about X”) is incompatible with pleasure-maximizing (the decisive second step), what happens in step two seems to be a misinterpretation of what is there in step one. It seems reasonable to get rid of any misinterpretation. For example: fast food tastes good and produces short-term pleasure, but that pleasure is a misinterpretation in that it makes our organism take fast food for something more nutritious and long-term good for us than it actually is. We should go for pleasure, but not necessarily by eating fast food. We should let ourselves be motivated by the phenomenon in “step two” (“I like the thought of...”), but we should be careful about which “step one”’s (“X” or “something about X”) we let “step two” lead us to decisions about. The pleasure derived from eating fast food is, in and of itself, intrinsically good (all other things equal), but the source of it: fast food, is not. Step two is always a good thing as long as step one is a good thing, but step one is sometimes not a good thing even when step two, in and of itself, is a good thing. Whether the goal is to get to step three or to just enjoy the happiness in step two, step one is dispensable and replaceable, whereas step two is always necessary. So, it seems reasonable to found all ethics exclusively on what happens in step two.
Or you can suggest that people are just mistaken about how pleasurable the results will be of any action >they take that doesn’t maximise pleasure.
Even if they are not too mistaken about that, they may still be shortsighted enough that, when trying to choose between decision A and decision B, they’ll prefer the brief but immediate pleasure of making decision A (regardless of its expected later consequences) to the much larger amount of pleasure that they know would eventually follow after the less immediately pleasurable decision B. Many of us are this shortsighted. Our reward mechanism needs fixing.
You’re missing the point, or perhaps I’m missing your point. A paperclip maximiser implemented by having the program experience subjective pleasure when considering an action that results in lots of paperclips, and which decides by taking the action with the highest associated subjective pleasure, is still a paperclip maximiser.
So, I think you’re confusing levels. On the decision making level, you can hypothesise that decisions are made by attaching a “pleasure” feeling to each option and taking the one with highest pleasure. Sure, fine. But this doesn’t mean it’s wrong for an option which predictably results in less physical pleasure later to feel less pleasurable during decision making. The decision system could have been implemented equally well by associating options with colors and picking the brightest or something, without meaning the agent is irrational to take an action that physically darkens the environment. This is just a way of implementing the algorithm, which is not about the brightness of the environment or the light levels observed by the agent.
This is what I mean by ”(I like the thought of X) would seem to be an unnecessary step”. The implementation is not particularly relevant to the values. Noticing that pleasure is there at a step in the decision process doesn’t tell you what should feel pleasurable and what shouldn’t, it just tells you a bit about the mechanisms.
Of course I believe that pleasure has intrinsic value. We value fun; pleasure can be fun. But I can’t believe pleasure is the only thing with intrinsic value. We don’t use nozick’s pleasure machine, we don’t choose to be turned into orgasmium, we are willing to be hurt for higher benefits. I don’t think any of those things are mistakes.
I’m not trying to show that. I agree that people try to get things they want, as long as with “things they want” we mean “things that they are tempted to go for because the thought of going for those things is so pleasurable”.
Why would you want to eliminate the pleasure involved in decision processes? Don’t you feel pleasure has intrinsic value? If you eliminate pleasure from decision processes, why not eliminate it altogether from life, for the same reasons that made you consider pleasure “unnecessary” in decision processes?
This, I think, is one thing that makes many people so reluctant to accept the idea of human-level and super-human AI: they notice that many advocates of the AI revolution seem to want to ignore the subjective part of being human and seem interested merely in how to give machines the objective abilities of humans (i.e. abilities to manipulate the outer environment rather than “intangibles” like love and happiness). This seems as backward as spending your whole life earning millions of dollars, having no fun doing it, and never doing anything fun or good with the money. For most people, at first at least, the purpose of earning money is to increase pleasure. So should the purpose of building human-level or super-human AI. If you start to think that step two (the pleasure) in decision processes is an unnecessary part of our decision processes and can be omitted, you are thinking like the money-hunter who has lost track of why money is important; by thinking that pleasure may as well be omitted in decision processes, you throw away the whole reason for having any decision processes at all.
It’s the second step (of your three steps above) - the step which is always “I like the though of...”, i.e. our striving to maximize pleasure—that determines our values and choices about whatever there is in the first step (“X” or “something about X”, the thing we happen to like the thought of). So, to the extent that the first step (“something about X”) is incompatible with pleasure-maximizing (the decisive second step), what happens in step two seems to be a misinterpretation of what is there in step one. It seems reasonable to get rid of any misinterpretation. For example: fast food tastes good and produces short-term pleasure, but that pleasure is a misinterpretation in that it makes our organism take fast food for something more nutritious and long-term good for us than it actually is. We should go for pleasure, but not necessarily by eating fast food. We should let ourselves be motivated by the phenomenon in “step two” (“I like the thought of...”), but we should be careful about which “step one”’s (“X” or “something about X”) we let “step two” lead us to decisions about. The pleasure derived from eating fast food is, in and of itself, intrinsically good (all other things equal), but the source of it: fast food, is not. Step two is always a good thing as long as step one is a good thing, but step one is sometimes not a good thing even when step two, in and of itself, is a good thing. Whether the goal is to get to step three or to just enjoy the happiness in step two, step one is dispensable and replaceable, whereas step two is always necessary. So, it seems reasonable to found all ethics exclusively on what happens in step two.
Even if they are not too mistaken about that, they may still be shortsighted enough that, when trying to choose between decision A and decision B, they’ll prefer the brief but immediate pleasure of making decision A (regardless of its expected later consequences) to the much larger amount of pleasure that they know would eventually follow after the less immediately pleasurable decision B. Many of us are this shortsighted. Our reward mechanism needs fixing.
You’re missing the point, or perhaps I’m missing your point. A paperclip maximiser implemented by having the program experience subjective pleasure when considering an action that results in lots of paperclips, and which decides by taking the action with the highest associated subjective pleasure, is still a paperclip maximiser.
So, I think you’re confusing levels. On the decision making level, you can hypothesise that decisions are made by attaching a “pleasure” feeling to each option and taking the one with highest pleasure. Sure, fine. But this doesn’t mean it’s wrong for an option which predictably results in less physical pleasure later to feel less pleasurable during decision making. The decision system could have been implemented equally well by associating options with colors and picking the brightest or something, without meaning the agent is irrational to take an action that physically darkens the environment. This is just a way of implementing the algorithm, which is not about the brightness of the environment or the light levels observed by the agent.
This is what I mean by ”
(I like the thought of X)would seem to be an unnecessary step”. The implementation is not particularly relevant to the values. Noticing that pleasure is there at a step in the decision process doesn’t tell you what should feel pleasurable and what shouldn’t, it just tells you a bit about the mechanisms.Of course I believe that pleasure has intrinsic value. We value fun; pleasure can be fun. But I can’t believe pleasure is the only thing with intrinsic value. We don’t use nozick’s pleasure machine, we don’t choose to be turned into orgasmium, we are willing to be hurt for higher benefits. I don’t think any of those things are mistakes.