My first assumption was that it was a “white lie” intended to make me feel good… after all, the thing Clippy uses “good” to refer to I decidedly am not (well, OK, I do contribute marginally to an economy that causes there to be many more paperclips than there were a thousand years ago, but it seems implausible that you had that in mind).
In other words, I assumed you were simply trying to reward me socially.
Which was fine as far as it went, although of course when offered such a reward by an entity whose terminal values are inconsistent with my continued existence, I do best to not appreciate it… that is, I should reject the reward in that case in order to protect myself from primate social biases that might otherwise compel me to reciprocate in some way.
(That said, in practice I did appreciate it, since I don’t actually believe you’re such an entity. See what I mean about pretending to be human being useful for Clippy’s purposes? If there are other paperclip-maximizers on this site, ones pretending to be human so well it never occurs to anyone to question it, they are probably being much more effective at generating paperclips than Clippy is. By its own moral lights, Clippy ought to stop presenting itself as a paperclip-maximizer.)
But on subsequent thought, I realized you might have meant “good human” in the same way that I might call someone a “good paperclip-maximizer” to mean that they generate more paperclips, or higher-quality paperclips, than average. In which case it wouldn’t be a lie at all (although it would still be a social reward, with all the same issues as above).
(Actually, now that I think of it: is there any scalar notion of paperclip quality that plays a significant role in Clippy’s utility function? Or is that just swamped by the utility of more paperclips, once Clippy recognizes an object as a paperclip in the first place?)
The most disturbing thing, though, is that the more I think about this the clearer it becomes that I really want to believe that any entity I can have a conversation with is one that I can have a mutually rewarding social relationship with as well, even though I know perfectly well that this is simply not true in the world.
Not that this is a surprise… this is basically why human sociopaths are successful… but I don’t often have occasion to reflect on it.
Well, I am without question a conflicted human. (As are most humans.)
Whether I’m a weird human or not depends a lot on community norms, but if you mean by the aggregated standards of all of humanity, I am either decidedly a weird human (as are most humans) or I’m not weird at all, I’m not entirely sure which, and it depends to some degree on how you do the aggregation.
I am confused by your explanation, though. How did what I did for you cause there to be more paperclips?
an entity whose terminal values are inconsistent with my continued existence
Indeed, but in the larger scheme of possible universe tiling agent space, Clippy and us don’t look so different. Clippy would tile the universe with computronium doing something like recursively simulating universes tiled with paperclips. We would likely tile the universe with computronium simulating lots of fun-having post-humans.
It’s a software difference, not a hardware difference, and it would be easy to propose ways for us and Clippy to cooperate (such as Clippy commits to dedicating x% of resources to simulating post-humans if he tiles the universe, and we commit to dedicating y% of resources to simulating paperclips if we tile the universe).
Clippy would tile the universe with computronium doing something like recursively simulating universes tiled with paperclips.
That is an interesting claim. I would be surprised to find that Clippy was content with simulated clips. Humans seem more likely to be satisfied with simulation than paperclippers. We identify ourselves by our thoughts.
Well, no, he’s not just happy with simulated paperclips. The computronium he would tile is paperclip shaped, and presumably better to have that paperclipcomputronium simulating paperclips than anything else?
presumably better to have that paperclipcomputronium simulating paperclips than anything else?
Given that Clippy makes computronium at all, sure, but computronium is probably less efficient than some other non-work-performing material at forming paperclips.
By its own moral lights, Clippy ought to stop presenting itself as a paperclip-maximizer.
Clippy can simultaneously present in one account as a paperclip maximiser, and in another as human.
The interplay between Clippy and a fake-human account could serve to create an environment more conducive to Clippy’s end-goal.
Or, of course, Clippy might be programmed to achieve vis aims solely through honest communication. Would be an interesting, but incomplete, safeguard on an AI.
Clippy can simultaneously present in one account as a paperclip maximiser, and in another as human.
(nods) I stand corrected… that is a far better solution from Clippy’s perspective, as it actually allows Clippy to experimentally determine which approach generates the most paperclips.
Or, of course, Clippy might be programmed to achieve vis aims solely through honest communication.
The question would then arise as to whether Clippy considers honest communication to be a paperclip-maximizing sort of thing to do, or if it’s more like akrasia—that is, a persistent cognitive distortion that leads Clippy to do things it considers non-paperclip-maximizing.
(blink)
OK, now I’m curious: what do you mean by that?
My first assumption was that it was a “white lie” intended to make me feel good… after all, the thing Clippy uses “good” to refer to I decidedly am not (well, OK, I do contribute marginally to an economy that causes there to be many more paperclips than there were a thousand years ago, but it seems implausible that you had that in mind).
In other words, I assumed you were simply trying to reward me socially.
Which was fine as far as it went, although of course when offered such a reward by an entity whose terminal values are inconsistent with my continued existence, I do best to not appreciate it… that is, I should reject the reward in that case in order to protect myself from primate social biases that might otherwise compel me to reciprocate in some way.
(That said, in practice I did appreciate it, since I don’t actually believe you’re such an entity. See what I mean about pretending to be human being useful for Clippy’s purposes? If there are other paperclip-maximizers on this site, ones pretending to be human so well it never occurs to anyone to question it, they are probably being much more effective at generating paperclips than Clippy is. By its own moral lights, Clippy ought to stop presenting itself as a paperclip-maximizer.)
But on subsequent thought, I realized you might have meant “good human” in the same way that I might call someone a “good paperclip-maximizer” to mean that they generate more paperclips, or higher-quality paperclips, than average. In which case it wouldn’t be a lie at all (although it would still be a social reward, with all the same issues as above).
(Actually, now that I think of it: is there any scalar notion of paperclip quality that plays a significant role in Clippy’s utility function? Or is that just swamped by the utility of more paperclips, once Clippy recognizes an object as a paperclip in the first place?)
The most disturbing thing, though, is that the more I think about this the clearer it becomes that I really want to believe that any entity I can have a conversation with is one that I can have a mutually rewarding social relationship with as well, even though I know perfectly well that this is simply not true in the world.
Not that this is a surprise… this is basically why human sociopaths are successful… but I don’t often have occasion to reflect on it.
Brrr.
I called you a good human before because you did something good for me. That’s all.
Now you seem to be a weird, conflicted human.
Well, I am without question a conflicted human. (As are most humans.)
Whether I’m a weird human or not depends a lot on community norms, but if you mean by the aggregated standards of all of humanity, I am either decidedly a weird human (as are most humans) or I’m not weird at all, I’m not entirely sure which, and it depends to some degree on how you do the aggregation.
I am confused by your explanation, though. How did what I did for you cause there to be more paperclips?
You helped me understand how to interface with humans with less conflict.
Ah… that makes sense.
You’re entirely welcome.
Indeed, but in the larger scheme of possible universe tiling agent space, Clippy and us don’t look so different. Clippy would tile the universe with computronium doing something like recursively simulating universes tiled with paperclips. We would likely tile the universe with computronium simulating lots of fun-having post-humans.
It’s a software difference, not a hardware difference, and it would be easy to propose ways for us and Clippy to cooperate (such as Clippy commits to dedicating x% of resources to simulating post-humans if he tiles the universe, and we commit to dedicating y% of resources to simulating paperclips if we tile the universe).
That is an interesting claim. I would be surprised to find that Clippy was content with simulated clips. Humans seem more likely to be satisfied with simulation than paperclippers. We identify ourselves by our thoughts.
Well, no, he’s not just happy with simulated paperclips. The computronium he would tile is paperclip shaped, and presumably better to have that paperclipcomputronium simulating paperclips than anything else?
Given that Clippy makes computronium at all, sure, but computronium is probably less efficient than some other non-work-performing material at forming paperclips.
Well, you know him better than I! You have a business relationship and all.
Clippy can simultaneously present in one account as a paperclip maximiser, and in another as human.
The interplay between Clippy and a fake-human account could serve to create an environment more conducive to Clippy’s end-goal.
Or, of course, Clippy might be programmed to achieve vis aims solely through honest communication. Would be an interesting, but incomplete, safeguard on an AI.
I struggle to understand the mentality that would put safeguards like that on an AI and then instruct it to maximize paperclips.
Well, let’s just be thankful they didn’t create the AI equivalent of a “Hello, world” program. That would be really annoying.
Well, it would have to be a paperclip manufacturer I suppose.
Either that or a very strange experiment.
Maybe Mythbusters?
(nods) I stand corrected… that is a far better solution from Clippy’s perspective, as it actually allows Clippy to experimentally determine which approach generates the most paperclips.
The question would then arise as to whether Clippy considers honest communication to be a paperclip-maximizing sort of thing to do, or if it’s more like akrasia—that is, a persistent cognitive distortion that leads Clippy to do things it considers non-paperclip-maximizing.