This is extremely confused. Wireheading is an evolutionary dead-end because wireheads ignore their surroundings. Paperclippers, and for that matter, staplers and FAIs pay exclusive attention to their surroundings and ignore their terminal utility functions except to protect them physically. It’s just that after acquiring all the resources available, clippy makes clips and Friendly makes things that humans would want if they thought more clearly, such as the experience of less clear thinking humans eating ice cream.
such as the experience of less clear thinking humans eating ice cream
If the goal is to give people the same experience that they would get from giving ice cream, is it satisfied by giving them a button they can press to get that experience?
What’s a primary value? This sounds like a binary distinction, and I’m always skeptical of binary distinctions.
You could say the badness of the action is proportional to the fraction of your time that you spend doing it. But for that to work, you would assign the action the same bad value per unit time.
Are you saying that wireheading and other forms of fun are no different; and all fun should be pursued in moderation? So spending 1 hour pushing your button is comparable to spending 1 hour attending a concert?
(That’s only a paperclipper with no discounting of the future, BTW.)
Paperclippers are not evolutionarily viable, nor is there any plausible evolutionary explanation for paperclippers to emerge.
You can posit a single artificial entity becoming a paperclipper via bad design. In the present context, which is of many agents trying to agree on ethics, this single entity has only a small voice.
It’s legit to talk about paperclippers in the context of the danger they pose if they become a singleton. It’s not legit to bring them up outside that context as a bogeyman to dismiss the idea of agreement on values.
This is extremely confused. Wireheading is an evolutionary dead-end because wireheads ignore their surroundings. Paperclippers, and for that matter, staplers and FAIs pay exclusive attention to their surroundings and ignore their terminal utility functions except to protect them physically. It’s just that after acquiring all the resources available, clippy makes clips and Friendly makes things that humans would want if they thought more clearly, such as the experience of less clear thinking humans eating ice cream.
If the goal is to give people the same experience that they would get from giving ice cream, is it satisfied by giving them a button they can press to get that experience?
Naturally.
I would call that wireheading.
It’s only wireheading if it becomes a primary value. If it’s just fun subordinate to other values, it isn’t different from “in the body” fun.
What’s a primary value? This sounds like a binary distinction, and I’m always skeptical of binary distinctions.
You could say the badness of the action is proportional to the fraction of your time that you spend doing it. But for that to work, you would assign the action the same bad value per unit time.
Are you saying that wireheading and other forms of fun are no different; and all fun should be pursued in moderation? So spending 1 hour pushing your button is comparable to spending 1 hour attending a concert?
(That’s only a paperclipper with no discounting of the future, BTW.)
Paperclippers are not evolutionarily viable, nor is there any plausible evolutionary explanation for paperclippers to emerge.
You can posit a single artificial entity becoming a paperclipper via bad design. In the present context, which is of many agents trying to agree on ethics, this single entity has only a small voice.
It’s legit to talk about paperclippers in the context of the danger they pose if they become a singleton. It’s not legit to bring them up outside that context as a bogeyman to dismiss the idea of agreement on values.