I can still think the CEV machine is better than whatever the alternative is (for instance, no AI at all.) But yes, in theory, you should prefer to make AIs that have your own values and not bother with CEV.
Having a body is irrelevant. Bodies are just one way to manipulate the world to optimize your goals.
“We convert the resources of the world into the things we want.” To some extent, but not infinitely, in a fanatical way. Again, that is the whole worry about AI—that it might do that fanatically. We don’t.
What do you mean by “fanatically”? This is a pretty vague word. Humans would sure seem fanatical to other animals. We’ve cut down entire continent sized forests, drained massive lakes, and built billions of complex structures.
The only reason we haven’t “optimized” the Earth further, is because of physical and economic limits. If we could we probably would.
Whether you call that “optimization” or not, is mostly irrelevant. If superintelligent AIs acted similarly, humans would be screwed.
I’m deeply concerned that you are theoretically ok with slave owning aliens. If the slaves are ok with it, then perhaps it could be justified. But if they strongly object to it, and suffer from it, and don’t get any benefit from it, then it’s just obviously wrong.
“Having a body is irrelevant. Bodies are just one way to manipulate the world to optimize your goals.”
This is not true. Bodies are physical objects that follow the laws of physics, and the laws of physics are not “just one way to manipulate the world to optimize your goals,” because the laws have nothing to do with your goals. For example, we often don’t keep doing something because we are tired, not because we have a goal of not continuing. AIs will be quite capable of doing the same thing, as for example if thinking too hard about something begins to weaken its circuits.
What I mean by fanatically is trying to optimize for a single goal as though it were the only thing that mattered. We do not do that, nor does anything else with a body, nor is it even possible, for the above reason.
Yes you should be concerned about what I said about slaves and aliens, as it suggests that the CEV machine might result in things that you consider utterly wicked. I said that from the beginning, when you claimed that it would eliminate all negative results, obviously intending that to mean from your subjective point of view.
I can still think the CEV machine is better than whatever the alternative is (for instance, no AI at all.) But yes, in theory, you should prefer to make AIs that have your own values and not bother with CEV.
Having a body is irrelevant. Bodies are just one way to manipulate the world to optimize your goals.
What do you mean by “fanatically”? This is a pretty vague word. Humans would sure seem fanatical to other animals. We’ve cut down entire continent sized forests, drained massive lakes, and built billions of complex structures.
The only reason we haven’t “optimized” the Earth further, is because of physical and economic limits. If we could we probably would.
Whether you call that “optimization” or not, is mostly irrelevant. If superintelligent AIs acted similarly, humans would be screwed.
I’m deeply concerned that you are theoretically ok with slave owning aliens. If the slaves are ok with it, then perhaps it could be justified. But if they strongly object to it, and suffer from it, and don’t get any benefit from it, then it’s just obviously wrong.
“Having a body is irrelevant. Bodies are just one way to manipulate the world to optimize your goals.”
This is not true. Bodies are physical objects that follow the laws of physics, and the laws of physics are not “just one way to manipulate the world to optimize your goals,” because the laws have nothing to do with your goals. For example, we often don’t keep doing something because we are tired, not because we have a goal of not continuing. AIs will be quite capable of doing the same thing, as for example if thinking too hard about something begins to weaken its circuits.
What I mean by fanatically is trying to optimize for a single goal as though it were the only thing that mattered. We do not do that, nor does anything else with a body, nor is it even possible, for the above reason.
Yes you should be concerned about what I said about slaves and aliens, as it suggests that the CEV machine might result in things that you consider utterly wicked. I said that from the beginning, when you claimed that it would eliminate all negative results, obviously intending that to mean from your subjective point of view.