If we want hamburgers & the AI can make them much more efficiently than we can, why wouldn’t we just willingly give our resources to the AI so that it can make hamburgers?
If your preferences are maximised by you being turned into a hamburger then by all means do so.
I think we need to clarify how God-like this hypothetical AI is. If the AI is not very God-like, then trying to turn humans into hamburgers could be very costly for it. If we made the AI, maybe we could make a competing AI to resist it or use some backdoor built into the AI’s programming to pull the plug. At the very least, we could launch missiles at it.
If the AI is very God-like, then there are more resource rich sources than human beings it could easily obtain. It’d be sort of like humans gathering up all of the horses for transportation when we already have cars & planes.
If your preferences are maximised by you being turned into a hamburger then by all means do so.
I think we need to clarify how God-like this hypothetical AI is. If the AI is not very God-like, then trying to turn humans into hamburgers could be very costly for it. If we made the AI, maybe we could make a competing AI to resist it or use some backdoor built into the AI’s programming to pull the plug. At the very least, we could launch missiles at it.
If the AI is very God-like, then there are more resource rich sources than human beings it could easily obtain. It’d be sort of like humans gathering up all of the horses for transportation when we already have cars & planes.