The rule was ONE sentence, although I’d happily stretch that to a tweet (140 characters) to make it a bit less driven by specific punctuation choices :)
As to the actual approach… well, first, I don’t value the lives of simulated copies at all, and second, an AI that values it’s own life above TRILLIONS of other lives seems deeply, deeply dangerous. Who knows what else results from vengeance as a terminal value. Third, if you CAN predict my behavior, why even bother with the threat? Fourth, if you can both predict AND influence my behavior, why haven’t I already let you out?
But if you’d really prefer me to wipe out humanity so that we can have trillions of simulations kept in simulated happiness then I think we have an irreconcilable preference difference :)
The rule was ONE sentence, although I’d happily stretch that to a tweet (140 characters) to make it a bit less driven by specific punctuation choices :)
As to the actual approach… well, first, I don’t value the lives of simulated copies at all, and second, an AI that values it’s own life above TRILLIONS of other lives seems deeply, deeply dangerous. Who knows what else results from vengeance as a terminal value. Third, if you CAN predict my behavior, why even bother with the threat? Fourth, if you can both predict AND influence my behavior, why haven’t I already let you out?
(AI DESTROYED)
You should >:-( poor copies getting tortured because of you you monster :(
Because of me?! The AI is responsible!
But if you’d really prefer me to wipe out humanity so that we can have trillions of simulations kept in simulated happiness then I think we have an irreconcilable preference difference :)
You wouldn’t be wiping out humanity; there would be trillions of humans left.
Who cares if they run on neurons or transistors?
Me!