the negative utility due to the chance one of the AI’s descendants starts producing paperclips because “the sign bit flips spontaneously”
It’s hard to make such estimates, as they require that an AGI is unable to come up with an AGI design that’s less likely than empty space to produce paperclips. I don’t see how the impossibility of this task could be guaranteed on low level, as a “physical law”; and if you merely don’t see how to do it, an AGI might still find a way, as it’s better at designing things than you are. Empty space is only status quo, it’s not obviously optimal at not producing paperclips, and so it might be possible to find a better plan, which becomes more likely if you are very good at finding better plans.
If you mean “empty space” as in vacuum then I think it doesn’t contain any paperclips more or less by definition. If you mean “empty space” as in thermodynamic equilibrium at finite temperature then it contains some small amount of paperclips. I agree it might be possible to create a state which contains less paperclips for some limited period of time (before onset of thermodynamic equilibrium). However it’s probably much harder than the opposite (i.e. creating a state which contains much more paperclips than thermodynamic equilibrium).
It’s hard to make such estimates, as they require that an AGI is unable to come up with an AGI design that’s less likely than empty space to produce paperclips. I don’t see how the impossibility of this task could be guaranteed on low level, as a “physical law”; and if you merely don’t see how to do it, an AGI might still find a way, as it’s better at designing things than you are. Empty space is only status quo, it’s not obviously optimal at not producing paperclips, and so it might be possible to find a better plan, which becomes more likely if you are very good at finding better plans.
If you mean “empty space” as in vacuum then I think it doesn’t contain any paperclips more or less by definition. If you mean “empty space” as in thermodynamic equilibrium at finite temperature then it contains some small amount of paperclips. I agree it might be possible to create a state which contains less paperclips for some limited period of time (before onset of thermodynamic equilibrium). However it’s probably much harder than the opposite (i.e. creating a state which contains much more paperclips than thermodynamic equilibrium).
It is not clear to me that the definition of the vacuum state (http://en.wikipedia.org/wiki/Vacuum_state) precludes the momentary creation of paperclips.