After considering this problem, what I found was surprisingly fast, the specifics of the boxes physical abilities and implementation becomes relevant. I mean, let’s say Clippy is given this box, and has already decided to wait a mere 1 year from day 1, which is 365.25 days of doubling, and 1 paperclip is 1 utilon. At some point, during this time, before the end of it, There are more paperclips then there used to be every atom in the visible universe. Since he’s predicted to gain 2^365.25 paperclips, (which is apparently close to 8.9*10^109) and the observable universe is only estimated to contain 10^80 atoms. So to make up for that, let’s say the box converts every visible subatomic particle into paperclips instead.
That’s just 1 year, and the box has already announced it will convert approximately every visible subatomic particle into pure paperclip bliss!
And then another single doubling… (1 year and 1 day) Does what? Even if Clippy has his utility function unbounded, it should presumably still link back to some kind of physical state, and at this point the box starts having to implement increasingly physically impossible ideas to have to double paperclip utility, like:
Breaking the speed of light.
Expanding the paperclip conversion into the past.
Expanding the paperclip conversion into additional branches of many worlds.
Magically protecting the paperclips from the ravages of time, physics, or condensing into blackholes, despite the fact it is supposed to lose all power after being opened.
And that’s just 1 year! We aren’t even close to a timeless eternity of waiting yet, and the box already has to smash the currently known laws of physics (more so than it did by converting every visible subatomic particle into paperclips) to do more doublings, and will then lose power afterwards.
Do the laws of physics resume being normal after the box loses power? If so, massive chunks of utility will fade away almost instantly (which would seem to indicate the Box was not very effective), but if not I’m not sure how the loop below would get resolved:
The Box is essentially going to rewrite the rules of the universe permanently,
Which would affect your utility calculations, which are based on physics,
Which would affect how the Box rewrote the rules of the universe,
Which would affect your utility calculations, which are based on physics,
Except instead of stopping and letting you and the box resolve this loop, it must keeps doubling, so it keeps changing physics more.
By year 2, it seems like you might be left with either:
A solution, in which case whatever the box will rewrite the laws of physics to, you understand and agree with it and can work on the problem based on whatever that solution is. (But I have no idea how you could figure out what this solution would be in advance, since it depends on the specific box?)
Or, an incredibly intractable metaphysics problem which is growing more complicated faster than you can ever calculate, in which case you don’t even understand what the box is doing anymore.
The reason I said that this was incredibly fast is that my original guess was that it would take at least 100 years of daily doubling for the proposed world to become that complicated, but when I tried doing a bit of math it didn’t take anywhere near that long.
This is a thought experiment which is not meant to be possible in our world. But such thought experiments are a way of testing the generality of your decision procedures—do they work in all possible worlds? If you must imagine a physics that makes the eternal doubling possible, try picturing a network of replicating baby universes linked by wormholes.
But such thought experiments are a way of testing the generality of your decision procedures—do they work in all possible worlds?
As in the old saw, part of your strength as a real decision-maker is that your decision procedures choose less well in impossible worlds than in possible worlds.
It doesn’t have to be true. It’s desirable because decision procedures that rely on other knowledge about reality are faster/better/cheaper than ones that don’t import knowledge about reality. Specialization for the situation you find yourself in is often useful, though it does limit flexibility.
Utility doesn’t have to be proportional to the amount of some particular kind of physical stuff in the universe. If the universe contained 1 paperclip, that could be worth 2 utilons, if it contained 2 paperclips then it could be worth 4 utilons, if it contained 20 paperclips then it could be worth 2^20 utilons. The box would then double your utility each day just by adding one physical paperclip.
I still think these kinds of considerations are worth thinking about though. Your utility function might grow faster than a busy beaver function, but then the doubling box is going to have trouble waiting the right length of time to deliver the
After considering this problem, what I found was surprisingly fast, the specifics of the boxes physical abilities and implementation becomes relevant. I mean, let’s say Clippy is given this box, and has already decided to wait a mere 1 year from day 1, which is 365.25 days of doubling, and 1 paperclip is 1 utilon. At some point, during this time, before the end of it, There are more paperclips then there used to be every atom in the visible universe. Since he’s predicted to gain 2^365.25 paperclips, (which is apparently close to 8.9*10^109) and the observable universe is only estimated to contain 10^80 atoms. So to make up for that, let’s say the box converts every visible subatomic particle into paperclips instead.
That’s just 1 year, and the box has already announced it will convert approximately every visible subatomic particle into pure paperclip bliss!
And then another single doubling… (1 year and 1 day) Does what? Even if Clippy has his utility function unbounded, it should presumably still link back to some kind of physical state, and at this point the box starts having to implement increasingly physically impossible ideas to have to double paperclip utility, like:
Breaking the speed of light.
Expanding the paperclip conversion into the past.
Expanding the paperclip conversion into additional branches of many worlds.
Magically protecting the paperclips from the ravages of time, physics, or condensing into blackholes, despite the fact it is supposed to lose all power after being opened.
And that’s just 1 year! We aren’t even close to a timeless eternity of waiting yet, and the box already has to smash the currently known laws of physics (more so than it did by converting every visible subatomic particle into paperclips) to do more doublings, and will then lose power afterwards.
Do the laws of physics resume being normal after the box loses power? If so, massive chunks of utility will fade away almost instantly (which would seem to indicate the Box was not very effective), but if not I’m not sure how the loop below would get resolved:
The Box is essentially going to rewrite the rules of the universe permanently,
Which would affect your utility calculations, which are based on physics,
Which would affect how the Box rewrote the rules of the universe,
Which would affect your utility calculations, which are based on physics,
Except instead of stopping and letting you and the box resolve this loop, it must keeps doubling, so it keeps changing physics more.
By year 2, it seems like you might be left with either:
A solution, in which case whatever the box will rewrite the laws of physics to, you understand and agree with it and can work on the problem based on whatever that solution is. (But I have no idea how you could figure out what this solution would be in advance, since it depends on the specific box?)
Or, an incredibly intractable metaphysics problem which is growing more complicated faster than you can ever calculate, in which case you don’t even understand what the box is doing anymore.
The reason I said that this was incredibly fast is that my original guess was that it would take at least 100 years of daily doubling for the proposed world to become that complicated, but when I tried doing a bit of math it didn’t take anywhere near that long.
Edit: Fixed a few typos and cleared up grammar.
This is a thought experiment which is not meant to be possible in our world. But such thought experiments are a way of testing the generality of your decision procedures—do they work in all possible worlds? If you must imagine a physics that makes the eternal doubling possible, try picturing a network of replicating baby universes linked by wormholes.
As in the old saw, part of your strength as a real decision-maker is that your decision procedures choose less well in impossible worlds than in possible worlds.
A world that can support paperclip production of arbitrary magnitude is not an impossible world. The speed of light is a contingent fact.
Why does that have to be true?
It doesn’t have to be true. It’s desirable because decision procedures that rely on other knowledge about reality are faster/better/cheaper than ones that don’t import knowledge about reality. Specialization for the situation you find yourself in is often useful, though it does limit flexibility.
Utility doesn’t have to be proportional to the amount of some particular kind of physical stuff in the universe. If the universe contained 1 paperclip, that could be worth 2 utilons, if it contained 2 paperclips then it could be worth 4 utilons, if it contained 20 paperclips then it could be worth 2^20 utilons. The box would then double your utility each day just by adding one physical paperclip.
I still think these kinds of considerations are worth thinking about though. Your utility function might grow faster than a busy beaver function, but then the doubling box is going to have trouble waiting the right length of time to deliver the