First, humanity’s cosmic endowment is astronomically large—there is plenty to go around even if our process involves some waste or accepts some unnecessary constraints. (p227)
This is saying that our values are diminishing enough in stuff that much of the universe doesn’t matter to us. This seems true under some plausible values and not under others. In particular, if we pursue some kind of proportional aggregative consequentialism, then if each individual has diminishing returns, we should create more individuals so that there is not so much to go around.
This issue is complicated by the fact that we don’t really know how much computation our physics will give us access to, or how relevant negentropy is going to be in the long run. In particular, our physics may allow access to (countably or more) infinite computational and storage resources given some superintelligent physics research.
For Expected Utility calculations, this possibility raises the usual issues of evaluating potential infinite utilities. Regardless of how exactly one decides to deal with those issues, the existence of this possibility does shift things in favor of prioritizing for safety over speed.
Seriously though. I’m highly in favor of infinity shades. This whole “let’s burn the universe searching for the Omega point or perpetual machines” makes me unhappy.
This is saying that our values are diminishing enough in stuff that much of the universe doesn’t matter to us. This seems true under some plausible values and not under others. In particular, if we pursue some kind of proportional aggregative consequentialism, then if each individual has diminishing returns, we should create more individuals so that there is not so much to go around.
This issue is complicated by the fact that we don’t really know how much computation our physics will give us access to, or how relevant negentropy is going to be in the long run. In particular, our physics may allow access to (countably or more) infinite computational and storage resources given some superintelligent physics research.
For Expected Utility calculations, this possibility raises the usual issues of evaluating potential infinite utilities. Regardless of how exactly one decides to deal with those issues, the existence of this possibility does shift things in favor of prioritizing for safety over speed.
Infinity Shades for the win!
Seriously though. I’m highly in favor of infinity shades. This whole “let’s burn the universe searching for the Omega point or perpetual machines” makes me unhappy.