Nick, note that he treats the pebblesorters in parallel with the humans. The pebblesorters’ values lead them to seek primeness and Eliezer optimistically supposes that human values lead humans to seek an analogous rightness.
What Eliezer is trying to say in that post, I think, is that he would not consider it right to eat babies even conditional on humanity being changed by the babyeaters to have their values.
But the choice to seek rightness instead of rightness’ depends on humans having values that lead to rightness instead of rightness’.
Psy-Kosh: I was using the example of pure baby eater values and conscious babies to illustrate the post Nick Tarleton linked to rather than apply it to this one.
Michael: if it’s “inevitable” that they will encounter aliens then it’s inevitable that each fragment will in turn encounter aliens, unless they do some ongoing pre-emptive fragmentation, no? But even then, if exponential growth is the norm among even some alien species (which one would expect) the universe should eventually become saturated with civilizations. In the long run, the only escape is opening every possible line from a chosen star and blowing up all the stars at the other ends of the lines.
Hmm. I guess that’s an argument in favour of cooperating with the superhappies. Though I wonder if they would still want to adopt babyeater values if the babyeaters were cut off, and if the ship would be capable of doing that against babyeater resistance.