I would choose the paperclipper, but not because I value its intelligence—paperclips are a human invention, and so the paperclipping AI represents a sort of memorial to humanity. A sign that humans once existed, that might last until the heat death of the universe.
My preference for this may perhaps be caused by a confusion, since this is effectively an aesthetic choice for a universe I will not be able to observe, but if other intelligences of a sort that I actually value will not be present either way, this problem doesn’t matter as long as it gives me reason enough to prefer one over the other.
I think the point where I start to not prefer the paperclipper is somewhere between the trilobites and the australopithecines, closer to the australopithecine end of that.
I would choose the paperclipper, but not because I value its intelligence—paperclips are a human invention, and so the paperclipping AI represents a sort of memorial to humanity. A sign that humans once existed, that might last until the heat death of the universe.
My preference for this may perhaps be caused by a confusion, since this is effectively an aesthetic choice for a universe I will not be able to observe, but if other intelligences of a sort that I actually value will not be present either way, this problem doesn’t matter as long as it gives me reason enough to prefer one over the other.
I think the point where I start to not prefer the paperclipper is somewhere between the trilobites and the australopithecines, closer to the australopithecine end of that.