The value systems we revile yet can’t prove wrong (paperclipping and wireheading) are both evolutionary dead-ends. This suggests that blind evolution still implements our values better than our reason does; and allowing evolution to proceed is still better than computing a plan of action with our present level of understanding.
I wonder if our problem with wireheading isn’t just the traditional ethic that sloth and gluttony are vices and hard-work a virtue.
I agree. I think that we’re conditioned at a young age, if not genetically, to be skeptical about the feasibility of long-term hedonism. While the ants were working hard collecting grain, the grasshopper was having fun playing music—and then the winter came. In our case, I think we’re genuinely afraid that while we’re wireheading, we’ll be caught unaware and unprepared for a real world threat. Even if some subset of the population wire-headed while others ‘manned the fort’, I wonder if Less Wrong selects for a personality type that would prefer the manning, or if our rates of non-wireheading aren’t any higher.
I wonder if our problem with wireheading isn’t just the traditional ethic that sloth and gluttony are vices and hard-work a virtue.
I agree. I think that we’re conditioned at a young age, if not genetically, to be skeptical about the feasibility of long-term hedonism. While the ants were working hard collecting grain, the grasshopper was having fun playing music—and then the winter came. In our case, I think we’re genuinely afraid that while we’re wireheading, we’ll be caught unaware and unprepared for a real world threat. Even if some subset of the population wire-headed while others ‘manned the fort’, I wonder if Less Wrong selects for a personality type that would prefer the manning, or if our rates of non-wireheading aren’t any higher.
More comments on this topic in this thread.