There exists a clear misalignment between the principles of evolution and human behavior. This discrepancy is evident in humans’ inclination towards pursuing immediate gratification, often referred to as “wireheading,” rather than prioritizing the fundamental goals of replication and survival.
An illustrative example of this misalignment can be observed in the unfortunate demise of certain indigenous communities due to excessive alcohol consumption. This tragic outcome serves as a poignant reminder of the consequences of prioritizing immediate pleasure over long-term sustainability.
In contemporary society, the majority of humans exhibit a preference for a high-quality and pleasurable life over the singular focus on reproduction. This shift in priorities has begun to exert a discernible impact on global fertility rates.
In fact, one could view the historical trajectory of civilization as a progression towards increasingly effective methods of wireheading, essentially the stimulation of pleasure centers in the human brain. Initially, the negative repercussions of such behavior were confined to specific individuals and groups, exerting limited global effects. However, as our capacity for wireheading continues to advance, there is a growing concern that it may gradually influence the entirety of human civilization.
In essence, the tension between our evolutionary legacy and our pursuit of immediate gratification is an issue that transcends individual actions and has the potential to reshape the course of our entire society.
“The training loop is still running” is not really a counterargument to “the training loop accidentally spat out something smarter than the training loop, that wants stuff other than the stuff the training loop is optimizing for, that iterates at a much faster speed than the training, and looks like it’s just about to achieve total escape velocity”.
I don’t think it looks like we are about to achieve escape velocity. The broader laws of natural selection and survival of the fittest even apply to future AI and Grabby Aliens. Essentially the argument from “Meditations On Moloch”.
The broader laws of natural selection and survival of the fittest
These are distinct from biological evolution, so if our descendants end up being optimized over by them rather than by biological evolution, that feels like it’s conceding the argument (that we’ll have achieved escape velocity).
There exists a clear misalignment between the principles of evolution and human behavior. This discrepancy is evident in humans’ inclination towards pursuing immediate gratification, often referred to as “wireheading,” rather than prioritizing the fundamental goals of replication and survival.
An illustrative example of this misalignment can be observed in the unfortunate demise of certain indigenous communities due to excessive alcohol consumption. This tragic outcome serves as a poignant reminder of the consequences of prioritizing immediate pleasure over long-term sustainability.
In contemporary society, the majority of humans exhibit a preference for a high-quality and pleasurable life over the singular focus on reproduction. This shift in priorities has begun to exert a discernible impact on global fertility rates.
In fact, one could view the historical trajectory of civilization as a progression towards increasingly effective methods of wireheading, essentially the stimulation of pleasure centers in the human brain. Initially, the negative repercussions of such behavior were confined to specific individuals and groups, exerting limited global effects. However, as our capacity for wireheading continues to advance, there is a growing concern that it may gradually influence the entirety of human civilization.
In essence, the tension between our evolutionary legacy and our pursuit of immediate gratification is an issue that transcends individual actions and has the potential to reshape the course of our entire society.
But natural selection favors people who prefer having children, and various forms of wireheading eventually stop working if they decrease fitness.
“The training loop is still running” is not really a counterargument to “the training loop accidentally spat out something smarter than the training loop, that wants stuff other than the stuff the training loop is optimizing for, that iterates at a much faster speed than the training, and looks like it’s just about to achieve total escape velocity”.
I don’t think it looks like we are about to achieve escape velocity. The broader laws of natural selection and survival of the fittest even apply to future AI and Grabby Aliens. Essentially the argument from “Meditations On Moloch”.
These are distinct from biological evolution, so if our descendants end up being optimized over by them rather than by biological evolution, that feels like it’s conceding the argument (that we’ll have achieved escape velocity).