I think the main reason Moloch doesn’t succeed very effectively is just because the common response to “hey, you could sacrifice everything of value and give up all slack to optimize X” is “yeah but have you considered just, yanno, hanging out and watching TV?”
And most people who optimize X aren’t actually becoming more competitive in the grand scheme of things. They’ll die (or hopefully not die) like everybody else and probably have roughly the same number of kids. The selection process that created humans in the first place won’t even favor them!
As a result, I’m not worried about Moloch imminently taking over the world. Frankly, I’m more short-term concerned with people just, yanno, hanging out and watching TV when this world is abjectly horrifying.
I am long-term concerned about Moloch as it pertains to value-drift. I doubt the sound of Moloch will be something like “people giving up all value to optimize X” and expect it to be something more like “thousands of years go by and eventually people just stop having our values.”
It’s more effective to retain more values since physics is basically unitary (at least up to the point we know) so you’ll have more people on your side if you retain the values of past people. So we’d be able to defeat this Moloch if we’re careful.
To be clear, the effectiveness of an action is defined by whatever values we use to make that judgement. Retaining the values of past people is not effective unless
past-people values positively compliment your current values so you can positively leverage the work of past people by adopting more of their value systems (which doesn’t necessarily mean you have to adopt their values)
past-people have coordinated to limit the instrumental capabilities of anyone who doesn’t have their values (for instance, by establishing a Nash equilibrium that makes it really hard for people to express drifting values or by building an AGI)
To be fair, maybe you’re referring to Molochian effectiveness of the form (whatever things tend to maximize the existence of similar thnigs). For humans, similarity is a complicated measure. Do we care about memetic similarity (ie reproducing people with similar attitudes as ourselves) or genetic similarity (ie having more kids)? Of course, this is a nonsense question because the answer is most humans don’t care strongly about either and we don’t really have any psychological intuitions on the matter (I guess you could argue hedonic utilitarianism can be Molochian under certain assumptions but that’s just because any strongly-optimizing morality becomes Molochian).
In the former case (memetic similarity), adopting values of past people is a strategy that makes you less fit because you’re sacrificing your memetics to more competitive ones. In the latter case (genetic similarity), pretending to adopt people’s values as a way to get them to have more kids with you is more dominant than just adopting their values.
But, overall, I agree that we could kind-of beat Moloch (in the sense of curbing Moloch on really long time-scales) just by setting up our values to be inherently more Molochian than those of people in the future. Effective altruism is actually a pretty good example of this. Utilitarian optimizers leveraging the far-future to manipulate things like value-drift over long-periods of time seem more memetically competitive than other value-sets.
I think the main reason Moloch doesn’t succeed very effectively is just because the common response to “hey, you could sacrifice everything of value and give up all slack to optimize X” is “yeah but have you considered just, yanno, hanging out and watching TV?”
And most people who optimize X aren’t actually becoming more competitive in the grand scheme of things. They’ll die (or hopefully not die) like everybody else and probably have roughly the same number of kids. The selection process that created humans in the first place won’t even favor them!
As a result, I’m not worried about Moloch imminently taking over the world. Frankly, I’m more short-term concerned with people just, yanno, hanging out and watching TV when this world is abjectly horrifying.
I am long-term concerned about Moloch as it pertains to value-drift. I doubt the sound of Moloch will be something like “people giving up all value to optimize X” and expect it to be something more like “thousands of years go by and eventually people just stop having our values.”
It’s more effective to retain more values since physics is basically unitary (at least up to the point we know) so you’ll have more people on your side if you retain the values of past people. So we’d be able to defeat this Moloch if we’re careful.
To be clear, the effectiveness of an action is defined by whatever values we use to make that judgement. Retaining the values of past people is not effective unless
past-people values positively compliment your current values so you can positively leverage the work of past people by adopting more of their value systems (which doesn’t necessarily mean you have to adopt their values)
past-people have coordinated to limit the instrumental capabilities of anyone who doesn’t have their values (for instance, by establishing a Nash equilibrium that makes it really hard for people to express drifting values or by building an AGI)
To be fair, maybe you’re referring to Molochian effectiveness of the form (whatever things tend to maximize the existence of similar thnigs). For humans, similarity is a complicated measure. Do we care about memetic similarity (ie reproducing people with similar attitudes as ourselves) or genetic similarity (ie having more kids)? Of course, this is a nonsense question because the answer is most humans don’t care strongly about either and we don’t really have any psychological intuitions on the matter (I guess you could argue hedonic utilitarianism can be Molochian under certain assumptions but that’s just because any strongly-optimizing morality becomes Molochian).
In the former case (memetic similarity), adopting values of past people is a strategy that makes you less fit because you’re sacrificing your memetics to more competitive ones. In the latter case (genetic similarity), pretending to adopt people’s values as a way to get them to have more kids with you is more dominant than just adopting their values.
But, overall, I agree that we could kind-of beat Moloch (in the sense of curbing Moloch on really long time-scales) just by setting up our values to be inherently more Molochian than those of people in the future. Effective altruism is actually a pretty good example of this. Utilitarian optimizers leveraging the far-future to manipulate things like value-drift over long-periods of time seem more memetically competitive than other value-sets.