This has long been my biggest worry for the future. Pleasant/attractive future experiences of intelligent agents (whether human, artificial, or alien) seem unlikely in highly-competitive environments. This is not because competition itself is unpleasant, but because agents will always seek their own utility (or at least existence if expected utility is positive) over other agents’.
It’s far more general than current cultural threats—my fear is that Scott’s Moloch is a fundamental truth of individual existence. The repugnant conclusion, however repugnant, is the true conclusion of maximizing overall utility within available resources.
The only out I currently see is to give up individuality (or more likely, to have it taken from us by a more powerful utility monster).
Only lower needs are ugly, higher needs include pleasant and attractive aspects of life. Only simple people think that material wealth is true wealth. Sadly, it seems that we’re outnumbered, and that the average human being is at a lower stage of development than in the past. Most people haven’t even begun the process of individualization, yet alone reached the point where petty things offend their taste.
This scale can also be understood as “lower” and “higher”. It’s hard to put into words what exactly is higher and lower, but this difference is a major part of linguistics, in which it’s visible. “Corrupt”, “Filthy”, “Indecent”, “Unrefined”. See how social status, cleanliness, manners, and standards correlate with language with positive evaluations? The vocal minority which indulges in petty identity politics and other such types offend me in the same way. Of course, a tradeoff here is that they can enjoy life much more than I, who considers most things below himself.
I otherwise agree with you, you put it into words much better than I ever could. But I think it’s a fundamental truth of groups more than of individual existence. After all, smaller communities tend to be better than big ones. Moloch might be an essential part of size, so that the problem is the utility monster rather than us. Maybe we’re not maximizing individual utility? We’re working for something else which grants us utility, but which eventually replaces us. If I automate a task, I will feel like I’m increasing my own power, but I’m actually just creating my own replacement, and decreasing my own value in the process.
I have one positive observation to share with you though, it’s that optimizing over longer spans of time removes all of these issues. Moloch is only a problem when people are greedy and optimizing immediate reward. If you were a greedy person destined to live for a million years, you’d likely not contribute to anything which reduces the overall quality of the world. Extend this time span to a quadrillion years, and you’d pretty much have solved the problem with human alignment. The world would be an extension of yourself and damaging it in any way would be hurting yourself. It would still be a “us vs them” situation, but the “us” would include all of humanity, rather than just a tiny part of it, so “them” wouldn’t be your fellow human beings but something like physics and possibly aliens.
(Apologies if necro-posting is frowed upon on here, and please let me know if it is)
This has long been my biggest worry for the future. Pleasant/attractive future experiences of intelligent agents (whether human, artificial, or alien) seem unlikely in highly-competitive environments. This is not because competition itself is unpleasant, but because agents will always seek their own utility (or at least existence if expected utility is positive) over other agents’.
It’s far more general than current cultural threats—my fear is that Scott’s Moloch is a fundamental truth of individual existence. The repugnant conclusion, however repugnant, is the true conclusion of maximizing overall utility within available resources.
The only out I currently see is to give up individuality (or more likely, to have it taken from us by a more powerful utility monster).
Only lower needs are ugly, higher needs include pleasant and attractive aspects of life. Only simple people think that material wealth is true wealth. Sadly, it seems that we’re outnumbered, and that the average human being is at a lower stage of development than in the past. Most people haven’t even begun the process of individualization, yet alone reached the point where petty things offend their taste.
This scale can also be understood as “lower” and “higher”. It’s hard to put into words what exactly is higher and lower, but this difference is a major part of linguistics, in which it’s visible. “Corrupt”, “Filthy”, “Indecent”, “Unrefined”. See how social status, cleanliness, manners, and standards correlate with language with positive evaluations? The vocal minority which indulges in petty identity politics and other such types offend me in the same way. Of course, a tradeoff here is that they can enjoy life much more than I, who considers most things below himself.
I otherwise agree with you, you put it into words much better than I ever could. But I think it’s a fundamental truth of groups more than of individual existence. After all, smaller communities tend to be better than big ones. Moloch might be an essential part of size, so that the problem is the utility monster rather than us. Maybe we’re not maximizing individual utility? We’re working for something else which grants us utility, but which eventually replaces us. If I automate a task, I will feel like I’m increasing my own power, but I’m actually just creating my own replacement, and decreasing my own value in the process.
I have one positive observation to share with you though, it’s that optimizing over longer spans of time removes all of these issues. Moloch is only a problem when people are greedy and optimizing immediate reward. If you were a greedy person destined to live for a million years, you’d likely not contribute to anything which reduces the overall quality of the world. Extend this time span to a quadrillion years, and you’d pretty much have solved the problem with human alignment. The world would be an extension of yourself and damaging it in any way would be hurting yourself. It would still be a “us vs them” situation, but the “us” would include all of humanity, rather than just a tiny part of it, so “them” wouldn’t be your fellow human beings but something like physics and possibly aliens.
(Apologies if necro-posting is frowed upon on here, and please let me know if it is)