Another thing whose True Name is probably a key ingredient for alignment (and which I’ve spent a lot of time trying to think rigorously about): collective values.
Which is interesting, because most of what we know so far about collective values is that, for naive definitions of “collective” and “values”, they don’t exist. Condorcet, Arrow, Gibbard and Satterthwaite, and (crucially) Sen have all helped show that.
I personally don’t think that means that the only useful things one can say about “collective values” are negative results like the ones above. I think there are positive things to say; definitions of collectivity (for instance, of democracy) that are both non-trivial and robust. But finding them means abandoning the naive concepts of “collective values”.
I think that this is probably a common pattern. You go looking for the True Name of X, but even if that search ever bears fruit, you’d rarely if ever look back and say “Y is the True Name of X”. Instead, you’d say something like “(long math notation) is the True Name of itself, or for short, of Y. Though I found this by looking for X, calling it ‘X’ was actually a misnomer; that phrase has baked-in misconceptions and/or red herrings, so from now on, let’s call it ‘Y’ instead.”
I’m pretty sure there is no such thing as collective values. Individual egregores (distributed agents running on human wetware, like governments, religions, businesses, etc) can have coherent values, but groups of people in general do not. Rather, there are more and less optimal (in the sense of causing minimal total regret—I’m probably thinking of Pareto optimality here) mechanisms for compromising.
The “collective values” that emerge are the result of the process, not something inherent before the process begins, and further, different processes will lead to different “collective values”, the same way that different ways of thinking and making decisions will lead a person to prioritize their various desires / subagents differently.
It does look, though, as if some mechanisms for compromising work better than others. Markets and democracies work very differently, but nearly everyone agrees either one is better than dictatorship.
Another thing whose True Name is probably a key ingredient for alignment (and which I’ve spent a lot of time trying to think rigorously about): collective values.
Which is interesting, because most of what we know so far about collective values is that, for naive definitions of “collective” and “values”, they don’t exist. Condorcet, Arrow, Gibbard and Satterthwaite, and (crucially) Sen have all helped show that.
I personally don’t think that means that the only useful things one can say about “collective values” are negative results like the ones above. I think there are positive things to say; definitions of collectivity (for instance, of democracy) that are both non-trivial and robust. But finding them means abandoning the naive concepts of “collective values”.
I think that this is probably a common pattern. You go looking for the True Name of X, but even if that search ever bears fruit, you’d rarely if ever look back and say “Y is the True Name of X”. Instead, you’d say something like “(long math notation) is the True Name of itself, or for short, of Y. Though I found this by looking for X, calling it ‘X’ was actually a misnomer; that phrase has baked-in misconceptions and/or red herrings, so from now on, let’s call it ‘Y’ instead.”
I’m pretty sure there is no such thing as collective values. Individual egregores (distributed agents running on human wetware, like governments, religions, businesses, etc) can have coherent values, but groups of people in general do not. Rather, there are more and less optimal (in the sense of causing minimal total regret—I’m probably thinking of Pareto optimality here) mechanisms for compromising.
The “collective values” that emerge are the result of the process, not something inherent before the process begins, and further, different processes will lead to different “collective values”, the same way that different ways of thinking and making decisions will lead a person to prioritize their various desires / subagents differently.
It does look, though, as if some mechanisms for compromising work better than others. Markets and democracies work very differently, but nearly everyone agrees either one is better than dictatorship.