With all the weaknesses and foibles of human nature; though we might try to mitigate those weaknesses, one of the biggest weaknesses in human nature is the belief that we have already mitigated those weaknesses, leading us to stop trying.
Except that what’s being discussed here is the exploitation of those weaknesses, not their mitigation. And seeking to exploit those weaknesses as an end in and of itself leads to a particular kind of affective death spiral that rationalists claim to want to avoid, so I’m trying to raise a “what’s up with that?” signal before a particular set of adverse cultural values lock in.
Ah, I see; so while I’m saying that I expect that some exploitation will happen with high probability in any sufficiently large social group, you are trying to point out the negative side of said exploitation and thus cut it off, or at least reduce it, at an early stage.
Except that what’s being discussed here is the exploitation of those weaknesses, not their mitigation. And seeking to exploit those weaknesses as an end in and of itself leads to a particular kind of affective death spiral that rationalists claim to want to avoid, so I’m trying to raise a “what’s up with that?” signal before a particular set of adverse cultural values lock in.
Ah, I see; so while I’m saying that I expect that some exploitation will happen with high probability in any sufficiently large social group, you are trying to point out the negative side of said exploitation and thus cut it off, or at least reduce it, at an early stage.