There is an aspect you did not mention, which seems important to me: it is easier to change minds by peer pressure than by one person’s arguments. The evolutionary story is that in case of a conflict, we “want” to join the stronger side; that’s how we more likely survive the fight.
Therefore, the greatest problem of one person’s arguments, no matter how smart or convincing, is that ultimately they are one person’s arguments. Even worse if they are merely your own. Your brain keeps waiting for greater social approval.
Cults, on the other hand, create the feeling of almost universal social approval. They often achieve it by cheating, for example they discourage talking to outsiders and reading/watching mainstream sources, so that everyone you interact with supports the same ideas. And this can change your behavior dramatically.
People often change their minds dramatically when they change their environment, for example when they move from their family to a campus.
Therefore, if you want to change your behavior, I think it would help to have a group that would hold you accountable for your actions.
Good catch, didn’t think of that. Definitely seems like peer pressure is a better way to change minds rather than one-on-one. This is still parasitism, though—I don’t know if I’d trust most people to form a group to hold me accountable for changes in my behavior. Seems too easy for them to, intentionally or not, shape my request into ways that benefit them.
For example, I might form a group to help me lose weight. I care very much about my physical wellbeing and reducing discomfort, but they might care more about my ugly appearance and assume that’s what I’m going for, too. Worse yet, my discomfort is invisible to them, and my ugliness in their eyes is invisible to me!
Certainly not an insurmountable obstacle, of course, but one to be aware of.
EDIT: I read your paragraph on cults and then completely ignored it when writing my response. Of course you know that peer pressure can be bad, you said it yourself. My mistake.
Seems too easy for them to, intentionally or not, shape my request into ways that benefit them.
Yes. (Alignment problems everywhere.) It is better if your goal is measurable somehow, so you could provide a report with numbers, and the audience would… clap if the numbers increase, or something.
“Losing weight to feel comfortable” is like the opposite of this, and it takes a lot of time. Probably would need to replace it with an instrumental goal such as “get weight from X to Y” (to make it obvious it is not your goal to keep going below Y; getting to Y counts as success full stop). And there may be other things that could make you comfortable, for example buying softer shoes. Or, exercise could improve your muscles and make you feel better, without actually losing weight.
Another possible approach is to reward work, not outcomes. Like, you could make a plan “exercise twice a week, stop drinking soda”, and then just report every week whether you did this or not. The group would reward the effort.
All approaches have their disadvantages (e.g. the work you reward may actually not lead to the desired goal), but if it’s up to you to define and change your goals, you can try different things and see what works.
This is good stuff, thank you. I think these are all good ways to avoid the trap of letting others decide your goals for you, and I like the idea of continuously changing your goals if you find they aren’t working/have been Goodharted/etc.
There is an aspect you did not mention, which seems important to me: it is easier to change minds by peer pressure than by one person’s arguments. The evolutionary story is that in case of a conflict, we “want” to join the stronger side; that’s how we more likely survive the fight.
Therefore, the greatest problem of one person’s arguments, no matter how smart or convincing, is that ultimately they are one person’s arguments. Even worse if they are merely your own. Your brain keeps waiting for greater social approval.
Cults, on the other hand, create the feeling of almost universal social approval. They often achieve it by cheating, for example they discourage talking to outsiders and reading/watching mainstream sources, so that everyone you interact with supports the same ideas. And this can change your behavior dramatically.
People often change their minds dramatically when they change their environment, for example when they move from their family to a campus.
Therefore, if you want to change your behavior, I think it would help to have a group that would hold you accountable for your actions.
Good catch, didn’t think of that. Definitely seems like peer pressure is a better way to change minds rather than one-on-one. This is still parasitism, though—I don’t know if I’d trust most people to form a group to hold me accountable for changes in my behavior. Seems too easy for them to, intentionally or not, shape my request into ways that benefit them.
For example, I might form a group to help me lose weight. I care very much about my physical wellbeing and reducing discomfort, but they might care more about my ugly appearance and assume that’s what I’m going for, too. Worse yet, my discomfort is invisible to them, and my ugliness in their eyes is invisible to me!
Certainly not an insurmountable obstacle, of course, but one to be aware of.
EDIT: I read your paragraph on cults and then completely ignored it when writing my response. Of course you know that peer pressure can be bad, you said it yourself. My mistake.
Yes. (Alignment problems everywhere.) It is better if your goal is measurable somehow, so you could provide a report with numbers, and the audience would… clap if the numbers increase, or something.
“Losing weight to feel comfortable” is like the opposite of this, and it takes a lot of time. Probably would need to replace it with an instrumental goal such as “get weight from X to Y” (to make it obvious it is not your goal to keep going below Y; getting to Y counts as success full stop). And there may be other things that could make you comfortable, for example buying softer shoes. Or, exercise could improve your muscles and make you feel better, without actually losing weight.
Another possible approach is to reward work, not outcomes. Like, you could make a plan “exercise twice a week, stop drinking soda”, and then just report every week whether you did this or not. The group would reward the effort.
All approaches have their disadvantages (e.g. the work you reward may actually not lead to the desired goal), but if it’s up to you to define and change your goals, you can try different things and see what works.
This is good stuff, thank you. I think these are all good ways to avoid the trap of letting others decide your goals for you, and I like the idea of continuously changing your goals if you find they aren’t working/have been Goodharted/etc.