When they saw their old cues, however, they unthinkingly pressed the lever and ate the food, or they walked across the floor, even as they vomited or jumped from the electricity. The habit was so ingrained the mice couldn’t stop themselves.
I’m not sure whether to believe this applies to more complex mammals. The McDonald’s example is problematic, because the punishment isn’t immediate. Were there any other examples of what happens if you replace/follow the reward with punishment? Replacing every harmful habit with a useful one seems like a lot of work.
I’ve been thinking about self-administering mild electric shocks or something of the like when I pursue an unwanted behaviour, but lack an appropriate device. Any suggestions, anyone?
People with damage to the basal ganglia
Usually caused by Parkinson’s disease. Using a familiar word might give people a better idea of what you’re talking about.
I’ve been thinking about self-administering mild electric shocks or something of the like when I pursue an unwanted behaviour, but lack an appropriate device. Any suggestions, anyone?
Note: self-punishment is particularly useless; you train down the act of punishing yourself more than you train down whatever behavior you’re punishing. This is unpleasant and useless!
Note: self-punishment is particularly useless; you train down the act of punishing yourself more than you train down whatever behavior you’re punishing. This is unpleasant and useless!
Ironically, this doesn’t work the same way for self-talk punishment. It’s still useless, but for some reason it doesn’t train down the self-punishment, it just fails to do anything about what you’re punishing.
When I’m doing self-talk punishment, it really doesn’t feel like I’m punishing myself, but rather berating some other part of my brain that did the wrong thing. Now that I tried the physical punishment, it feels like I’m punishing myself for something that my true identity didn’t do.
When I thought about mild electric shocks, I wasn’t really thinking about anything as powerful as a dog collar. Also I was thinking about applying it somewhere safer than my neck, like my arm. The punishment has to be self administered and not automated (whatever that would mean), to get the timing right.
self-punishment is particularly useless …
Seems like a real risk, but he seems to offer little support for it. I think I’ll know when to stop if it isn’t working. I think the likeliest thing to happen is I grow averse to it like gwern cites in the example.
Seems like a real risk, but he seems to offer little support for it
Read the original book. Punishment is useless, you want negative reinforcement, and yes there is a difference.
“Punishment” is something bad that happens when you do something. “Negative reinforcement” is something bad that goes away when you stop doing something.
The trick is that brains have a kind of reinforcement kluge: instead of having an “avoid this, it’s painful” circuit, we are reinforced by positive changes, including the removal of a negative stimulus.
So technically, the thing about punishment is, it’s not really punishment. Animals and people don’t learn to stop doing something in response to punishment, they learn to do whatever makes the punishment stop the quickest. If this happens to be avoiding the thing being punished, it’s purely a matter of luck. They may also learn to say, hide their behavior from whoever’s punishing it, run away, etc.
So the catch to all this is that self-punishment is useless because the fastest way to stop the punishment is just to stop punishing yourself in the first place. The only consistent self-punishment people can apply is the kind they’ve been trained to do by someone else—i.e., the kind that they got rewarded (or negatively reinforced) for doing.
(Well, technically I suppose you might be able to train yourself to continue punishing yourself by rewarding yourself for punishing yourself, but...)
If this happens to be avoiding the thing being punished, it’s purely a matter of luck. They may also learn to say, hide their behavior from whoever’s punishing it, run away, etc.
Indeed, they will often learn all of these at once, and then the punisher must do extra work to negate the latter set. So, yeah, negative reinforcement typically works better than positive punishment in the long run. Negative punishment (that is, removing something good when I do something) can work OK too, though it has some of the same problems. Training an incompatible behavior via positive reinforcement is often faster, though sometimes not an option.
Positive punishment is done with a noxious stimulus. Negative punishment is taking away a rewarding stimulus. Has worked wonders with my little brother in quenching unwanted i.e. violent, behaviour. Worked well for me too when I was a kid. Usually applied by taking away a favorite toy or activity for a while, and explaining why it’s happening.
Not an expert, but I believe the distinction is that such abstract punishments as taking away toys effectively provide a motivation to change the behavior, effectively incentivizing the punished to try to change the habit. This can only work insofar as the punished is able to recognize the unwanted behavior and meaningfully control their response to it. This is inherently different from directly rewarding or punishing the behavior, and it certainly doesn’t work on any animal besides humans.
I agree. This also implicates abstract punishment works differently for different developmental ages. Abstractly punishing kids too young enough to understand it is just cruel, and it’s just a stupid way to punish older kids who understand it too well.
You’re clearly using positive, not negative punishment.
Mostly negative punishment (removing location privileges and so typically most activities that the child likes to do). The ultimatum was of similarly marginal positive punishment.
So the catch to all this is that self-punishment is useless because the fastest way to stop the punishment is just to stop punishing yourself in the first place. The only consistent self-punishment people can apply is the kind they’ve been trained to do by someone else—i.e., the kind that they got rewarded (or negatively reinforced) for doing.
Why doesn’t negative reinforcement have the same problem?
Why doesn’t negative reinforcement have the same problem?
Self-applied negative reinforcement has the same problem, for the most part: the fastest way to stop it is to just stop doing it.
The reason that negative reinforcement applied by others works (when it works), is because the fastest way to make it stop is to comply with the wishes of the one applying it.
Ideally, the act of compliance itself should make the disturbance stop, in the way that a properly used rein or choke chain on an animal should produce relief from the restraint as soon as the animal stops, turns, etc.
For a more human-relevant example, one can imagine a parent’s frowning look as the child approaches a vase—a frown that goes away the instant the tiny hand withdraws.
(Whether this specific example of training is actually a good idea is an entirely separate question from whether it’s effective.)
I’ve been thinking about self-administering mild electric shocks or something of the like when I pursue an unwanted behaviour, but lack an appropriate device. Any suggestions, anyone?
I’ve heard of someone having success wearing an elastic band around his wrist and snapping it as punishment.
This sounds doable and less crazy and expensive than an electric shock device. I think the sensation caused is pretty similar to an electric shock, too.
Thanks for the thought provoking article!
I’m not sure whether to believe this applies to more complex mammals. The McDonald’s example is problematic, because the punishment isn’t immediate. Were there any other examples of what happens if you replace/follow the reward with punishment? Replacing every harmful habit with a useful one seems like a lot of work.
I’ve been thinking about self-administering mild electric shocks or something of the like when I pursue an unwanted behaviour, but lack an appropriate device. Any suggestions, anyone?
Usually caused by Parkinson’s disease. Using a familiar word might give people a better idea of what you’re talking about.
This has been discussed here previously. Short answer: don’t. Also, from a LWer’s excellent summary of Don’t Shoot the Dog:
Ironically, this doesn’t work the same way for self-talk punishment. It’s still useless, but for some reason it doesn’t train down the self-punishment, it just fails to do anything about what you’re punishing.
When I’m doing self-talk punishment, it really doesn’t feel like I’m punishing myself, but rather berating some other part of my brain that did the wrong thing. Now that I tried the physical punishment, it feels like I’m punishing myself for something that my true identity didn’t do.
When I thought about mild electric shocks, I wasn’t really thinking about anything as powerful as a dog collar. Also I was thinking about applying it somewhere safer than my neck, like my arm. The punishment has to be self administered and not automated (whatever that would mean), to get the timing right.
Seems like a real risk, but he seems to offer little support for it. I think I’ll know when to stop if it isn’t working. I think the likeliest thing to happen is I grow averse to it like gwern cites in the example.
Thanks for bringing that summary to my attention.
Read the original book. Punishment is useless, you want negative reinforcement, and yes there is a difference.
“Punishment” is something bad that happens when you do something. “Negative reinforcement” is something bad that goes away when you stop doing something.
The trick is that brains have a kind of reinforcement kluge: instead of having an “avoid this, it’s painful” circuit, we are reinforced by positive changes, including the removal of a negative stimulus.
So technically, the thing about punishment is, it’s not really punishment. Animals and people don’t learn to stop doing something in response to punishment, they learn to do whatever makes the punishment stop the quickest. If this happens to be avoiding the thing being punished, it’s purely a matter of luck. They may also learn to say, hide their behavior from whoever’s punishing it, run away, etc.
So the catch to all this is that self-punishment is useless because the fastest way to stop the punishment is just to stop punishing yourself in the first place. The only consistent self-punishment people can apply is the kind they’ve been trained to do by someone else—i.e., the kind that they got rewarded (or negatively reinforced) for doing.
(Well, technically I suppose you might be able to train yourself to continue punishing yourself by rewarding yourself for punishing yourself, but...)
Indeed, they will often learn all of these at once, and then the punisher must do extra work to negate the latter set. So, yeah, negative reinforcement typically works better than positive punishment in the long run.
Negative punishment (that is, removing something good when I do something) can work OK too, though it has some of the same problems.
Training an incompatible behavior via positive reinforcement is often faster, though sometimes not an option.
Ok. I’m going to read the book. If I don’t keep reading, I’ll slap myself furiously with rubber bands.
In my experience, negative punishment works very well with children. Any takes on that?
I’m not sure whether you’re joking, serious, or being sarcastic.
I don’t know what you mean by “negative punishment”, nor what you mean by “works very well”. Works very well to accomplish what, specifically?
Just joking with good intentions.
here’s a nice diagram of what I’m talking about.
Positive punishment is done with a noxious stimulus. Negative punishment is taking away a rewarding stimulus. Has worked wonders with my little brother in quenching unwanted i.e. violent, behaviour. Worked well for me too when I was a kid. Usually applied by taking away a favorite toy or activity for a while, and explaining why it’s happening.
Not an expert, but I believe the distinction is that such abstract punishments as taking away toys effectively provide a motivation to change the behavior, effectively incentivizing the punished to try to change the habit. This can only work insofar as the punished is able to recognize the unwanted behavior and meaningfully control their response to it. This is inherently different from directly rewarding or punishing the behavior, and it certainly doesn’t work on any animal besides humans.
I agree. This also implicates abstract punishment works differently for different developmental ages. Abstractly punishing kids too young enough to understand it is just cruel, and it’s just a stupid way to punish older kids who understand it too well.
It works, but poorly.
Got a bit emotional. Sorry about that.
Thanks for letting me know.
ETA: Anyone else notice (besides wedrifid, obviously), that this thread is full of claims not backed at all, and inexplicable upvotes, to boot?
You are being silly, go to your room and behave or you’ll get a spanking.
You’re clearly using positive, not negative punishment. So am I :P
Mostly negative punishment (removing location privileges and so typically most activities that the child likes to do). The ultimatum was of similarly marginal positive punishment.
I got that. I was talking about what you were actually doing to me.
Why doesn’t negative reinforcement have the same problem?
Self-applied negative reinforcement has the same problem, for the most part: the fastest way to stop it is to just stop doing it.
The reason that negative reinforcement applied by others works (when it works), is because the fastest way to make it stop is to comply with the wishes of the one applying it.
Ideally, the act of compliance itself should make the disturbance stop, in the way that a properly used rein or choke chain on an animal should produce relief from the restraint as soon as the animal stops, turns, etc.
For a more human-relevant example, one can imagine a parent’s frowning look as the child approaches a vase—a frown that goes away the instant the tiny hand withdraws.
(Whether this specific example of training is actually a good idea is an entirely separate question from whether it’s effective.)
I’ve heard of someone having success wearing an elastic band around his wrist and snapping it as punishment.
This sounds doable and less crazy and expensive than an electric shock device. I think the sensation caused is pretty similar to an electric shock, too.