Okay, we agree that the simple robot described here is behaviorist and the thermostat is PCT. And I certainly see where you’re coming from with the rats being PCT because hunger only works as a motivator if you’re hungry. But I do have a few questions:
There are some things behaviorism can explain pretty well that I don’t know how to model in PCT. For example, consider heroin addiction. An animal can go its whole life not wanting heroin until it’s exposed to some. Then suddenly heroin becomes extraordinarily motivating and it will preferentially choose shots of heroin to food, water, or almost anything else. What is the PCT explanation of that?
I’m not entirely sure which correlation studies you’re talking about here; most psych studies I read are done in an RCT type design and so use p-values rather than r-values; they can easily end up with p < .001 if they get a large sample and a good hypothesis. Some social psych studies work off of correlations (eg correlation between being observer-rated attractiveness and observer-rated competence at a skill); correlations are “lamentably low” in social psychology because high level processes (like opinion formation, social interaction, etc.) have a lot of noise. Are there any PCT studies of these sorts of processes (not simple motor coordination problems) that have any higher correlation than standard models do? Any with even the same level of correlation?
What’s the difference between control theory and stimulus-response in a context? For example, if we use a simplified version of hunger in which the hormone leptin is produced in response to hunger and the hormone ghrelin is produced in response to satiety, we can explain this in two ways: the body is trying to PCT itself to the perfect balance of leptin and ghrelin, or in the context of the stimulus leptin the response of eating is rewarded and in the context of the stimulus ghrelin the response of eating is punished. Are these the same theory, or are there experiments that would distinguish between them? Do you know of any?
Does PCT still need reinforcement learning to explain why animals use some strategies and not others to achieve equilibrium? For example, when a rat in a Skinner box is hungry (ie its satiety variable has deviated in the direction of hunger), and then it presses a lever and gets a food pellet and its satiety variable goes back to its reference range, would PCTists explain that as getting rewarded for pressing the lever and expect it to press the lever again next time it’s hungry?
An animal can go its whole life not wanting heroin until it’s exposed to some. Then suddenly heroin becomes extraordinarily motivating and it will preferentially choose shots of heroin to food, water, or almost anything else
Summary: An experimenter thought drug addiction in rats might be linked to being kept in distressing conditions, made a Rat Park to test the idea, and found that the rats in the enriched Rat Park environment ignored the morphine on offer.
EDIT: apparently the study had methodological issues and hasn’t been replicated, making the results somewhat suspect, as pointed out by Yvain below
I hate to admit I get science knowledge from Reddit, but the past few times this was posted there it was ripped apart by (people who claimed to be) professionals in the field—riddled with metholodogical errors, inconsistently replicated, et cetera. The fact that even its proponents admit the study was rejected by most journals doesn’t speak well of it.
I think it’s very plausible that situation contributes to addiction; we know that people in terrible situations have higher discount rates than others and so tend to short-term thinking that promotes that kind of behavior, and certainly they have fewer reasons to try to live life as a non-addict. But I think the idea that morphine is no longer interesting and you can’t become addicted when you live a stimulating life is wishful thinking.
Well, like I said, all I have to go on is stuff people said on Reddit and one failed replication study I was able to find somewhere by a grad student of the guy who did the original research. The original research is certainly interesting and relevant and does speak to the problems with a very reductionist model.
This actually gets to the same problem I’m having looking up stuff on perceptual control theory, which is that I expect a controversial theory to be something where there are lots of passionate arguments on both sides, but on both PCT and Rat Park, when I’ve tried to look them up I get a bunch of passionate people arguing that they’re great, and then a few scoffs from more mainstream people saying “That stuff? Nah.” without explaining themselves. I don’t know whether it’s because of Evil Set-In-Their-Ways Mainstream refusing to acknowledge the new ideas, or whether they’re just so completely missing the point that people think it’s not worth their while to respond. It’s a serious problem and I wish that “skeptics” would start addressing this kind of thing instead of debunking ghosts for the ten zillionth time.
Just a brief note to say that I do intend to get back to this, but I’ve been largely offline since the end of last week, and will be very busy at least until the end of this month on things other than LessWrong. I would like to say a lot more about PCT here than I have in the past (here, here, and in various comments), but these things take me long periods of concentrated effort to write.
BTW, one of the things I’m busy with is PCT itself, and I’ll be in Boulder, Colorado for a PCT-related meeting 28-31 July, and staying on there for a few days. Anyone around there then?
For example, when a rat in a Skinner box is hungry (ie its satiety variable has deviated in the direction of hunger), and then it presses a lever and gets a food pellet and its satiety variable goes back to its reference range, would PCTists explain that as getting rewarded for pressing the lever and expect it to press the lever again next time it’s hungry?
The PCT learning model doesn’t require reinforcement at the control level, as its model of memory is a mapping from reference levels to predicted levels of other variables. I.e., when the rat notices that the lever-pressing is paired with food, a link is made between two perceptual variables: the position of the lever, and the availability of food. This means that the rat can learn that food is available, even when it’s not hungry.
Where reinforcement is relevant to PCT is in the strength of the linkage and in the likelihood of its being recorded. If the rat is hungry, then the linkage is more salient, and more likely to be learned.
Notice though, that again the animal’s internal state is of primary importance, not the stimulus/response. In a sense, you could say that you can teach an animal that a stimulus and response are paired, but this isn’t the same as making the animal behave. If we starved you and made you press a lever for your food, you might do it, or you might tell us to fork off. Yet, we don’t claim that you haven’t learned that pressing the lever leads to food in that case.
(As Richard says, it’s well established that you can torture living creatures until they accede to your demands, but it won’t necessarily tell you much about how the creature normally works.)
In any case, PCT allows for the possibility of learning without “reinforcement” in the behaviorist sense, unless you torture the definition of reinforcement to the point that anything is reinforcement.
Regarding the leptin/ghrelin question, my understanding is that PCT as a psych-physical model primarily addresses those perceptual variables that are modeled by neural analog—i.e., an analog level maintained in a neural delay loop. While Powers makes many references to other sorts of negative feedback loops in various organisms from cats to E. coli, the main thrust of his initial book deals with building up a model of what’s going on, feedback-loopwise, in the nervous system and brain, not the body’s endocrine systems.
To put it another way, PCT doesn’t say that control systems are universal, only that they are ubiquitous, and that the bulk of organisms’ neural systems are assembled from a relatively small number of distinct component types that closely resemble the sort of components that humans use when building machinery.
IOW, we should not expect that PCT’s model of neural control systems would be directly applicable to a hormone level issue. However, we can reason from general principles and say that one difference between a PCT model of the leptin/ghrelin question is that PCT includes an explicit model of hierarchy and conflict in control networks, so that we can answer questions about what happens if both leptin and ghrelin are present (for example).
If those signals are at the same level of control hierarchy, we can expect conflict to result in oscillation, where the system alternates between trying to satisfy one or the other. Or, if they’re at different levels of hierarchy, then we can expect one to override the other.
But, unlike a behavioral model where the question of precedence between different stimuli and contexts is open to interpretation, PCT makes some testable predictions about what actually constitutes hierarchy, both in terms of expected behavior, and in terms of the physical structure of the underlying control circuitry.
That is, if you could dissect an organism and find the neurons, PCT predicts a certain type of wiring to exist, i.e., that a dominant controller will have wiring to set the reference levels for lower-level controllers, but not vice-versa.
Second, PCT predicts that a dominant perception must be measured at a longer time scale than a dominated one. That is, the lower-level perception must have a higher sampling rate than the higher-level perception. Thus, for example, as a rat becomes hungrier (a longer-term perceptual variable), its likelihood of pressing a lever to receive food in spite of a shock is increased.
AFAICT, behaviorism can “explain” results like these, but does not actually predict them, in the sense that PCT is spelling out implementation-level details that behaviorism leaves to hand-waving. IOW, PCT is considerably more falsifiable than behaviorism, at least in principle. Eventually, PCT’s remaining predictions (i.e., the ones that haven’t already panned out at the anatomical level) will either be proven or disproven, while behaviorism doesn’t really make anatomical predictions about these matters.
To answer question 3, one could perform the experiment of surgically balancing leptin and ghrelin and not feeding or otherwise nourishing the subject. If the subject eventually dies of starvation, I would say the second theory is more likely.
Okay, we agree that the simple robot described here is behaviorist and the thermostat is PCT. And I certainly see where you’re coming from with the rats being PCT because hunger only works as a motivator if you’re hungry. But I do have a few questions:
There are some things behaviorism can explain pretty well that I don’t know how to model in PCT. For example, consider heroin addiction. An animal can go its whole life not wanting heroin until it’s exposed to some. Then suddenly heroin becomes extraordinarily motivating and it will preferentially choose shots of heroin to food, water, or almost anything else. What is the PCT explanation of that?
I’m not entirely sure which correlation studies you’re talking about here; most psych studies I read are done in an RCT type design and so use p-values rather than r-values; they can easily end up with p < .001 if they get a large sample and a good hypothesis. Some social psych studies work off of correlations (eg correlation between being observer-rated attractiveness and observer-rated competence at a skill); correlations are “lamentably low” in social psychology because high level processes (like opinion formation, social interaction, etc.) have a lot of noise. Are there any PCT studies of these sorts of processes (not simple motor coordination problems) that have any higher correlation than standard models do? Any with even the same level of correlation?
What’s the difference between control theory and stimulus-response in a context? For example, if we use a simplified version of hunger in which the hormone leptin is produced in response to hunger and the hormone ghrelin is produced in response to satiety, we can explain this in two ways: the body is trying to PCT itself to the perfect balance of leptin and ghrelin, or in the context of the stimulus leptin the response of eating is rewarded and in the context of the stimulus ghrelin the response of eating is punished. Are these the same theory, or are there experiments that would distinguish between them? Do you know of any?
Does PCT still need reinforcement learning to explain why animals use some strategies and not others to achieve equilibrium? For example, when a rat in a Skinner box is hungry (ie its satiety variable has deviated in the direction of hunger), and then it presses a lever and gets a food pellet and its satiety variable goes back to its reference range, would PCTists explain that as getting rewarded for pressing the lever and expect it to press the lever again next time it’s hungry?
Rats don’t always choose drugs over everything else
Summary: An experimenter thought drug addiction in rats might be linked to being kept in distressing conditions, made a Rat Park to test the idea, and found that the rats in the enriched Rat Park environment ignored the morphine on offer.
EDIT: apparently the study had methodological issues and hasn’t been replicated, making the results somewhat suspect, as pointed out by Yvain below
I hate to admit I get science knowledge from Reddit, but the past few times this was posted there it was ripped apart by (people who claimed to be) professionals in the field—riddled with metholodogical errors, inconsistently replicated, et cetera. The fact that even its proponents admit the study was rejected by most journals doesn’t speak well of it.
I think it’s very plausible that situation contributes to addiction; we know that people in terrible situations have higher discount rates than others and so tend to short-term thinking that promotes that kind of behavior, and certainly they have fewer reasons to try to live life as a non-addict. But I think the idea that morphine is no longer interesting and you can’t become addicted when you live a stimulating life is wishful thinking.
Damn. Oh well, noted and edited in to the original comment.
Well, like I said, all I have to go on is stuff people said on Reddit and one failed replication study I was able to find somewhere by a grad student of the guy who did the original research. The original research is certainly interesting and relevant and does speak to the problems with a very reductionist model.
This actually gets to the same problem I’m having looking up stuff on perceptual control theory, which is that I expect a controversial theory to be something where there are lots of passionate arguments on both sides, but on both PCT and Rat Park, when I’ve tried to look them up I get a bunch of passionate people arguing that they’re great, and then a few scoffs from more mainstream people saying “That stuff? Nah.” without explaining themselves. I don’t know whether it’s because of Evil Set-In-Their-Ways Mainstream refusing to acknowledge the new ideas, or whether they’re just so completely missing the point that people think it’s not worth their while to respond. It’s a serious problem and I wish that “skeptics” would start addressing this kind of thing instead of debunking ghosts for the ten zillionth time.
Just a brief note to say that I do intend to get back to this, but I’ve been largely offline since the end of last week, and will be very busy at least until the end of this month on things other than LessWrong. I would like to say a lot more about PCT here than I have in the past (here, here, and in various comments), but these things take me long periods of concentrated effort to write.
BTW, one of the things I’m busy with is PCT itself, and I’ll be in Boulder, Colorado for a PCT-related meeting 28-31 July, and staying on there for a few days. Anyone around there then?
The PCT learning model doesn’t require reinforcement at the control level, as its model of memory is a mapping from reference levels to predicted levels of other variables. I.e., when the rat notices that the lever-pressing is paired with food, a link is made between two perceptual variables: the position of the lever, and the availability of food. This means that the rat can learn that food is available, even when it’s not hungry.
Where reinforcement is relevant to PCT is in the strength of the linkage and in the likelihood of its being recorded. If the rat is hungry, then the linkage is more salient, and more likely to be learned.
Notice though, that again the animal’s internal state is of primary importance, not the stimulus/response. In a sense, you could say that you can teach an animal that a stimulus and response are paired, but this isn’t the same as making the animal behave. If we starved you and made you press a lever for your food, you might do it, or you might tell us to fork off. Yet, we don’t claim that you haven’t learned that pressing the lever leads to food in that case.
(As Richard says, it’s well established that you can torture living creatures until they accede to your demands, but it won’t necessarily tell you much about how the creature normally works.)
In any case, PCT allows for the possibility of learning without “reinforcement” in the behaviorist sense, unless you torture the definition of reinforcement to the point that anything is reinforcement.
Regarding the leptin/ghrelin question, my understanding is that PCT as a psych-physical model primarily addresses those perceptual variables that are modeled by neural analog—i.e., an analog level maintained in a neural delay loop. While Powers makes many references to other sorts of negative feedback loops in various organisms from cats to E. coli, the main thrust of his initial book deals with building up a model of what’s going on, feedback-loopwise, in the nervous system and brain, not the body’s endocrine systems.
To put it another way, PCT doesn’t say that control systems are universal, only that they are ubiquitous, and that the bulk of organisms’ neural systems are assembled from a relatively small number of distinct component types that closely resemble the sort of components that humans use when building machinery.
IOW, we should not expect that PCT’s model of neural control systems would be directly applicable to a hormone level issue. However, we can reason from general principles and say that one difference between a PCT model of the leptin/ghrelin question is that PCT includes an explicit model of hierarchy and conflict in control networks, so that we can answer questions about what happens if both leptin and ghrelin are present (for example).
If those signals are at the same level of control hierarchy, we can expect conflict to result in oscillation, where the system alternates between trying to satisfy one or the other. Or, if they’re at different levels of hierarchy, then we can expect one to override the other.
But, unlike a behavioral model where the question of precedence between different stimuli and contexts is open to interpretation, PCT makes some testable predictions about what actually constitutes hierarchy, both in terms of expected behavior, and in terms of the physical structure of the underlying control circuitry.
That is, if you could dissect an organism and find the neurons, PCT predicts a certain type of wiring to exist, i.e., that a dominant controller will have wiring to set the reference levels for lower-level controllers, but not vice-versa.
Second, PCT predicts that a dominant perception must be measured at a longer time scale than a dominated one. That is, the lower-level perception must have a higher sampling rate than the higher-level perception. Thus, for example, as a rat becomes hungrier (a longer-term perceptual variable), its likelihood of pressing a lever to receive food in spite of a shock is increased.
AFAICT, behaviorism can “explain” results like these, but does not actually predict them, in the sense that PCT is spelling out implementation-level details that behaviorism leaves to hand-waving. IOW, PCT is considerably more falsifiable than behaviorism, at least in principle. Eventually, PCT’s remaining predictions (i.e., the ones that haven’t already panned out at the anatomical level) will either be proven or disproven, while behaviorism doesn’t really make anatomical predictions about these matters.
To answer question 3, one could perform the experiment of surgically balancing leptin and ghrelin and not feeding or otherwise nourishing the subject. If the subject eventually dies of starvation, I would say the second theory is more likely.