How do we learn from experience without a control group?
I can discern cause and effect when there’s a clear mechanism and immediate feedback. Brushing my teeth makes them feel clean. Eating pizza makes me full. Yet I’ve met plenty of people who claim to have had direct, powerful experiences confirming that forms of pseudoscience are real—astrology, crystal healing, reiki, etc.
While I don’t believe in astrology or crystal healing based on their reports, I think that in at least some cases, they’ve experienced something similar to what it’s like to eat and feel full, except that they received reiki and felt pain disappear.
Selective reporting might explain some of these effects. But I think that for some believers, they’ve experienced, at least once, an immediate and powerful cause/effect sensation connecting something like reiki to physical healing. Even if it only happened once, they believe in the cause/effect relationship, like a person who eats a strange mushroom in the woods and starts hallucinating might later feel confident that the mushroom caused the hallucinations.
We have a prior social belief that eating strange mushrooms is a plausible causal mechanism for having hallucinations. That might be why we believe in the cause/effect relationship in that case, but not in the case of reiki.
But what do we do if we believe in, or at least want to investigate, a cause/effect relationship that’s not as socially widespread? I worked as a telemarketer for a year in my first year out of college, and after having the same 5-minute phone call with people 40 hours a week for a year, I do believe I found subtle and hard-to-explain ways of influencing the outcome of that call so that we got more conversions and less tension with the client. How does that happen, especially given that I myself had no clear hypothesis going in about how these conversational subtleties would affect the call, or how exactly to translate these psychological intuitions into my own physical behaviors?
I had similar experiences as a piano teacher, having psychological intuitions and making hypotheses about how to connect with a student and trying them out over the years. Some of these survived by getting results similar to what I expected across multiple students, or at least consistently with one student. Others I tried, and they either failed right away, or weren’t reliable.
Nowadays, I spend a lot of time reflecting and tinkering with my own learning process as I take classes in grad school. The effects are equally psychological and subtle. I keep or discard them based on how repeatable they are for me. Although I could make more sure I’m not tricking myself by figuring out how to run an RCT on myself, I believe that in the past, I’ve gained more knowledge faster using uncontrolled experiments and seeing if the results I got matched my expectations and worked out reliably. I come up with so many bad ideas that I think I’d waste too much time doing RCTs to gather legible evidence for the good ones, if I could even come up with the good ones at all without rapidly sifting through a ton of bad ideas along the way.
Nevertheless, I’d love to have some sort of account for how all this “tinkering” can possibly amount to knowledge. I am pretty confident that it does, based on my own past experience with similar tinkering over long periods of time in other domains.
The problem is that that knowledge just isn’t comprehensive enough to explain to others, or even to use mechanically on myself. In telemarketing, teaching, and self-study, I’ve found that any time I think I’ve hit on a “technique,” it’s rarely something I can implement in a blind, context-free manner and get the same results. Either the technique loses its novelty and turns out to have depended on that novelty to function, or it creates a new context in which it no longer functions, or a crucial difference enters between the technique on a conceptual level and how it’s implemented.
Instead, it seems like when they work best, these subtle techniques get preserved in a semi-conscious way, and my subconscious sort of “recommends” them to my conscious mind at opportune moments. System 2 helps bring these techniques to conscious attention for fine-tuning, but then system 1 needs to run the show to decide how to use them in the moment.
This all feels annoyingly unscientific, unrationalist, and illegible. Yet it also describes what gaining self-understanding and personal capability in the real world actually has felt like throughout my life. This even translates to physical skills, like pipetting in the lab, moving gracefully through the house without dropping things or hitting my head on stuff, or building a fence this last summer. There’s a combination or back and forth between paying close conscious attention and forming intellectual models of how to do things in a system-2-flavored way, and then relinquishing control back to system 1. Even deciding when to transfer control from system 1 to system 2 and back again is subject to this dynamic.
When I read scientific literature on psychology to inform these practices, it feels like it’s mainly just grist for the mill of this practice. Ultimately the things I’m trying to do are so multifaceted and contextual that it’s almost always impossible to just rote apply even the strongest findings from the psych literature. It’s just a way of supporting this back and forth between forming models of thought and behavior consciously and trying them out almost unconsciously as I work. I’m pessimistic about the ability to transmit this intuitive knowledge to others absent a very substantial effort to put it in context and committed students who really want to learn and believe there is something in fact to learn.
At the same time, overall, this doesn’t feel like an implausible mechanism for how the conscious and unconcious parts of our mind are “designed” to relate to each other. One part of the mind observes our behavior, environment, and inner state of thoughts and feelings. It looks for patterns and associations, and tries to articulate a mechanistic relationship. It can do this with deeper insight in some ways, because it has direct access to our inner state and doesn’t need to be legible to others—only to ourselves. Eventually, it crystallizes a pattern and hands it off to another part of the mind.
This other part of the mind receives crystallized patterns of trigger sensation, behavior, and expected result, and runs in the background making “guesses” about when it has spotted a trigger. When it does, it tries to reproduce the behavior and sees what kind of result it gets.
The conscious mind then evaluates the result and tries to articulate cause and effect. It may add nuance, reinforcement, or confidence, or it may discard the hypothesis entirely. The accumulation of patterns that survive this selection process constitute our intuitive skills, though probably not our whole intuition.
This seems to track roughly with how my own processing works. Yet I’d also say that it doesn’t just happen automatically, the way you get hungry automatically. I find that I have to deliberately construct at least a toy model, however wrong, and deliberately engage this back-and-forth for a specific challenge before it really kicks in. It’s also not automatically functional once it does start. I find there’s a lot of variation, sort of like the stock market, with “crashes,” ups and downs, and usually an overall upward trend in average performance over time. It’s not just that I sometimes am playing with patterns that are actually dysfunctional. It’s that the meta-process by which I conceive, implement and evaluate them is not always very good.
While this seems to be helpful and necessary for my growth as a person, it also seems to me that this weird, illegible process could also easily generate a lot of BS. If I was trying to be an astrologer, I expect this is the process that would eventually result in a large, loyal clientele. Instead, I’m trying to be an engineer, so I’m lucky that at least at this stage in my career, I get objective feedback on my performance in studying the material.
But it’s uncomfortable when my “meta-process” for self-improvement looks just as irrational as anybody else’s, and is similar if not identical to the process I was using prior to engaging with the rationalist community. As good as this community is for having practical guidance on how to evaluate legible claims on publicly-relevant topics, my practical understanding and control of the intuitive thoughts and behavior that govern my life moment-to-moment is pretty similar to what it was before I got involved in LessWrong and the rationalist movement. That’s a lot of life to leave nearly untouched. I even worry that we Goodhart/streetlamp effect ourselves into avoiding that topic or downplaying its reality or importance in favor of topics and activities where it’s easier to look like a rationalist.
How do we learn from experience without a control group?
I can discern cause and effect when there’s a clear mechanism and immediate feedback. Brushing my teeth makes them feel clean. Eating pizza makes me full. Yet I’ve met plenty of people who claim to have had direct, powerful experiences confirming that forms of pseudoscience are real—astrology, crystal healing, reiki, etc.
While I don’t believe in astrology or crystal healing based on their reports, I think that in at least some cases, they’ve experienced something similar to what it’s like to eat and feel full, except that they received reiki and felt pain disappear.
Selective reporting might explain some of these effects. But I think that for some believers, they’ve experienced, at least once, an immediate and powerful cause/effect sensation connecting something like reiki to physical healing. Even if it only happened once, they believe in the cause/effect relationship, like a person who eats a strange mushroom in the woods and starts hallucinating might later feel confident that the mushroom caused the hallucinations.
We have a prior social belief that eating strange mushrooms is a plausible causal mechanism for having hallucinations. That might be why we believe in the cause/effect relationship in that case, but not in the case of reiki.
But what do we do if we believe in, or at least want to investigate, a cause/effect relationship that’s not as socially widespread? I worked as a telemarketer for a year in my first year out of college, and after having the same 5-minute phone call with people 40 hours a week for a year, I do believe I found subtle and hard-to-explain ways of influencing the outcome of that call so that we got more conversions and less tension with the client. How does that happen, especially given that I myself had no clear hypothesis going in about how these conversational subtleties would affect the call, or how exactly to translate these psychological intuitions into my own physical behaviors?
I had similar experiences as a piano teacher, having psychological intuitions and making hypotheses about how to connect with a student and trying them out over the years. Some of these survived by getting results similar to what I expected across multiple students, or at least consistently with one student. Others I tried, and they either failed right away, or weren’t reliable.
Nowadays, I spend a lot of time reflecting and tinkering with my own learning process as I take classes in grad school. The effects are equally psychological and subtle. I keep or discard them based on how repeatable they are for me. Although I could make more sure I’m not tricking myself by figuring out how to run an RCT on myself, I believe that in the past, I’ve gained more knowledge faster using uncontrolled experiments and seeing if the results I got matched my expectations and worked out reliably. I come up with so many bad ideas that I think I’d waste too much time doing RCTs to gather legible evidence for the good ones, if I could even come up with the good ones at all without rapidly sifting through a ton of bad ideas along the way.
Nevertheless, I’d love to have some sort of account for how all this “tinkering” can possibly amount to knowledge. I am pretty confident that it does, based on my own past experience with similar tinkering over long periods of time in other domains.
The problem is that that knowledge just isn’t comprehensive enough to explain to others, or even to use mechanically on myself. In telemarketing, teaching, and self-study, I’ve found that any time I think I’ve hit on a “technique,” it’s rarely something I can implement in a blind, context-free manner and get the same results. Either the technique loses its novelty and turns out to have depended on that novelty to function, or it creates a new context in which it no longer functions, or a crucial difference enters between the technique on a conceptual level and how it’s implemented.
Instead, it seems like when they work best, these subtle techniques get preserved in a semi-conscious way, and my subconscious sort of “recommends” them to my conscious mind at opportune moments. System 2 helps bring these techniques to conscious attention for fine-tuning, but then system 1 needs to run the show to decide how to use them in the moment.
This all feels annoyingly unscientific, unrationalist, and illegible. Yet it also describes what gaining self-understanding and personal capability in the real world actually has felt like throughout my life. This even translates to physical skills, like pipetting in the lab, moving gracefully through the house without dropping things or hitting my head on stuff, or building a fence this last summer. There’s a combination or back and forth between paying close conscious attention and forming intellectual models of how to do things in a system-2-flavored way, and then relinquishing control back to system 1. Even deciding when to transfer control from system 1 to system 2 and back again is subject to this dynamic.
When I read scientific literature on psychology to inform these practices, it feels like it’s mainly just grist for the mill of this practice. Ultimately the things I’m trying to do are so multifaceted and contextual that it’s almost always impossible to just rote apply even the strongest findings from the psych literature. It’s just a way of supporting this back and forth between forming models of thought and behavior consciously and trying them out almost unconsciously as I work. I’m pessimistic about the ability to transmit this intuitive knowledge to others absent a very substantial effort to put it in context and committed students who really want to learn and believe there is something in fact to learn.
At the same time, overall, this doesn’t feel like an implausible mechanism for how the conscious and unconcious parts of our mind are “designed” to relate to each other. One part of the mind observes our behavior, environment, and inner state of thoughts and feelings. It looks for patterns and associations, and tries to articulate a mechanistic relationship. It can do this with deeper insight in some ways, because it has direct access to our inner state and doesn’t need to be legible to others—only to ourselves. Eventually, it crystallizes a pattern and hands it off to another part of the mind.
This other part of the mind receives crystallized patterns of trigger sensation, behavior, and expected result, and runs in the background making “guesses” about when it has spotted a trigger. When it does, it tries to reproduce the behavior and sees what kind of result it gets.
The conscious mind then evaluates the result and tries to articulate cause and effect. It may add nuance, reinforcement, or confidence, or it may discard the hypothesis entirely. The accumulation of patterns that survive this selection process constitute our intuitive skills, though probably not our whole intuition.
This seems to track roughly with how my own processing works. Yet I’d also say that it doesn’t just happen automatically, the way you get hungry automatically. I find that I have to deliberately construct at least a toy model, however wrong, and deliberately engage this back-and-forth for a specific challenge before it really kicks in. It’s also not automatically functional once it does start. I find there’s a lot of variation, sort of like the stock market, with “crashes,” ups and downs, and usually an overall upward trend in average performance over time. It’s not just that I sometimes am playing with patterns that are actually dysfunctional. It’s that the meta-process by which I conceive, implement and evaluate them is not always very good.
While this seems to be helpful and necessary for my growth as a person, it also seems to me that this weird, illegible process could also easily generate a lot of BS. If I was trying to be an astrologer, I expect this is the process that would eventually result in a large, loyal clientele. Instead, I’m trying to be an engineer, so I’m lucky that at least at this stage in my career, I get objective feedback on my performance in studying the material.
But it’s uncomfortable when my “meta-process” for self-improvement looks just as irrational as anybody else’s, and is similar if not identical to the process I was using prior to engaging with the rationalist community. As good as this community is for having practical guidance on how to evaluate legible claims on publicly-relevant topics, my practical understanding and control of the intuitive thoughts and behavior that govern my life moment-to-moment is pretty similar to what it was before I got involved in LessWrong and the rationalist movement. That’s a lot of life to leave nearly untouched. I even worry that we Goodhart/streetlamp effect ourselves into avoiding that topic or downplaying its reality or importance in favor of topics and activities where it’s easier to look like a rationalist.