Some forms of therapy, especially ones that help you notice blindspots or significantly reframe your experience or relationship to yourself or the world (e.g. parts work where you first shift to perceiving yourself as being made of parts, and then to seeing those parts with love)
What is your take on the Dodo bird verdict, in relation to both therapy and Buddhism-adjacent things? All this stuff seems to be very heavy on personal anecdotes and just-so stories, and light on RCT-type things. Maybe there’s a there there, but it doesn’t seem like serious systematic study of this whole field has even begun, and there’s plenty of suspicious resistance to even the idea of that from certain quarters.
For whatever reason, it looks like when these kinds of delusions are removed, people gravitate towards being compassionate, loving, etc.
This is also a big if true type claim which from the outside doesn’t seem remotely clear, and to the extent that it is true causation may well be reversed.
The first thing to note is that that very page says that the state of evidence on the verdict is mixed, with different studies pointing in different directions, results depending on how you conduct your meta-analyses, and generally significant disagreement about whether this is really a thing.
I also think that this comment from @DaystarEld , our resident rationalist therapist, makes a lot of sense:
For one thing, the Dodo bird verdict is (maybe not surprisingly, given point 3) not as well supported as people widely think. It originated decades ago, and may have set in motion the very effects that led to its own eventual lack of relevance. The study I linked to in the OP, if correct, points to just such an invalidation by presenting findings that a particular modality works better for a certain type of treatment than alternatives.
But if we take it at face value, the answer could just come down to “the human element.” Maybe good therapists are what matter and the modality, as long as it’s not utterly bankrupt, is just a vehicle. Personally I don’t believe that’s the full story, but a good relationship with the therapist does seem more important than anything else, and that factor being mostly independent from what modality the therapist uses may account for a large part of it.
Ultimately though, I think part of what my post is tries to do is point out that these different philosophies don’t necessarily contradict each other, but rather are different lenses through which to view the problems the client has. When I get a client that responds super well to CBT, and then another client who doesn’t but grabs IFS and runs with it, I don’t think “well I guess these modalities are equally effective” or think that some kind of paradox is occurring, I just think that different maps are better for different people at navigating the territory, even if they’re dealing with the same “problem.”
I know it feels a bit like a cop-out, but honestly given how complex people are, and how different each problem can be even if it shares the same diagnosis, I would be pretty shocked if a single modality just blew all the others out of the water for every kind of problem that someone might face. Which isn’t to say that they’re all the same, either, just that guidelines for good therapy have to include more than just singling out specific modalities, but also identifying which ones might work best with each client.
This touches upon a related issue, which is that there are some serious challenges with trying to apply an RCT-type methodology on something like this. For an RCT, you’ll want to try to standardize things as much as possible, so that—for example—if you are measuring the effectiveness of Cognitive Behavioral Therapy, then every therapist you’ve classified as “doing CBT” actually does do CBT and nothing else. But a good therapist won’t just blindly apply one method, they’ll consider what they think might work best for this particular client and then use that.
Suppose you have a study where therapists A and B are both familiar with both CBT and Internal Family Systems. Therapist A is assigned to the CBT condition and therapist B is assigned to the IFS condition. As a result, A spends some time doing CBT on clients CBT is a poor match for, and B spends some time doing IFS on clients IFS is a poor match for. The study finds that CBT and IFS have roughly similar, moderate efficacy. What the study fails to pick up on is that if both A and B had been allowed to pick the method that works best on each client, doing IFS for some and CBT for some, then the effect of the method might have been significantly greater.
But you can’t really compare the efficiency of methods by doing an RCT where everyone is allowed to just do whatever method they like, or worse, some hybrid method that pulls in from many different therapy techniques. Or maybe you could do it and just ask the therapists to write down what method they used with each client afterward… but that would probably require some really complicated statistical method to try to analyze, exactly the kind of thing where everyone will just end up fighting over the right way to interpret the results afterward.
Another thing is that changes in subjective well-being are often just really hard to measure well. For one, various measures you’d naively expect to correlate with each other, don’t really:
The study asked people in 156 countries to “value their lives today on a 0 to 10 scale, with the worst possible life as a 0 and the best possible life as a 10.” [...] societal factors such as gross domestic product per capita; extensiveness of social services; freedom from oppression; and trust in government and fellow citizens can explain a significant proportion of people’s average life satisfaction [...] the Nordic countries—Finland, Sweden, Norway, Denmark, Iceland—tend to score highest in the world.
But when you look at how much positive emotion people experience, the top of the world looks very different. Suddenly, Latin American countries such as Paraguay, Guatemala and Costa Rica are the happiest countries on earth. [...]
Things get even more complicated when we look at the prevalence of depression in different countries. In one comparison made by the World Health Organization, the per capita prevalence of unipolar depressive disorders is highest in the world in the United States. Among Western countries, Finland is number two. Paradoxically then, the same country can be high on both life satisfaction and depression. [...]
Finally, some people might argue that neither life satisfaction, positive emotions nor absence of depression are enough for happiness. Instead, something more is required: One has to experience one’s life as meaningful. [...] [On a measure of meaningfulness] African countries including Togo and Senegal were at the top of the ranking, while the U.S. and Finland were far behind. Here, religiosity might play a role: The wealthier countries tend to be less religious on average, and this might be the reason why people in these countries report less meaningfulness.
What I’m trying to say is that, as regards happiness, it’s complicated. Different people define happiness very differently. And the same person or country can be high on one dimension of happiness while being low on another dimension of happiness. Maybe there is no such thing as happiness as such. Instead we should look at these dimensions separately and examine how well various nations are able to support each of them.
I think this finding actually makes a lot of sense. At one point in my life, if you had asked me to measure my well-being on a scale from 0 to 10, I might have said something like, “well I’m feeling pretty depressed and don’t find my work very satisfying and my romantic relationships aren’t working out, but then on the other hand I do have a secure job, it pays reasonably well, I live in a safe country, and overall I feel like there are a lot of things that are objectively just pretty good, so let’s say a 7”.
Setting aside the fact that someone might simultaneously report being depressed while also giving a relatively high life-satisfaction number, here are some ways in which I feel like I’ve benefited that feel like they’re at least somewhat linked to these practices:
It used to be that I would randomly remember various faux pases or mistakes I’d made that felt embarrassing or shameful, or generally experience painful feelings of regret for past life decisions. These could feel pretty intensely unpleasant. While something like this might still occasionally happen for a bit if I make a new mistake, or I might get some very mild twinge of unpleasantness related to some past thing, to a first approximation it’s correct to say that this has stopped happening. What’s in the past is in the past, and I don’t really experience much regret or “damn I should(n’t) have said X” anymore.
When I’m on my phone, it’s been getting more common recently that I maintain awareness of my peripheral vision and the rest of room, as well as still continuing to feel my body, rather than just getting completely sucked into the phone and forgetting about the world around me. This makes it somewhat easier to stop being on my phone and done something else.
I feel like pain and displeasure are less strongly linked to suffering than they used to be. It’s easier to experience something as painful and unpleasant while simultaneously being okay with having that experience. I was at men’s retreat recently where the facilitator asked if we’d want to try some amateur boxing (with the appropriate safety gear) after he’d drill us on some basic technique. I’ve had a pretty strong fear of physical pain as well as feeling generally bad at physical things, so this offer felt pretty frightening to me, but I figured that objectively thinking it’d probably be fine and that the fear is just a sensation, so I can be with that sensation. Then I kept being in a state of mild panic for the entirety of the drills and then in the actual match I was hyperventilating, but also that felt okay and a positive experience overall, rather than traumatizing and like the end of the world as it would have felt some years earlier.
Now if you asked me to report on my life satisfaction now… I do feel like those things represent some significant improvements to my life, but a scale of 0 to 10 is pretty coarse, and I still haven’t gotten romantic relationships working out. So maybe I’d still say 7? Possibly 8, but 7 doesn’t feel unreasonable either.
It’s also interesting that out of the various ways of measuring well-being (life satisfaction, positive emotion experienced, amount of depression, sense of meaning), “pain being linked with less suffering” wouldn’t necessarily show up on any of those. Someone might still have feel essentially the same emotionally, but just e.g. feel more okay with being depressed and sad often.
There’s also other pieces of weirdness about life satisfaction measures in general—for example, people might give a slightly negative score on the overall life satisfaction, while giving positive scores on all subareas of life satisfaction that are asked about.
The Wikipedia article on the verdict was saying that it’s controversial since there are debates about exactly what kinds of measures to include in the meta-analyses, with people on the “pro-verdict side” accusing people on the “anti-verdict side” of sometimes cherry-picking measures and results that do show a positive change. But it does seem to me possible to drastically improve a person’s well-being on one measure without it showing up on any others, in ways that you might not be able to predict before running the study. And if someone then appeals to that one measure having improved, they might be cherry-picking or they might just be legitimately drawing attention to the one thing that does manage to measure a real change.
There’s also the fact that when I offer emotion coaching for people, the results will vary and some people probably get basically no benefit, others get a moderate benefit, and then some others benefit massively. Here’s a testimonial from one client I had:
I attended a few IFS sessions with Kaj towards the end of last year.
I don’t say this lightly, but the sessions with Kaj had a transformative impact on my life. Before these sessions, I was grappling with significant work and personal-related challenges. It’s hard to capture how bad it had become—my life felt like it had ground to a halt. My ability to function was very limited and this had lasted for over two years. Despite trying various methods, and seeing various professionals, I hadn’t seen much improvement in this time.
However, after just a few sessions (<5) with Kaj, I overcame substantial internal barriers. This not only enabled me to be more productive again on the work I cared about but also to be kinder to myself. My subjective experience was not one of constant cycling in mental pain. I could finally apply many of the lessons I had previously learned from therapists but had been unable to implement.
I remember being surprised at how real the transformation felt. I can say now, almost a year later, that it was also not transient, but has lasted this whole time.
As a result, I successfully completed some major professional milestones. On the personal front, my life has also seen positive changes that bring me immense joy.
I owe this success to the support from Kaj and IFS. I had been sceptical of ‘discrete step’ changes after so many years of pain with little progress, but I can now say I am convinced it is possible to have significant and enduring large shifts in how you approach yourself, your life and your pursuits.
Now, this person is definitely a bit of an outlier, but only somewhat. I get a client who gets utterly transformed in less than five sessions maybe… 1-3 times a year? Whereas some smaller but still significant change in a session or two, like a client saying that the first session with me feels more useful than all of the therapy they previously had combined, feels basically just normal and expected at this point. And anecdotally this—going from very little previous experience to hearing “wow you’re better than all the ‘real’ therapists I saw previously”—seems like a common experience for people who start doing offering kind of coaching or therapy that effectively focuses on memory reconsolidation.
Maybe it’s just personal fit, and my personality happens to match super-well with some people and then they get enormous benefits, and the problem was that none of the ordinary therapists they tried were an equally good match? And this is also the case with everyone else I keep hearing stories from? I guess that could be the case, but...
I realize that this is still very anecdotal. But my perspective is something like: my experience is that either I have personally benefited massively from these things or I am utterly delusional to the point of not being able to trust anything about my own experience, many of my friends who have done something similar give me plausible accounts of how they have benefited massively from these things, many of my clients give me plausible accounts of how they have benefited massively from these things… and main point against is that some controversial studies have difficulty establishing that the methods are effective, at least depending on who you ask and how you analyze them, and this seems like the kind of thing that is just really hard to analyze with RCTs in general.
And also, things being hard to analyze with RCTs is a common thing! Famously, nobody did an RCT measuring the effectiveness of parachutes against a control group without them, before parachutes became broadly used. If someone tells me that their name is Mark, I tend to believe that this is their name, even though it’d be really hard for me to run an RCT establishing this. Of all the things that I know and act based on, only a very small portion of it is something that can be established with RCTs.
Now, of course, just the fact that I’m convinced doesn’t mean that you would have a reason to be convinced. It’s totally valid to go “Okay but I still think you’re just going by anecdotal evidence and possibly deluding yourself”. And maybe I am! But again, that doesn’t seem like the most likely hypothesis to me.
---
Talking about Buddhism and meditation more specifically, there are various findings showing results, e.g. this summary from the book Altered Traits:
… at the start of contemplative practice, little or nothing seems to change in us. After continued practice, we notice some changes in our way of being, but they come and go. Finally, as practice stabilizes, the changes are constant and enduring, with no fluctuation. They are altered traits.
Taken as a whole, the data on meditation track a rough vector of progressive transformations, from beginners through the long-term meditators and on to the yogis. This arc of improvement seems to reflect both lifetime hours of practice as well as time on retreat with expert guidance.
The studies of beginners typically look at the impacts from under 100 total hours of practice—and as few as 7. The long-term group, mainly vipassana meditators, had a mean of 9,000 lifetime hours (the range ran from 1,000 to 10,000 hours and more).
And the yogis studied in Richie’s lab, had all done at least one Tibetan-style three-year retreat, with lifetime hours up to Mingyur’s 62,000. Yogis, on average had three times more lifetime hours than did long-term meditators—9,000 hours versus 27,000.
A few long-term vipassana meditators had accumulated more than 20,000 lifetime hours and one or two up to 30,000, though none had done a three-year retreat, which became a de facto distinguishing feature of the yogi group. Despite the rare overlaps in lifetime hours, the vast majority of the three groups fall into these rough categories.
There are no hard-and-fast lifetime hour cutoffs for the three levels, but research on them has clustered in particular ranges. We’ve organized meditation’s benefits into three dose-response levels, roughly mapping on the novice to amateur to professional rankings found in expertise of all kinds, from ballerinas to chess champions. [...]
Sticking with meditation over the years offers more benefits as meditators reach the long-term range of lifetime hours, around 1,000 to 10,000 hours. This might mean a daily meditation session, and perhaps annual retreats with further instruction lasting a week or so—all sustained over many years. The earlier effects deepen, while others emerge.
For example, in this range we see the emergence of neural and hormonal indicators of lessened stress reactivity. In addition, functional connectivity in the brain in a circuit important for emotion regulation is strengthened, and cortisol, a key hormone secreted by the adrenal gland in response to stress, lessens.
Loving-kindness and compassion practice over the long term enhance neural resonance with another person’s suffering, along with concern and a greater likelihood of actually helping. Attention, too, strengthens in many aspects with long-term practice: selective attention sharpens, the attentional blink diminishes, sustained attention becomes easier, and an alert readiness to respond increases. And long-term practitioners show enhanced ability to down-regulate the mind-wandering and self-obsessed thoughts of the default mode, as well as weakening connectivity within those circuits—signifying less self-preoccupation. These improvements often show up during meditative states, and generally tend to become traits.
Shifts in very basic biological processes, such as a slower breath rate, occur only after several thousand hours of practice. Some of these impacts seem more strongly enhanced by intensive practice on retreat than by daily practice.
While evidence remains inconclusive, neuroplasticity from long-term practice seems to create both structural and functional brain changes, such as greater working connection between the amygdala and the regulatory circuits in the prefrontal areas. And the neural circuits of the nucleus accumbens associated with “wanting” or attachment appear to shrink in size with longer-term practice.
While in general we see a gradient of shifts with more lifetime meditation hours, we suspect there are different rates of change in disparate neural systems. For instance, the benefits of compassion come sooner than does stress mastery. We expect studies in the future will fill in the details of a dose-response dynamic for various brain circuits. Intriguing signs suggest that long-term meditators to some degree undergo state-by-trait effects that enhance the potency of their practice. Some elements of the meditative state, like gamma waves, may continue during sleep.
Now as I recall, the book does have a number of caveats about how many of the studies on meditation are low-quality, relatively small, et cetera. But again, there are again serious challenges to running an RCT—sure you can do a study where you give someone a limited meditation intervention, like asking them to do some meditation practice for a month. But if some of the benefits are going to start showing up around 1,000 hours of practice and some at 10,000, you can’t just find a random assortment of people who have never meditated before and tell them to do thousands of hours of it or to go on month- or year-long retreats.
Thanks for such a thorough response! I have enjoyed reading your stuff over the years, from all the spirituality-positive people I find your approach especially lucid and reasonable, up there with David Chapman’s.
I also agree with many of the object-level claims that you say spiritual practices helped you reach, like the multi-agent model of mind, cognitive fusion, etc. But, since I seem to be able to make sense of them without having to meditate myself, it has always left me bemused as to whether meditation really is the “royal road” to these kinds of insight, and if whatever extra it might offer is worth the effort. Like, for example, I already rate my life satisfaction at around 7, and this seems adequate given my objective circumstances.
So, I guess, my real question for the therapy and spirituality-positive people is why they think that their evidence for believing what they believe is stronger than that of other people in that field who have different models/practices/approaches but about the same amount of evidence for its effectiveness. Granted that RCTs aren’t always, or even often, easy, but it seems to me that the default response to lack of strong evidence of that sort, or particularly reliable models of reality like those that justify trusting parachutes even in the absence of RCTs, is to be less sure that you have grasped the real thing. I have no reason to doubt that plenty of therapists/coaches etc. have good evidence that something that they do works, but having a good, complete explanation of what exactly works or why is orders of magnitude harder, and I don’t think that anybody in the world could reasonably claim to have the complete picture, or anything close to it.
I have enjoyed reading your stuff over the years, from all the spirituality-positive people I find your approach especially lucid and reasonable, up there with David Chapman.
Thank you! That’s high praise. :)
But, since I seem to be able to make sense of them without having to meditate myself, it always left me bemused as to whether meditation really is the “royal road” to these kinds of insight, and if whatever extra it might offer is worth the effort.
Heh, I remember that at one point, a major point of criticism about people talking about meditation on LW was that they were saying something like “you can’t understand the benefits of meditation without actually meditating so I’m not going to try, it’s too ineffable”. Now that I’ve tried explained things, people wonder what the point of meditating might be if they can understand the explanation without meditating themselves. :) (I’m not annoyed or anything, just amused. And I realize that you’re not one of the people who was making this criticism before.)
Anyway, I’d say it’s one thing to understand an explanation of the general mechanism of how insights are gotten, and another to actually experience the insights from the inside in a way that shifts your unconscious predictions.
That being said, is it worth the effort for you? I don’t know, we kinda concluded in our dialogue that it might not be for everyone. And there are risks too. Maybe give some of it a try if you haven’t already, see if you feel motivated to continue doing it for the immediate benefits, and then just stick to reading about it out of curiosity if not?
So, I guess, my real question for the therapy and spirituality-positive people is why they think that their evidence for believing what they believe is stronger than that of other people in that field who have different models/practices/approaches but about the same amount of evidence for its effectiveness.
Good question. One thing that I’m particularly confused about is why me and Scott Alexander seem to have such differing views on the effectiveness of the “weird therapies”. My current position is just something like… “people seem to inhabit genuinely different worlds for reasons that are somewhat mysterious, so they will just have different experiences and priors leading to different beliefs, and often you just have to go with your own beliefs even if other smart people disagree because just substituting the beliefs of others for your own doesn’t seem like a good either”. And then hopefully if we continue discussing our reasons for our beliefs for long enough, at some point someone will figure out something.
My current epistemic position is also like… it would be interesting to understand the reason, and I don’t have a great model of where the disagreement comes from. And I’m happy to discuss it with people who might disagree. But it also doesn’t feel like a huge priority, given that I do feel convinced enough that these things work on a level that’s sufficient for me.
I have no reason to doubt that plenty of therapists/coaches etc. have good evidence that something that they do works, but believing that they have a good and complete explanation of what exactly works or why is orders of magnitude harder, and I don’t think that anybody in the world could reasonably claim to have the complete picture, or anything close to it.
and another to actually experience the insights from the inside in a way that shifts your unconscious predictions.
Right, so my experience around this is that I’m probably one of the lucky ones in that I’ve never really had those sorts of internal conflicts that make people claim that they suffer from akrasia, or excessive shame/guilt/regret. I’ve always been at peace with myself in this sense, and so reading people trying to explain their therapy/spirituality insights usually makes me go “Huh, so apparently this stuff doesn’t come naturally to most people, shame that they have to bend themselves backwards to get to where I have always been. Cool that they have developed all these neat theoretical constructions meanwhile though.”
Maybe give some of it a try if you haven’t already, see if you feel motivated to continue doing it for the immediate benefits, and then just stick to reading about it out of curiosity if not?
Trying to dismiss the content of my thoughts does seem to help me fall asleep faster (sometimes), so there’s that at least :)
I think western psychotherapies are predicated on incorrect models of human psychology. RCTs mostly can’t capture the effects of serious practice over a long period of time, but of the ones that have tried, the most robust effect is lowered neuroticism, afaik. This was also my experience. It corresponded to a big positive shift subjectively, as well as expressions of shock from friends and family about the change.
I think western psychotherapies are predicated on incorrect models of human psychology.
Yet they all seem to have positive effects of similar magnitude. This suggests that we don’t understand the mechanism through which they actually work, and it seems straightforward to expect that this extends to less orthodox practices.
RCTs mostly can’t capture the effects of serious practice over a long period of time
But my understanding is that benefits of (good) spiritual practices are supposed to be continuous, if not entirely linear. However much effort you invest correlates with the amount of benefits you get, until enlightenment and becoming as gods.
What is your take on the Dodo bird verdict, in relation to both therapy and Buddhism-adjacent things? All this stuff seems to be very heavy on personal anecdotes and just-so stories, and light on RCT-type things. Maybe there’s a there there, but it doesn’t seem like serious systematic study of this whole field has even begun, and there’s plenty of suspicious resistance to even the idea of that from certain quarters.
This is also a big if true type claim which from the outside doesn’t seem remotely clear, and to the extent that it is true causation may well be reversed.
The first thing to note is that that very page says that the state of evidence on the verdict is mixed, with different studies pointing in different directions, results depending on how you conduct your meta-analyses, and generally significant disagreement about whether this is really a thing.
I also think that this comment from @DaystarEld , our resident rationalist therapist, makes a lot of sense:
This touches upon a related issue, which is that there are some serious challenges with trying to apply an RCT-type methodology on something like this. For an RCT, you’ll want to try to standardize things as much as possible, so that—for example—if you are measuring the effectiveness of Cognitive Behavioral Therapy, then every therapist you’ve classified as “doing CBT” actually does do CBT and nothing else. But a good therapist won’t just blindly apply one method, they’ll consider what they think might work best for this particular client and then use that.
Suppose you have a study where therapists A and B are both familiar with both CBT and Internal Family Systems. Therapist A is assigned to the CBT condition and therapist B is assigned to the IFS condition. As a result, A spends some time doing CBT on clients CBT is a poor match for, and B spends some time doing IFS on clients IFS is a poor match for. The study finds that CBT and IFS have roughly similar, moderate efficacy. What the study fails to pick up on is that if both A and B had been allowed to pick the method that works best on each client, doing IFS for some and CBT for some, then the effect of the method might have been significantly greater.
But you can’t really compare the efficiency of methods by doing an RCT where everyone is allowed to just do whatever method they like, or worse, some hybrid method that pulls in from many different therapy techniques. Or maybe you could do it and just ask the therapists to write down what method they used with each client afterward… but that would probably require some really complicated statistical method to try to analyze, exactly the kind of thing where everyone will just end up fighting over the right way to interpret the results afterward.
Another thing is that changes in subjective well-being are often just really hard to measure well. For one, various measures you’d naively expect to correlate with each other, don’t really:
I think this finding actually makes a lot of sense. At one point in my life, if you had asked me to measure my well-being on a scale from 0 to 10, I might have said something like, “well I’m feeling pretty depressed and don’t find my work very satisfying and my romantic relationships aren’t working out, but then on the other hand I do have a secure job, it pays reasonably well, I live in a safe country, and overall I feel like there are a lot of things that are objectively just pretty good, so let’s say a 7”.
Setting aside the fact that someone might simultaneously report being depressed while also giving a relatively high life-satisfaction number, here are some ways in which I feel like I’ve benefited that feel like they’re at least somewhat linked to these practices:
It used to be that I would randomly remember various faux pases or mistakes I’d made that felt embarrassing or shameful, or generally experience painful feelings of regret for past life decisions. These could feel pretty intensely unpleasant. While something like this might still occasionally happen for a bit if I make a new mistake, or I might get some very mild twinge of unpleasantness related to some past thing, to a first approximation it’s correct to say that this has stopped happening. What’s in the past is in the past, and I don’t really experience much regret or “damn I should(n’t) have said X” anymore.
When I’m on my phone, it’s been getting more common recently that I maintain awareness of my peripheral vision and the rest of room, as well as still continuing to feel my body, rather than just getting completely sucked into the phone and forgetting about the world around me. This makes it somewhat easier to stop being on my phone and done something else.
I feel like pain and displeasure are less strongly linked to suffering than they used to be. It’s easier to experience something as painful and unpleasant while simultaneously being okay with having that experience. I was at men’s retreat recently where the facilitator asked if we’d want to try some amateur boxing (with the appropriate safety gear) after he’d drill us on some basic technique. I’ve had a pretty strong fear of physical pain as well as feeling generally bad at physical things, so this offer felt pretty frightening to me, but I figured that objectively thinking it’d probably be fine and that the fear is just a sensation, so I can be with that sensation. Then I kept being in a state of mild panic for the entirety of the drills and then in the actual match I was hyperventilating, but also that felt okay and a positive experience overall, rather than traumatizing and like the end of the world as it would have felt some years earlier.
Now if you asked me to report on my life satisfaction now… I do feel like those things represent some significant improvements to my life, but a scale of 0 to 10 is pretty coarse, and I still haven’t gotten romantic relationships working out. So maybe I’d still say 7? Possibly 8, but 7 doesn’t feel unreasonable either.
It’s also interesting that out of the various ways of measuring well-being (life satisfaction, positive emotion experienced, amount of depression, sense of meaning), “pain being linked with less suffering” wouldn’t necessarily show up on any of those. Someone might still have feel essentially the same emotionally, but just e.g. feel more okay with being depressed and sad often.
There’s also other pieces of weirdness about life satisfaction measures in general—for example, people might give a slightly negative score on the overall life satisfaction, while giving positive scores on all subareas of life satisfaction that are asked about.
The Wikipedia article on the verdict was saying that it’s controversial since there are debates about exactly what kinds of measures to include in the meta-analyses, with people on the “pro-verdict side” accusing people on the “anti-verdict side” of sometimes cherry-picking measures and results that do show a positive change. But it does seem to me possible to drastically improve a person’s well-being on one measure without it showing up on any others, in ways that you might not be able to predict before running the study. And if someone then appeals to that one measure having improved, they might be cherry-picking or they might just be legitimately drawing attention to the one thing that does manage to measure a real change.
There’s also the fact that when I offer emotion coaching for people, the results will vary and some people probably get basically no benefit, others get a moderate benefit, and then some others benefit massively. Here’s a testimonial from one client I had:
Now, this person is definitely a bit of an outlier, but only somewhat. I get a client who gets utterly transformed in less than five sessions maybe… 1-3 times a year? Whereas some smaller but still significant change in a session or two, like a client saying that the first session with me feels more useful than all of the therapy they previously had combined, feels basically just normal and expected at this point. And anecdotally this—going from very little previous experience to hearing “wow you’re better than all the ‘real’ therapists I saw previously”—seems like a common experience for people who start doing offering kind of coaching or therapy that effectively focuses on memory reconsolidation.
Maybe it’s just personal fit, and my personality happens to match super-well with some people and then they get enormous benefits, and the problem was that none of the ordinary therapists they tried were an equally good match? And this is also the case with everyone else I keep hearing stories from? I guess that could be the case, but...
I realize that this is still very anecdotal. But my perspective is something like: my experience is that either I have personally benefited massively from these things or I am utterly delusional to the point of not being able to trust anything about my own experience, many of my friends who have done something similar give me plausible accounts of how they have benefited massively from these things, many of my clients give me plausible accounts of how they have benefited massively from these things… and main point against is that some controversial studies have difficulty establishing that the methods are effective, at least depending on who you ask and how you analyze them, and this seems like the kind of thing that is just really hard to analyze with RCTs in general.
And also, things being hard to analyze with RCTs is a common thing! Famously, nobody did an RCT measuring the effectiveness of parachutes against a control group without them, before parachutes became broadly used. If someone tells me that their name is Mark, I tend to believe that this is their name, even though it’d be really hard for me to run an RCT establishing this. Of all the things that I know and act based on, only a very small portion of it is something that can be established with RCTs.
Now, of course, just the fact that I’m convinced doesn’t mean that you would have a reason to be convinced. It’s totally valid to go “Okay but I still think you’re just going by anecdotal evidence and possibly deluding yourself”. And maybe I am! But again, that doesn’t seem like the most likely hypothesis to me.
---
Talking about Buddhism and meditation more specifically, there are various findings showing results, e.g. this summary from the book Altered Traits:
Now as I recall, the book does have a number of caveats about how many of the studies on meditation are low-quality, relatively small, et cetera. But again, there are again serious challenges to running an RCT—sure you can do a study where you give someone a limited meditation intervention, like asking them to do some meditation practice for a month. But if some of the benefits are going to start showing up around 1,000 hours of practice and some at 10,000, you can’t just find a random assortment of people who have never meditated before and tell them to do thousands of hours of it or to go on month- or year-long retreats.
Thanks for such a thorough response! I have enjoyed reading your stuff over the years, from all the spirituality-positive people I find your approach especially lucid and reasonable, up there with David Chapman’s.
I also agree with many of the object-level claims that you say spiritual practices helped you reach, like the multi-agent model of mind, cognitive fusion, etc. But, since I seem to be able to make sense of them without having to meditate myself, it has always left me bemused as to whether meditation really is the “royal road” to these kinds of insight, and if whatever extra it might offer is worth the effort. Like, for example, I already rate my life satisfaction at around 7, and this seems adequate given my objective circumstances.
So, I guess, my real question for the therapy and spirituality-positive people is why they think that their evidence for believing what they believe is stronger than that of other people in that field who have different models/practices/approaches but about the same amount of evidence for its effectiveness. Granted that RCTs aren’t always, or even often, easy, but it seems to me that the default response to lack of strong evidence of that sort, or particularly reliable models of reality like those that justify trusting parachutes even in the absence of RCTs, is to be less sure that you have grasped the real thing. I have no reason to doubt that plenty of therapists/coaches etc. have good evidence that something that they do works, but having a good, complete explanation of what exactly works or why is orders of magnitude harder, and I don’t think that anybody in the world could reasonably claim to have the complete picture, or anything close to it.
I think cognitive understanding is overrated and physical changes to the CNS are underrated, as explanations for positive change from practices.
Thank you! That’s high praise. :)
Heh, I remember that at one point, a major point of criticism about people talking about meditation on LW was that they were saying something like “you can’t understand the benefits of meditation without actually meditating so I’m not going to try, it’s too ineffable”. Now that I’ve tried explained things, people wonder what the point of meditating might be if they can understand the explanation without meditating themselves. :) (I’m not annoyed or anything, just amused. And I realize that you’re not one of the people who was making this criticism before.)
Anyway, I’d say it’s one thing to understand an explanation of the general mechanism of how insights are gotten, and another to actually experience the insights from the inside in a way that shifts your unconscious predictions.
That being said, is it worth the effort for you? I don’t know, we kinda concluded in our dialogue that it might not be for everyone. And there are risks too. Maybe give some of it a try if you haven’t already, see if you feel motivated to continue doing it for the immediate benefits, and then just stick to reading about it out of curiosity if not?
Good question. One thing that I’m particularly confused about is why me and Scott Alexander seem to have such differing views on the effectiveness of the “weird therapies”. My current position is just something like… “people seem to inhabit genuinely different worlds for reasons that are somewhat mysterious, so they will just have different experiences and priors leading to different beliefs, and often you just have to go with your own beliefs even if other smart people disagree because just substituting the beliefs of others for your own doesn’t seem like a good either”. And then hopefully if we continue discussing our reasons for our beliefs for long enough, at some point someone will figure out something.
My current epistemic position is also like… it would be interesting to understand the reason, and I don’t have a great model of where the disagreement comes from. And I’m happy to discuss it with people who might disagree. But it also doesn’t feel like a huge priority, given that I do feel convinced enough that these things work on a level that’s sufficient for me.
Yes I definitely agree with this.
Right, so my experience around this is that I’m probably one of the lucky ones in that I’ve never really had those sorts of internal conflicts that make people claim that they suffer from akrasia, or excessive shame/guilt/regret. I’ve always been at peace with myself in this sense, and so reading people trying to explain their therapy/spirituality insights usually makes me go “Huh, so apparently this stuff doesn’t come naturally to most people, shame that they have to bend themselves backwards to get to where I have always been. Cool that they have developed all these neat theoretical constructions meanwhile though.”
Trying to dismiss the content of my thoughts does seem to help me fall asleep faster (sometimes), so there’s that at least :)
I think western psychotherapies are predicated on incorrect models of human psychology. RCTs mostly can’t capture the effects of serious practice over a long period of time, but of the ones that have tried, the most robust effect is lowered neuroticism, afaik. This was also my experience. It corresponded to a big positive shift subjectively, as well as expressions of shock from friends and family about the change.
Yet they all seem to have positive effects of similar magnitude. This suggests that we don’t understand the mechanism through which they actually work, and it seems straightforward to expect that this extends to less orthodox practices.
But my understanding is that benefits of (good) spiritual practices are supposed to be continuous, if not entirely linear. However much effort you invest correlates with the amount of benefits you get, until enlightenment
and becoming as gods.Was not linear for me afaict