An Invitation to Measure Meditation
Update: Thank you all for the suggestions, and all of the links to resources!
Here is my plan, which I hope will balance measuring useful things with being easy enough that I’ll actually stick with it:
Every Monday, I will complete a meditation test on Quantified Mind. On alternating Mondays, I will alternate whether I do the test immediately after meditating or not. These don’t look like a gold standard for attention tests, but they’re better than what I would code by hand and they’re more of a standard.
Every Monday, I will play a game(s?) of Dual 2-back and record my score. (I’ll switch to a higher N if 2-back ever gets easy.)
I will use TagTime to randomly ping me for an experience sample as a set of free-form tags, about twice per day, every day.
I’ll report back in a couple months with some plots and commentary.
When I wrote this post, my mindset was embarrassingly close to “I’m a Bayesian scientist meditating! It’s going to be like Harry and Hermione in HPMOR!”. Except of course, if there really were magic, a ton of people would already have been studying it, and the best way to start would be by asking about it. Thanks for all the advice.
An Invitation to Measure Meditation
I’ve begun to meditate regularly, so it seems like a good opportunity to experiment and search for measurable effects of meditation. This is an open invitation to propose easy measurements for me to intermittently perform. I don’t have any brilliant ideas for what to measure (beyond looking at Wikipedia’s list of research on meditation), but it seems silly to waste the opportunity.
How am I meditating?
For the past ~20 days, I’ve been meditating while walking for an hour or two, though I intend to switch to primarily breath-following meditation. At least five days a week, one hour a day. I expect to keep this up for a couple months, but it’s a big commitment so no promises.
I am following a ~500 page textbook on meditation called The Mind
Illuminated, by John Yates (a.k.a., Culadasa) and others. It is, to
quote the cover, “A complete meditation guide”, which “integrat[es]
Buddhist wisdom and brain science for greater mindfulness”. Most of it
is organized into instructions and techniques for how to meditate in
10 stages (corresponding roughly to the 9 stages described by
Asanga, which I can’t find a
reference for), but there are also five “interludes” that describe
five increasingly complete models of the mind.
The book is full of gears. If they’re accurate, this is fantastic, but if they’re not I’m likely to find out the hard way. Fortunately, there are some good signs: it has already given me a couple pieces of useful advice, and its models seem to be consistent with Bernard Barrs’ global workspace theory, the leading scientific theory of consciousness. Though take that with a grain of salt, as it’s a pretty new field. (While I haven’t done a detailed comparison, I have read Bernard Baars’ book In the Theater of Consciousness, and the models at least generally agree.)
What will I Measure?
Anyways, to quote the book, the two main objectives of this kind of meditation are:
Developing stable attention.
Cultivating powerful mindfulness that optimizes the interaction between attention and awareness.
For context, “attention” is when you focus on one thing in particular. It tends to isolate that thing, be analytical, and be more “self” centered. On the other hand, “peripheral awareness”, or “awareness” for short, tends to take in a whole sensory field at once (e.g. everything you’re seeing), is more contextual and involves less analysis, and is less personal and more objective.
So I would like to find tasks that will plausibly measure something related to attention or awareness. I am willing to spend up to one hour each week doing these tasks. Some possible tasks are:
Something subliminal?
A stroop test? I haven’t found a stroop game online that’s not easily gamable.
You’ll notice that there’s no control, so how can the results be interpreted? Well, if I improve a lot on some task, then hopefully that will interest someone else to be a control by doing the task over the same time-frame without meditating.
Questions for You:
Is this worthwhile, or am I unlikely to find anything significant with a sample size of 1?
If it is worthwhile, what should I measure?
How interested would people be to see a post on the mind model given in The Mind Illuminated, and compared/contrasted with global workspace theory?
Maybe set up something on your phone that pings you a few times each day at random times to track your mood across the day. Whenever you get a ping, write down the time, and then for example what you were doing, your subjective mood, subjective energy level and how spaced out or focused you’re feeling.
FYI, this is called the experience sampling method, and is used extensively in the social sciences. (The Wikipedia page I linked also lists a number of software platforms, including smartphone apps, that let you set up and conduct ESM data collection.)
That’s an excellent idea!
Does anyone happen to know off the top of their head an easy way to set this up on Android? All I need is random pings in a specified time range, and optionally a text box that records my answers.
I found an app called
trackthat looked perfect for this, but it crashes on my phone.Tagtime
Installed, thanks!
Heart Rate Varience Biofeedback is a good meditation target if you want to meditate in a quantifieable way.
If you want to go a less direct way http://www.quantified-mind.com/experiment/meditation is an existing program to measure effects of meditation by Quantified Self people.
I’ll use this. Thanks for the pointer.
Mostly I think measuring things is besides the point of meditating, but I’m not going to stop you, only warn you that it’s creating a feedback loop that encourages you to identify with meditation attainment that may eventually leave you stuck in a local maximum.
Wholely agreed: the goals of meditation are internal. The kind of external measurements I can perform will at best capture the shadow of internal changes, and the measurements themselves are going to distract me from meditation. (Literally distract me, as in be a common cause of mind wandering.)
However, this experiment is part of a ploy for myself: I desire the respect of this community; plus I want to keep promises that I made to myself → I need to follow up on the measurements I said I would do → I need to meditate regularly.
I expect the net effect to be positive, but how about this: if I find myself not meditating, or stuck at a particular point, and I think the measurements are largely to blame, then I promise to stop measuring.
Meditation acutely improves psychomotor vigilance, and may decrease sleep need:
https://behavioralandbrainfunctions.biomedcentral.com/articles/10.1186/1744-9081-6-47
The data (currently unpublished) I spoke of regarding the 45min mark is part of a larger ongoing effort to characterize changes that occur to the “mind” as people shift to a different way of experiencing their reality.
http://nonsymbolic.org/publications/
In my own experience as well the 40-45 min phase change is often blurred to the point of being unnoticeable. I’m still not 100% convinced it isn’t a priming effect though I think it has decent theoretical support. My simplified explanation is the brain cannot switch modes too rapidly. For example it’s easy to get HR up to 180 bpm in the blink of an eye if you think a tiger is leaping at you but takes considerably longer for it to go back to normal when you realize it was just the shadow of a harmless bush).
Regarding the tally counter: what you consider mind-wandering can/should change over time. Initially only click for completely losing attention and as improvement occurs you can include more and more subtle attentional deviations. Like any other tool it has a limited range where its use is appropriate. I’m not sure it helps too much with learning meditation; its more of a very simple way to vaguely-objectively track your progress over time.
A key principle to keep in mind is that not every meditation style is right for everyone all the time. TMI is heavily focused on anapanasati but there are a multitude of other styles. You may find your mind is more “in tune” with body scanning (e.g. Goenka) or something a little more wacky like Headless Way:
http://www.headless.org/experiments.htm
The research I linked to above found that people progressed much more rapidly if they were practicing a technique that was right for them. This is not an excuse to flail around randomly. At least one week of bona fide dedicated practice should be attempted before trying something else but months/years of painstaking effort with little gain is almost always a needless waste of time. That being said there is a fine line between the right amount of dogged persevering and dogmatic perseveration.
[deleted]
Big 5. Use the same test each time. Ideally you’d also have an external assessor complete a questionnaire about you (someone who interacts with you closely) and compare the 4 results.
This seems reasonable. I’d expect an increase in conscientiousness and openness and a decrease in neuroticism with no change to agreeableness or extroversion.
Rationalist types are often already maxed on openess. I experienced an uptick in extraversion in addition to the effects you mentioned.
Big 5 is a personality test. Personality is not supposed to change over a short period of time. All results should be measurement errors and random fluctuations from testing conditions. It would be interesting to see personality changes over a long period of time though.
Very large changes that are concordant with others’ reported changes in your behavior are likely not measurement error.
But you’re not supposed to see large changes in a personality test, Big 5′s dimensions are theorically stable over time. Very large changes can be attributed inter alia to a Pygmalion effect due to self-evaluation (It can also be founded in hetero-evaluation). In these conditions it’s common to see very large changes in Big 5, i’m agreed that it is a language abuse to qualify this as measurement errors but the result is the same, it do not measure real changes. Correct me if i’m wrong but this is why it is very common to confirm the fidelity of a big five with a test-retest process, because Big five is supposed to have a strong over time consistency.
I agree there should be skepticism when people report significant changes on psychometrics that are in general testing populations stable. The same thing arises in small studies sowing meditation boosting IQ. But the hypothesis that meditation can change things that other interventions can’t should be chased down since if true it’s very important.
Yup, this was the conclusion of my first answer ! It would be wonderful, but I truly believe that a two weeks study won’t show any valid result in terms of personnality. But yes i agree, what are they waiting to for to conduct a large scale study ?! :)
Oh, right I was not paying much attention to the two week thing. I got a large effect with a retest one year later with daily practice and multiple retreats.
Yes of course a year is a priori enough to see changes.
I’ve used TMI as a meditation guide off and on for some time. One thing you might consider tracking is “generalized motivation”, or “energy level”, or something like that. You might have to measure this subjectively, by rating how motivated you feel, or you could keep track of your ability to objectively get things done. I find that too much* meditation results in an undesirable degree of affective flattening and a reduction in motivation and energy level. For these reasons, I actually don’t meditate currently.
*”Too much” may vary, but I think 20 minutes per day is a low enough level to avoid the negative side effects. Of course, at 20 minutes a day, you’re also not going to achieve the desirable outcomes.
Thanks for the warning. I’m going to ignore it and charge ahead, but subjectively measure the results, but in terms of felt motivation actually getting things done (i.e., inside and outside view?).
Firstly, I would like to say TMI is a great book and the “interludes” match up well with my subjective experience.
Secondly, the concerns moridinamael mentions are typically not an issue for beginning meditators. Motivation issues usually don’t manifest until one has accumulated a number of “insights” which often requires hundreds or thousands of hours meditating. Flat affect can be mitigated by interleaving metta style meditation into the follow-breath-at-nose routine. Anecdotally from some meditation lineages—and from examining self report data from roughly 200 meditation practitioners—there seems to be a bump in meditation quality that happens after 40 to 45 minutes of sitting on the cushion so I would recommend you aim for 1 hour/day.
As justinpombrio said (for certain types of focused meditation training) attentional blink was shown to decrease:
https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.0050138
Another task is the psychomotor vigilance (PVT) task which has a long history in sleep deprivation research:
https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.0050138
A simple technique to “objectively” measure focused attention style meditation progress is to use a hand tally counter. During a meditation session whenever you catch yourself mind wandering you simple click the counter. At first remembering to click is hard but after a week or two of daily practice it becomes second nature. For me after a while it became so second nature that I would sometimes have enough awareness to click to register the mind wandering but not enough awareness to actually return to the breath. An upgrade would be to hook up to some sort of arduino and timestamp each click looking for trends (e.g. does your mind quiet down earlier in a session as your cumulative practice time increases).
A step up is to track physiological data (e.g. heart rate variability) and EEG (e.g. using the MUSE)
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5838011/
OTOH it is likely you will experience changes to well-being before anything more quantifiable shows up:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5464416/
When reading reports like this it’s worth to be specific about what’s meant with “meditation quality”. Meditating that long gets you into different states. Those states are helpful if the goal is to have spiritual experiences. They also make it more likely to have some negative effects.
My prediction (based on a bunch of informal theory) would be that those states are less valuable when your goal is to train attention and the style in which I train recommends to not go over 20 minutes for beginners who have trouble holding focus anyway.
Is this data available? I’d be interested to see it.
Ah, I may add that task as it seems relevant. Your link is going to the wrong place, though, what did you mean to link to?
I don’t know what to consider a mind wander. Sometimes there’s a big one, and I lose attention and awareness completely until I remember that I’m meditating, and that’s pretty clear. But sometimes there’s a little one, and I maybe lose attention for half a second but after that my attention returns and the thoughts continue “in the background”, and as far as I can tell my attention didn’t wander during a breath/footstep but it’s hard to tell. And sometimes there’s a potential sensory distraction that becomes a little more than that, and either it seems like I’m paying attention to both the distraction and my breath/feet at once, or it seems to jump to the distraction and back, but only briefly.
The issue is that I don’t see any clear dividing line between these different situations, and I don’t see a clear dividing line between attention and awareness. So I don’t know what to consider a mind wander. This isn’t a big deal when labeling, because it probably doesn’t matter how wide I cast my labeling net, but I wouldn’t know how to interpret the counter. Any advice?
Been there, done that :-).
I haven’t experienced this myself: subjectively the first 5 minutes seem strong, and everything after that seems messy. But I’ve seen this stated enough places (TMI, MCTB I think, you, other sources that I forget) that I trust it. Yes, I will continue with 1 hour sessions.
Yeah, this seems likely. I’m already noticing it chipping away slightly at my neuroses. They’re harder to maintain when you see them.
This sounds cool, and I am interested in what you discover.
A problem I can see with doing this as a single-person experiment is that tests like Dual N-Back also function to improve your attention and working memory as you practice them, so improvement over time would be expected whether you were meditating or not.
Right. I’m hoping for a sudden increase in performance for some task. I’m also skeptical of Dual N-Back, as it’s hard to imagine suddenly getting better at it. But I can picture, e.g., an “attentional blink” task quickly go from “impossible” to “possible”.
Or of course, meditation is a gradual process and all effects are gradual and can be trained independently, and I don’t see much.
I just… wouldn’t feel like a proper scientist if I didn’t test it.
Would suggest using n = 1 methodologies. For example, switch between meditating every day for one week, then a week of not meditating. See: http://media.sethroberts.net/blog/pdf/2012-09-24-The-Growth-of-Personal-Science-Implications-For-Statistics.pdf
Oh, yes, in retrospect that’s exactly how you should measure this! However, I think taking full weeks off of meditation would jeopardize my chances of seeing benefits, and I’m not willing to do that. But I will alternate doing tests before and after meditating, as a smaller weaker version of the same idea.
It’s seems difficult in a first place to measure… But why not. Validity and sensitivity won’t be that big of a problem if you decide to use standardized tests. I’m more concern about fidelity.
It will most likely contain some error. Your test can give you some result that you might interpret when in reality it can only show random fluctuation. Understand here : There are standardized tests, so they won’t show you random fluctuations but they’re designed to measure a criteria IN GENERAL and not the part of this measure which is due to your meditation. Here there is kind of a blurring between sensitivity and fidelity...
Fidelity : I’m concern about consistency, there is so much things that can interfere with this especially when you’re trying to measure stress (Stroop), attention and time reaction : tiredness, motivation etc.. And each of these factors are altered by your meditation routine ! It’s kind of a mess. (Maybe Alpha Cronbach ?)
You would need a large amount of try to get some usable data out of it. You would also need to control general tiredness and motivation, who can have a large impact on your result (would be the first thing i’d measure in fact)… Well it seems difficult and I won’t be the person who’s going to stat this up !
I suggest you use standardized test used for meditation if it exists in pseudo clean condition (the cleaner you can)
Good luck
I’m interested in the causal effect that meditation has on my attention, etc. If that effect is mediated through, e.g., reducing fatigue, all the better!
Do you know of any? Right now I plan to use
tagtimeto do free-form experience sampling, plus the meditation games on Quantified Mind once a week or so. (Thanks all for these suggestions!)Yes, attention can be indeed mediated through general tiredness and general motivation but also a large number of other factors like cognition need. The idea is that all of these factors can be well modified by other things than your meditation routine (for example tiredness can be modified just by what you did the very same you make the test), making the measurement difficult implement, can be done but with a hell ton of stats...
No i don’t know any, with a quick research i found this article in the NYT that maybe can give some hints
https://well.blogs.nytimes.com/2013/04/03/how-meditation-might-boost-your-test-scores/
Studies are linked but of course it’ll cost you some money… But you might note that results needs a intensive routine in order to be significant. with a strong training, tests like Wais might be relevant after a few weeks but again i’m not sure these test are made for this purpose unfortunatly...
Good night !
EDIT : Testing bottom up process could be interesting in order to measure attention. A lot of tests are available for visual, auditory attention. Create your own items and make the passation randomized, measure reaction time. One or two weeek with no meditation at all and same duration for with meditation. Present results in term of Z-score and check for trust-interval with the standard deviation of you consider relevant.