I want to apologize for lack of proper engagement with the post and lack of replies to you as well as a high level of combativeness in the comments I did leave.
This stuff makes me anxious and I feel like I’m just unable to properly explain what makes me disagree so much with you.
My best attempt is that my takes are some combination of:
Academic literature is a highly adversarial playing field and although it’s very tempting for smart outsiders to think that they can just go on Google Scholar and figure out what’s going on, I don’t think this is possible for the vast majority of fields of science and this is certainly not possible for sleep literature. It seems that this surface-area survey is basically what you tried to do and my understanding is that you
never studied biology or neuroscience with critical discussions with specialists in the field
never discussed the papers in-depth with people in the field
never tried to holistically assess the quality of the literature
never studied the history of science in-depth and the dynamics of scientific progress in general
very deep skepticism of sleep literature and the knowledge of how entire fields of science can be led astray and become detached from reality with me becoming convinced that this is indeed what happened with sleep science, thus me being extremely selective with the kind of evidence I take seriously (I tried to explain this in https://guzey.com/theses-on-sleep/#our-priors-about-sleep-research-should-be-weak).
Thus, I have absolutely no issue with me disagreeing with meta-analyses in the field. They very barely count as real evidence for me in fields like sleep science.
The “successful replications” you appeal to similarly just don’t count for me. I have a pretty simple model of how they happen: sleep scientists sit around and come up with an experiment to do. Then if the experiment shows something in line with “sleep good, no sleep bad” they publish it; otherwise they try to make it seems like it’s “sleep good, no sleep bad” or just don’t publish it.
Why would I think that such a strange thing happens? Because I think that the field is dominated by people who have the view “sleep good, no sleep bad” and publishing anything else is way more difficult + people usually become sleep scientists because they think that sleep is very important.
personal experience (very deep conviction that brains are totally fucked up and try to trick us all the time while also being incredibly liable to both self- and external psyops e.g. https://guzey.com/personal/my-journal/, e.g. my video game addiction, e.g. me literally falling asleep when I become anxious or really bored, e.g. me sometimes sleeping 3 hours and feeling fine without any stimulants and sometimes sleeping 8 hours and feeling sleepy, me knowing how much the expectations affect the mental state and how the video game addiction can be literally get expunged from my brain and be brought down to 0 by saying a single sentence to my wife [it is “I will not play video games for {duration}”]; me personally literally jumping out of bed without an alarm after 3 hours of sleep if i’m allowed to play video games and at the same time ability to sleep 8-9 hours in most other cases)
You called some of my explanations of self and external psyops “epicycles” (in particular, the picture with how lack of sleep per se doesn’t actually lead to lack of productivity) but I’m very confident that brains do stuff like this to us all the fucking time (my one certainly does). Most people are just not very good at introspection and don’t notice this.
I think that the very fact that acute sleep deprivation sometimes increases energy and mood should make almost everyone almost completely re-evaluate everything they know or think about sleep because this should just not happen under the “sleep is restorative and is necessary for good functioning” paradigm.
even if we only take people with bipolar disorder: how the hell can they go on so few number of hours a night with their brain being manic but not simply breaking down?
to continue the previous points. The evidence I take seriously is:
trusted personal experience
where the way “trusted” experience are chosen relies on lots of heuristics, background knowledge, intuition, etc. and for an outsider it might look like cherry-picking, “epicycles”, etc. (this also applied to 5.2 and 5.3 below)
trusted anecdotes
trusted field observations
n-of-1-proof-by-contradiction-style arguments
e.g. “A single-point mutation can decrease the amount of required sleep by 2 hours, with no negative side-effects” and “A brain surgery can decrease the amount of sleep required by 3 hours, with no negative-side effects” from my theses are good examples of this. My take is that just these two examples invalidate large chunks of the sleep literature by showing that normal people can have something tweaked just a little bit, radically decrease the amount of required sleep but remain the same otherwise, which simply would not have been possible if these 2-3 hours of sleep they lost were essential for anything important.
Similarly with anecdotes of people sleeping very little for days on end yet being very functional because they are very stressed out/caffeinated/etc. The brains just keep working! Here (https://www.youtube.com/watch?v=3xtwu0xOrls), for example, you can see a 2004 UK TV show in which volunteers slept an average of 1.5-2 hours a night for about a week competing for 100,000 pounds. Yes, they become pretty sleepy over time and the cognition of some of them becomes pretty fucked up. But then, many of them remain close to 100% functional even after many days of super severe sleep deprivation (they don’t have bipolar or anything like this—just normal people), as you can see from the later episodes. Again, this TV show from 2004 for me invalidates probably like 90% of published sleep literature because according to the sleep literature consensus people should become almost incapacitated under such conditions and yet they just don’t when they are sufficiently motivated.
Similarly with Walker. I believe that he’s a charlatan. He doesn’t know the most basic facts about sleep (as the single most outrageous example for me, in paragraph 2 of chapter 1 of his book he wrote “Routinely sleeping less than six or seven hours a night demolishes your immune system, more than doubling your risk of cancer.”), if you watch his talks, it’s very clear that he just has no ability to evaluate evidence critically or just doesn’t care. The fact that a charlatan directs the Center for Human Sleep Science at UC Berkeley and that he gives premier talks to neuroscience conferences tells me that that the entire field is deeply deeply broken. This, even with n-of-1 simply should not happen and shows that there are no well-functioning error-correction mechanisms in the field.
Just to reiterate: I believe that the points 4, 5.4.1, and 5.4.2 invalidate large chunks of sleep literature and are simply not possible under the default “sufficient sleep is good and necessary for proper functioning for normal people on the scale of a few days to a week”.
BDNF thing is, I think, much stronger than you appreciate because: sleep literature is SUPER pro sleep. For everything under the sun they try to show that sleep is good, no sleep bad. The very fact that there’s an open debate with lots of papers showing that BDNF is increased when you are sleep deprived, tells me that there’s something super interesting going on, as otherwise the entire literature would very quickly converge on “oh yeah, just like with everything else good under the soon, BDNF decreases with lack of sleep”
Coincidentally, I wrote this comment after less than 3 hours of sleep today (had 100 mg of caffeine 5 hours ago and another 100 mg of caffeine 3 hours ago).
Speculation on OP’s education is irrelevant. You reject lots of studies by PhDs that did study the field. If she misunderstood something, address the specific error.
Deep skepticism of the sleep literature is fine, even if you rely on some sleep research yourself, but it’s insufficient to respond to the objection of hypocrisy of relying on the sleep literature with “well, I’m really careful about which studies I use”. You need to explain why the studies you use somehow avoid the methodological problems that cause you to reject other studies. If you don’t, it seems like you are just cherry-picking supporting studies because they support you.
It is SO strange to me that you rely so heavily on your personal experience, which is almost the least reliable scientific method available. Who’s to say that your experience generalizes? You’re a very unusual person. I’m always incredibly skeptical of people who have the position “everyone else’s introspection is unreliable, but somehow I’m above all of that” or “you are all brainwashed by consensus, but I can figure out what’s true and what’s BS because I’m smart, educated, and careful”.
Your claim here relies on an apparent failure to seriously consider the gap between ordinary language and scientific claims, which is a gap I’m confident that you are aware of in other contexts: “I think that the very fact that acute sleep deprivation sometimes increases energy and mood should make almost everyone almost completely re-evaluate everything they know or think about sleep because this should just not happen under the “sleep is restorative and is necessary for good functioning” paradigm.” Here’s the thing. A common error in science articles is using a non-standard, specific, operationalized definition for a concept in the actual study and then using an ordinary language meaning for the concept in the conclusions, leading to overgeneralization. IF science were formal logic, then yes, one counter-example should cause us to reject a general proposition. BUT, science articles that say “sleep is restorative” aren’t claiming “sleep is restorative in every case for every person on every night”, they are making a claim about average population effects. THUS, the fact that acute sleep deprivation sometimes increases energy and mood should not make us think that we must reject everything we know about sleep. This also responds to your no-of-1 proof by contradiction point. An analogy: cold weather makes us feel cold. The fact that people suffering from hypothermia will, just before death, suddenly feel super-hot and take off their clothes doesn’t mean that we should reject everything we know about weather and subjective temperature. It means that a general probabilistic claim doesn’t always apply. That’s why I’m not persuaded by your personal experience. You live a super atypical lifestyle. Your experience makes me think that reduced sleep times is something worth studying further, but doesn’t make me reject existing findings.
Some is addressed above, but I am just floored that someone who is skeptical of much of published science due to methodological problems is so willing to be persuaded by “trusted personal experience” and “trusted anecdote”. I agree that we should be much more skeptical of popularized scientific findings due to valid methodological criticisms, but the solution is not to embrace lousy methodology. If we embraced trusted personal experience and anecdotes as reliable methods for truth findings, we lose the ability to reject massively harmful alternative medicine and new age movements that are propped up by them. The whole reason we set up scientific methods such as systematic data-gathering and double-blind trials is to prevent self-deception.
Again, re: the default claim “sufficient sleep is good and necessary for proper functioning for normal people on the scale of a few days to a week”, this is not a formal logic claim about all cases, it is a claim about trends and norms. And I am not convinced that the default claim is limited to the scale of a few days to a week, but is instead about sleep in general.
I’m pretty surprised at Guzey’s tone in responding; even this last response starting with an apology makes arguments that suggest argument motivated by some sort of psychological trigger rather than rational consideration.
I don’t know if this generalizes, but my experience with tone is that it’s mostly unintentional. There’ve been many instances where I’ve written something that seemed perfectly appropriate to me at the time, only to be horrified at how sound when I read it a month later (and the result pattern-matches to guzey’s comment). It also does not require a psychological trigger, it just happens by default when arguing with someone in text form (and it happens more easily when it’s about something status-related like who made better arguments). Took a lot of deliberate effort to change the default to sounding respectful.
I agree that it’s bad enough to be worth mentioning, but I’d be quite surprised if it’s the result of a strategic effort rather than of an unconscious signaling-related instinct.
I’ve found that I’ve lessened this experience of reading something I’ve wrote and being horrified at its tone by going back and reading my comments at various sites. At least once a year I find myself going to my profile page at LW or some other site and just spending a couple of hours reading what I’ve wrote in the past. I think this has helped me be more aware of what my tone is conveying.
Deep skepticism of the sleep literature is fine, even if you rely on some sleep research yourself, but it’s insufficient to respond to the objection of hypocrisy of relying on the sleep literature with “well, I’m really careful about which studies I use”. You need to explain why the studies you use somehow avoid the methodological problems that cause you to reject other studies. If you don’t, it seems like you are just cherry-picking supporting studies because they support you.
I have a tentative guess on why he’s doing that, based on Scott Alexander’s post about trapped priors.
I’ll give an example of the basic problem outlined in the post myself, to spare you from having to read all of it before understanding my comment. Suppose that a physicist spends two hours trying to convince you that the Earth is flat. Would you see that as strong evidence that the Earth is flat? Personally, I’d see that as extremely weak evidence. Instead of updating much that the Earth is flat, the conversation would instead make me seriously consider the following more-plausible-to-me hypotheses:
The physicist really enjoys pulling very elaborate pranks on people.
It’s April fools or something similar and I for some reason just haven’t realized it yet.
The physicist currently has some sort of untreated psychosis.
Any physics department the physicist has studied or worked on is extremely terrible and ought not to be trusted in the future.
I am hallucinating, or otherwise have perceptions of the world that don’t track reality. Perhaps I’m in a dream, perhaps I have extremely-early-onset Alzheimer’s, perhaps I’m on a potent perception-altering drug.
Similarly, if I saw a lot of studies claiming to show something absurd like that prohibited-by-the-laws-of-physics “psychic” phenomena are real, I won’t need to read their methodology to conclude that there’s something wrong with them. And if a group of people claims to have such psychic powers, I won’t think twice before dismissing their personal experience as unreliable. And, at the same time, I’ll accept, without batting an eye, studies and anecdotes claiming that such powers are not possible.
So dismissing arguments from experts, studies, and personal anecdotes as horribly flawed and no more than weak evidence — even before trying to assess their quality — is perfectly reasonable and Bayesian if they’re claiming something that you think is absurd. But clearly, if you have that attitude towards a belief of yours that does not reflect the territory, that will be problematic.
Guzey’s attitude towards sleep research and anecdotes seems compatible with him having a prior that it’s outright absurd that sleep restriction could be harmful. And, like, it’s not an epistemological sin to be born with that prior. But it does mean that what he sees as very weak evidence against his theses won’t necessarily be very weak evidence for people with different priors, if he can’t explain why he thinks that sleep restriction being harmful is absurd.
(I’m not claiming that that is what is going on here. I just thought that the idea of trapped priors was probably relevant.)
Again, re: the default claim “sufficient sleep is good and necessary for proper functioning for normal people on the scale of a few days to a week”, this is not a formal logic claim about all cases, it is a claim about trends and norms. And I am not convinced that the default claim is limited to the scale of a few days to a week, but is instead about sleep in general.
My main objection to Guzey’s response would be a bit different than yours here. I’d point out that I haven’t argued anywhere that “sufficient sleep” is required for “proper functioning,” and that the meta-analyses I quoted don’t purport to show that either. Rather, they report the magnitude and direction of the effect of experimental sleep restriction on e.g. cognitive ability across a variety of different studies, and the magnitude of the effect they show, although substantial, is a far cry from being enough to outright incapacitate you from most everyday tasks. Guzey seems to be responding to a claim like “people do worse on math exams when they’re tipsy compared to when they’re sober” with “but have you talked to tipsy people? They act almost entirely normally!”
If a physicist were to spend two hours trying to explain to me how they knew that the earth was flat, I’d expect to come away from that conversation with a better understanding of the physical world or the social construction of physics knowledge, which would better help me navigate my life, even if I ended up wronger on the bottom-line answer—because that’s how epistemically persuasive explanations work, they have to show an ability to win bets either more often or with less computational cost than alternative hypotheses.
I feel sort of weird about the ‘trapped prior’ point, because I think it’s more reasonably pointed at academic fields than individual people? Like, it is not that surprising for an academic field to have ‘core beliefs’ that everyone who disagrees with is ‘not in the field’, given the forces that people in the field can exert on each other. One fun example of this is PhilGoetz’s post Too Good To Be True, wherein he points to a claim that out of 60 studies studying vaccines and autism, none of them find any link, and then observes that by standard frequentist analysis, ~3 of them should have been significant at the p=0.05 level, and getting none of them significant at that level is pretty unlikely without suppression. And it’s obvious why there would be suppression; no one wants to give ammunition to the enemy.
Separately, it feels like it doesn’t really distinguish ‘justified priors’ from ‘unjustified priors’. If you tell me that a psychology experiment found evidence of psychic effects, I will basically just not believe it. But hopefully if you transported my mind to a universe where psychics were real, I would believe the corresponding studies in those universes—because I had seen things like dowsing being used by oil companies in that universe. This is, from the perspective of a new study, ‘my prior’, but that prior is built out of all of the evidence that I’ve seen before. It’s kind of fair to call my position on psychics a “trapped prior” but it feels more fair to call it “a mountain of evidence”.
I’m pretty surprised at Guzey’s tone in responding; even this last response starting with an apology makes arguments that suggest argument motivated by some sort of psychological trigger rather than rational consideration.
Maybe better to assume good intentions? (And even if someone is biased or motivated by “impure” motives, we try to turn this into a high quality discussion if we can?)
Fair point, although I wasn’t assuming any bad intentions, more like a hard-to-explain emotional intensity that seemed out of character for someone whose writing I am familiar with. But perhaps expressing my genuine surprise was not constructive—thanks. I removed this intro from my post.
I think that if you ask anyone who knows me in-person they will tell you that I’m an unusually emotionally intensive person. My writing is also usually very emotionally intense but it tend to go through getting feedback from like 20 people who tell me to remove all of the excessive language and to tone it down before publication, so it ends up sounding normal. Comment do not go through this kind of process.
Thanks for engaging. I hope we can figure out what our cruxes are.
On points 1 and 2: I still think you haven’t addressed my counter-argument to points of that nature, which I’ve raised (1) in the post itself, (2) in my previous reply to you, and (3) in this comment. To reiterate some of it: you use sleep research to support some of your points, mostly in your review of Walker’s book but also in Theses on Sleep, and it’s not clear to me why research that shows that sleep deprivation is not as bad as people think is admissible evidence to you but research that shows that sleep deprivation is not harmless is not. I was, and remain, skeptical that you have a consistent and rigorous standard for what research you think is admissible and what research you think is not.
...
Just to reiterate: I believe that the points 4, 5.4.1, and 5.4.2 invalidate large chunks of sleep literature and are simply not possible under the default “sufficient sleep is good and necessary for proper functioning for normal people on the scale of a few days to a week”.
Notice that my post does not argue against anything that is invalidated by 4, 5.4.1 or 5.4.2.
As I elaborated before in my post, 5.4.1 pretty much just shows that sleep deprivation doesn’t make you very acutely sick or something, which is (1) not something I’m arguing against, (2) not something the sleep literature is arguing against, (3) not a surprising claim, and (4) not a novel claim.
Concerning 5.4.2, we don’t know how well people in the show would be doing in a rigorous assessment of cognition not subject to learning effects, such as the psychomotor vigilance task or other similar tests. I appreciate that the show is evidence that people are able to perform some tasks even while very sleep-deprived, but again, that’s not something I or the sleep literature are arguing against, and it isn’t novel or surprising.
...
even if we only take people with bipolar disorder: how the hell can they go on so few number of hours a night with their brain being manic but not simply breaking down?
Uh, whatever manic people are doing, it’s putting them at hospitals and making them unable to functionally recover even 2 years later at incredibly high rates, so they really aren’t a good example of a group that sleeps very little consistently without breaking down.
...
Also, notice that I could have written a post just like yours, with the same kinds of evidence that you use — not bringing up any meta-analysis at all — but instead arguing that sleep deprivation impairs cognition and can worsen mood. The kind of evidence that you use does not asymmetrically favor you. You may have a high prior that all anecdotes about sleep deprivation impairing cognition are untrustworthy, and all anecdotes denying that are trustworthy, but that too isn’t asymmetric; why can’t I claim that it’s the other way around because there’s a psyop making people wrongly believe that there are no harms to sleep restriction, fueled by bosses that want their reports to spend more time working and less time sleeping? (This is not something I actually believe, but I’m just pointing out that the kind of evidence that is consistent with your points is not asymmetrically so; you don’t elaborate on why we should expect people to be massively wrong about their experiences in that specific direction and not the other).
I don’t think you’re engaging with the substance of the argument.
The crux is:
Also, notice that I could have written a post just like yours, with the same kinds of evidence that you use, but instead arguing that sleep deprivation impairs cognition and can worsen mood.
This is incorrect. If you argue that “proper sleep is necessary for x” and I show you a counterexample where a person has x but doesn’t have proper sleep, then however many studies or arguments you give for why “proper sleep is necessary for x”, the argument will remain invalidated, which I claim 4, 5.4.1., and 5.4.2. (which you do not represent accurately in your comment—the participants were able to perform complex mental tasks and function in general at what appears to be ~full-capacity, not just “able to perform basic tasks”) all demonstrate.
I don’t recall having argued that “proper sleep is necessary for x.” [1]
I’ve argued that sleep restriction impairs cognition, is associated with negative mood and suicidality (not only positive mood) and causes overeating. Not that you need a certain minimum amount of sleep to perform competently at something. So I still don’t think that the kind of evidence that you bring up is asymmetric.
I’ll come up with an example to explain the difference. Suppose that, after 8 hours of sleep, humans complete a task with an average accuracy of 83%. After 6 hours of sleep, the average accuracy decreases to 71%. And even after a full night of sleep deprivation, accuracy is still 60%, substantially better than a dart-throwing monkey (which would have, say, a 10% accuracy at the task). Suppose that accuracy above 50% is considered acceptable. You point out that the average accuracy of the sleep-restricted/deprived groups is still pretty good, and argue that this shows that sleeping 8 hours per night, or sleeping at all the previous night, is not necessary to complete the task acceptably. And obviously, you’ll be right — but that wouldn’t be an argument against the claim that sleep restriction impairs performance on that task.
you do not represent accurately in your comment—the participants were able to perform complex mental tasks and function in general at what appears to be ~full-capacity, not just “able to perform basic tasks”
I edited my comment. Notice, however, that it’s still unclear how people in the show would be doing in a rigorous assessment of cognition not subject to learning effects, and, as far as I know, we don’t know how well they did on their tasks compared to their baseline performance.
Claims of that sort would not be very general, and their truth-value would obviously depend on what you mean by “necessary,” so I don’t think they’re that interesting. Also, in some tasks there’s probably a very big difference between acceptable performance and peak performance.
Does anybody know if there have been any sleep-deprivation studies that attempt to control for belief effects? I’m think about this sort of thing. The knock-on ramifications in either direction seem like they could be potentially significant. Among other things, belief effects could help to explain the swaths of contradictory studies around this topic.
This isn’t as important as my previous reply (in which I address your object-level arguments), but I wanted to perhaps note that most of your points 1.1 through 1.4 sound, to me, more like an attempt to generate an emotional reaction in the reader than a good-faith effort at pointing out mistakes you think I’ve made or investigating object-level disagreements (although I could be wrong). I don’t recall criticizing you for not having an MD or something, or publicly speculating that you have never thought about [important meta-level epistemological consideration].
I understand that the fact that I did not take biology or neuroscience classes in college is evidence that I would not have a good understanding of sleep research, but I think it is perhaps important to keep in mind that argument screens off authority here, and it sounds plausible that, a lot of the time, domain experts in the area would acquire knowledge in it the same way I do (by reading meta-analyses and systematic reviews, or textbooks based on those). They don’t have some sort of magical essence that makes them more knowledgeable than everybody else could become. They do original research, but not in every sub-area of their field. If I saw you explaining to someone why a massless particle can carry energy and momentum if it travels at the speed of light, and you were using the same arguments that convinced me why that was the case, I would not object simply because that (probably) wasn’t part of your formal education, as it was of mine, or because you didn’t talk to a physicist about it.
(I’m also a bit confused about why you are criticizing me for not talking to people in the field if, in your view, those people are mostly untrustworthy and just want to show that sleep deprivation is bad[1]).
Which, in my experience, was not the case — I was able to find many systematic reviews and meta-analyses claiming that sleep deprivation is probably not bad for some things. (For example, blood pressure, inflammation, and mortality, as I’ve pointed out in the post).
a lot of the time, domain experts in the area would acquire knowledge in it the same way I do (by reading meta-analyses and systematic reviews, or textbooks based on those).
Domain experts usually do learn not only from public information. When it comes to the question of whether or not to believe a scientific paper domain experts usually learn the skills from talking with their collegues.
I had one bioinformatics professor who made a point of saying something in every lecture about how we shouldn’t just believe the literature and how there’s a lot of mistaken papers out there.
One of your disagreements with guzey is whether “X is true because a meta-analysis says so” is a reasonable argument.
(I’m also a bit confused about why you are criticizing me for not talking to people in the field if, in your view, those people are mostly untrustworthy and just want to show that sleep deprivation is bad[1]).
Talking to people is a good way to understand how untrustworthy they are.
even if we only take people with bipolar disorder: how the hell can they go on so few number of hours a night with their brain being manic but not simply breaking down?
Just wanted to tune in on this from anecdotal experience:
My last ever (non-iatrogenic) hypomanic episode started unprompted. But I was terrified of falling back into depression again! My solution was to try to avoid the depression by extending my hypomania as long as possible.
How did I do this? By intentionally not sleeping and by drinking more coffee (essentially doing the opposite of whatever the internet said stabilized hypomanic patients). I had a strong intuition that this would work. (I also had a strong intuition that the depression later would be worse, but I figured I’d cross that bridge when I came to it, even though my depression was life-threatening, because I was cognitively impaired by my episode.)
It worked! It was my longest and most intense (most euphoric and erratic, but least productive) hypomanic episode, and I don’t think this is fully explained by it being later in the progression of my illness.
Did I “not simply break down?” I wouldn’t say that’s the case, even after iirc less than a week of hypomania and ~3 hours of sleep per night.
I would say that the urge to extend my episode was already an obvious thinking error from the hypomania.
I had the worst depression I had ever experienced immediately afterwards, and I would be willing to bet that, within-subject, longer hypomanic episodes in bipolar II patients are followed by more severe (more symptomatic/disabling, not necessarily longer) depressive episodes.
Generally, I would say that bipolar I patients with months-long mania are also “breaking down.” Mania is severely disruptive. Manic patients are constantly making thinking mistakes (inappropriate risks resulting in long-term disability/losses/hospitalizations, delusions, hallucinations). They’re also not happy all the time—a lot of mania and hypomania presents with severe anger and irritability! I would consider this a breakdown. I can’t say how much of the breaking down is because of the sleep deprivation vs. the other factors of the illness.
(Fortunately, I’ve been episode-free for 8 years now, save for a couple of days of hypomanic symptoms on the days I was given new anxiety medications that didn’t work out.)
Hi Natália,
I want to apologize for lack of proper engagement with the post and lack of replies to you as well as a high level of combativeness in the comments I did leave.
This stuff makes me anxious and I feel like I’m just unable to properly explain what makes me disagree so much with you.
My best attempt is that my takes are some combination of:
Academic literature is a highly adversarial playing field and although it’s very tempting for smart outsiders to think that they can just go on Google Scholar and figure out what’s going on, I don’t think this is possible for the vast majority of fields of science and this is certainly not possible for sleep literature. It seems that this surface-area survey is basically what you tried to do and my understanding is that you
never studied biology or neuroscience with critical discussions with specialists in the field
never discussed the papers in-depth with people in the field
never tried to holistically assess the quality of the literature
never studied the history of science in-depth and the dynamics of scientific progress in general
very deep skepticism of sleep literature and the knowledge of how entire fields of science can be led astray and become detached from reality with me becoming convinced that this is indeed what happened with sleep science, thus me being extremely selective with the kind of evidence I take seriously (I tried to explain this in https://guzey.com/theses-on-sleep/#our-priors-about-sleep-research-should-be-weak).
Thus, I have absolutely no issue with me disagreeing with meta-analyses in the field. They very barely count as real evidence for me in fields like sleep science.
The “successful replications” you appeal to similarly just don’t count for me. I have a pretty simple model of how they happen: sleep scientists sit around and come up with an experiment to do. Then if the experiment shows something in line with “sleep good, no sleep bad” they publish it; otherwise they try to make it seems like it’s “sleep good, no sleep bad” or just don’t publish it.
Why would I think that such a strange thing happens? Because I think that the field is dominated by people who have the view “sleep good, no sleep bad” and publishing anything else is way more difficult + people usually become sleep scientists because they think that sleep is very important.
Mass preference falsification is real https://www.youtube.com/watch?v=PW2kxREtdCs, mistaken consensus is real.
personal experience (very deep conviction that brains are totally fucked up and try to trick us all the time while also being incredibly liable to both self- and external psyops e.g. https://guzey.com/personal/my-journal/, e.g. my video game addiction, e.g. me literally falling asleep when I become anxious or really bored, e.g. me sometimes sleeping 3 hours and feeling fine without any stimulants and sometimes sleeping 8 hours and feeling sleepy, me knowing how much the expectations affect the mental state and how the video game addiction can be literally get expunged from my brain and be brought down to 0 by saying a single sentence to my wife [it is “I will not play video games for {duration}”]; me personally literally jumping out of bed without an alarm after 3 hours of sleep if i’m allowed to play video games and at the same time ability to sleep 8-9 hours in most other cases)
You called some of my explanations of self and external psyops “epicycles” (in particular, the picture with how lack of sleep per se doesn’t actually lead to lack of productivity) but I’m very confident that brains do stuff like this to us all the fucking time (my one certainly does). Most people are just not very good at introspection and don’t notice this.
mass psychogenic illness is real https://en.wikipedia.org/wiki/Mass_psychogenic_illness
I think that the very fact that acute sleep deprivation sometimes increases energy and mood should make almost everyone almost completely re-evaluate everything they know or think about sleep because this should just not happen under the “sleep is restorative and is necessary for good functioning” paradigm.
(Sleep deprivation can certainly increase energy/decrease brain fog/etc. both on depressed non-bipolar people and bipolar people in general and it seems that people in general are affected (e.g. https://www.reddit.com/r/slatestarcodex/comments/9tpk94/writing_in_the_morning/?st=jozurz3y&sh=4c0150e1).)
even if we only take people with bipolar disorder: how the hell can they go on so few number of hours a night with their brain being manic but not simply breaking down?
to continue the previous points. The evidence I take seriously is:
trusted personal experience
where the way “trusted” experience are chosen relies on lots of heuristics, background knowledge, intuition, etc. and for an outsider it might look like cherry-picking, “epicycles”, etc. (this also applied to 5.2 and 5.3 below)
trusted anecdotes
trusted field observations
n-of-1-proof-by-contradiction-style arguments
e.g. “A single-point mutation can decrease the amount of required sleep by 2 hours, with no negative side-effects” and “A brain surgery can decrease the amount of sleep required by 3 hours, with no negative-side effects” from my theses are good examples of this. My take is that just these two examples invalidate large chunks of the sleep literature by showing that normal people can have something tweaked just a little bit, radically decrease the amount of required sleep but remain the same otherwise, which simply would not have been possible if these 2-3 hours of sleep they lost were essential for anything important.
Similarly with anecdotes of people sleeping very little for days on end yet being very functional because they are very stressed out/caffeinated/etc. The brains just keep working! Here (https://www.youtube.com/watch?v=3xtwu0xOrls), for example, you can see a 2004 UK TV show in which volunteers slept an average of 1.5-2 hours a night for about a week competing for 100,000 pounds. Yes, they become pretty sleepy over time and the cognition of some of them becomes pretty fucked up. But then, many of them remain close to 100% functional even after many days of super severe sleep deprivation (they don’t have bipolar or anything like this—just normal people), as you can see from the later episodes. Again, this TV show from 2004 for me invalidates probably like 90% of published sleep literature because according to the sleep literature consensus people should become almost incapacitated under such conditions and yet they just don’t when they are sufficiently motivated.
Similarly with Walker. I believe that he’s a charlatan. He doesn’t know the most basic facts about sleep (as the single most outrageous example for me, in paragraph 2 of chapter 1 of his book he wrote “Routinely sleeping less than six or seven hours a night demolishes your immune system, more than doubling your risk of cancer.”), if you watch his talks, it’s very clear that he just has no ability to evaluate evidence critically or just doesn’t care. The fact that a charlatan directs the Center for Human Sleep Science at UC Berkeley and that he gives premier talks to neuroscience conferences tells me that that the entire field is deeply deeply broken. This, even with n-of-1 simply should not happen and shows that there are no well-functioning error-correction mechanisms in the field.
Just to reiterate: I believe that the points 4, 5.4.1, and 5.4.2 invalidate large chunks of sleep literature and are simply not possible under the default “sufficient sleep is good and necessary for proper functioning for normal people on the scale of a few days to a week”.
BDNF thing is, I think, much stronger than you appreciate because: sleep literature is SUPER pro sleep. For everything under the sun they try to show that sleep is good, no sleep bad. The very fact that there’s an open debate with lots of papers showing that BDNF is increased when you are sleep deprived, tells me that there’s something super interesting going on, as otherwise the entire literature would very quickly converge on “oh yeah, just like with everything else good under the soon, BDNF decreases with lack of sleep”
Coincidentally, I wrote this comment after less than 3 hours of sleep today (had 100 mg of caffeine 5 hours ago and another 100 mg of caffeine 3 hours ago).
I found this reply unpersuasive.
By numerical point:
Speculation on OP’s education is irrelevant. You reject lots of studies by PhDs that did study the field. If she misunderstood something, address the specific error.
Deep skepticism of the sleep literature is fine, even if you rely on some sleep research yourself, but it’s insufficient to respond to the objection of hypocrisy of relying on the sleep literature with “well, I’m really careful about which studies I use”. You need to explain why the studies you use somehow avoid the methodological problems that cause you to reject other studies. If you don’t, it seems like you are just cherry-picking supporting studies because they support you.
It is SO strange to me that you rely so heavily on your personal experience, which is almost the least reliable scientific method available. Who’s to say that your experience generalizes? You’re a very unusual person. I’m always incredibly skeptical of people who have the position “everyone else’s introspection is unreliable, but somehow I’m above all of that” or “you are all brainwashed by consensus, but I can figure out what’s true and what’s BS because I’m smart, educated, and careful”.
Your claim here relies on an apparent failure to seriously consider the gap between ordinary language and scientific claims, which is a gap I’m confident that you are aware of in other contexts: “I think that the very fact that acute sleep deprivation sometimes increases energy and mood should make almost everyone almost completely re-evaluate everything they know or think about sleep because this should just not happen under the “sleep is restorative and is necessary for good functioning” paradigm.”
Here’s the thing. A common error in science articles is using a non-standard, specific, operationalized definition for a concept in the actual study and then using an ordinary language meaning for the concept in the conclusions, leading to overgeneralization. IF science were formal logic, then yes, one counter-example should cause us to reject a general proposition. BUT, science articles that say “sleep is restorative” aren’t claiming “sleep is restorative in every case for every person on every night”, they are making a claim about average population effects. THUS, the fact that acute sleep deprivation sometimes increases energy and mood should not make us think that we must reject everything we know about sleep. This also responds to your no-of-1 proof by contradiction point.
An analogy: cold weather makes us feel cold. The fact that people suffering from hypothermia will, just before death, suddenly feel super-hot and take off their clothes doesn’t mean that we should reject everything we know about weather and subjective temperature. It means that a general probabilistic claim doesn’t always apply. That’s why I’m not persuaded by your personal experience. You live a super atypical lifestyle. Your experience makes me think that reduced sleep times is something worth studying further, but doesn’t make me reject existing findings.
Some is addressed above, but I am just floored that someone who is skeptical of much of published science due to methodological problems is so willing to be persuaded by “trusted personal experience” and “trusted anecdote”. I agree that we should be much more skeptical of popularized scientific findings due to valid methodological criticisms, but the solution is not to embrace lousy methodology. If we embraced trusted personal experience and anecdotes as reliable methods for truth findings, we lose the ability to reject massively harmful alternative medicine and new age movements that are propped up by them. The whole reason we set up scientific methods such as systematic data-gathering and double-blind trials is to prevent self-deception.
Again, re: the default claim “sufficient sleep is good and necessary for proper functioning for normal people on the scale of a few days to a week”, this is not a formal logic claim about all cases, it is a claim about trends and norms. And I am not convinced that the default claim is limited to the scale of a few days to a week, but is instead about sleep in general.
I have no comment on this!
I don’t know if this generalizes, but my experience with tone is that it’s mostly unintentional. There’ve been many instances where I’ve written something that seemed perfectly appropriate to me at the time, only to be horrified at how sound when I read it a month later (and the result pattern-matches to guzey’s comment). It also does not require a psychological trigger, it just happens by default when arguing with someone in text form (and it happens more easily when it’s about something status-related like who made better arguments). Took a lot of deliberate effort to change the default to sounding respectful.
I agree that it’s bad enough to be worth mentioning, but I’d be quite surprised if it’s the result of a strategic effort rather than of an unconscious signaling-related instinct.
I agree with you.
I’ve found that I’ve lessened this experience of reading something I’ve wrote and being horrified at its tone by going back and reading my comments at various sites. At least once a year I find myself going to my profile page at LW or some other site and just spending a couple of hours reading what I’ve wrote in the past. I think this has helped me be more aware of what my tone is conveying.
I have a tentative guess on why he’s doing that, based on Scott Alexander’s post about trapped priors.
I’ll give an example of the basic problem outlined in the post myself, to spare you from having to read all of it before understanding my comment. Suppose that a physicist spends two hours trying to convince you that the Earth is flat. Would you see that as strong evidence that the Earth is flat? Personally, I’d see that as extremely weak evidence. Instead of updating much that the Earth is flat, the conversation would instead make me seriously consider the following more-plausible-to-me hypotheses:
The physicist really enjoys pulling very elaborate pranks on people.
It’s April fools or something similar and I for some reason just haven’t realized it yet.
The physicist currently has some sort of untreated psychosis.
Any physics department the physicist has studied or worked on is extremely terrible and ought not to be trusted in the future.
I am hallucinating, or otherwise have perceptions of the world that don’t track reality. Perhaps I’m in a dream, perhaps I have extremely-early-onset Alzheimer’s, perhaps I’m on a potent perception-altering drug.
Similarly, if I saw a lot of studies claiming to show something absurd like that prohibited-by-the-laws-of-physics “psychic” phenomena are real, I won’t need to read their methodology to conclude that there’s something wrong with them. And if a group of people claims to have such psychic powers, I won’t think twice before dismissing their personal experience as unreliable. And, at the same time, I’ll accept, without batting an eye, studies and anecdotes claiming that such powers are not possible.
So dismissing arguments from experts, studies, and personal anecdotes as horribly flawed and no more than weak evidence — even before trying to assess their quality — is perfectly reasonable and Bayesian if they’re claiming something that you think is absurd. But clearly, if you have that attitude towards a belief of yours that does not reflect the territory, that will be problematic.
Guzey’s attitude towards sleep research and anecdotes seems compatible with him having a prior that it’s outright absurd that sleep restriction could be harmful. And, like, it’s not an epistemological sin to be born with that prior. But it does mean that what he sees as very weak evidence against his theses won’t necessarily be very weak evidence for people with different priors, if he can’t explain why he thinks that sleep restriction being harmful is absurd.
(I’m not claiming that that is what is going on here. I just thought that the idea of trapped priors was probably relevant.)
My main objection to Guzey’s response would be a bit different than yours here. I’d point out that I haven’t argued anywhere that “sufficient sleep” is required for “proper functioning,” and that the meta-analyses I quoted don’t purport to show that either. Rather, they report the magnitude and direction of the effect of experimental sleep restriction on e.g. cognitive ability across a variety of different studies, and the magnitude of the effect they show, although substantial, is a far cry from being enough to outright incapacitate you from most everyday tasks. Guzey seems to be responding to a claim like “people do worse on math exams when they’re tipsy compared to when they’re sober” with “but have you talked to tipsy people? They act almost entirely normally!”
If a physicist were to spend two hours trying to explain to me how they knew that the earth was flat, I’d expect to come away from that conversation with a better understanding of the physical world or the social construction of physics knowledge, which would better help me navigate my life, even if I ended up wronger on the bottom-line answer—because that’s how epistemically persuasive explanations work, they have to show an ability to win bets either more often or with less computational cost than alternative hypotheses.
I feel sort of weird about the ‘trapped prior’ point, because I think it’s more reasonably pointed at academic fields than individual people? Like, it is not that surprising for an academic field to have ‘core beliefs’ that everyone who disagrees with is ‘not in the field’, given the forces that people in the field can exert on each other. One fun example of this is PhilGoetz’s post Too Good To Be True, wherein he points to a claim that out of 60 studies studying vaccines and autism, none of them find any link, and then observes that by standard frequentist analysis, ~3 of them should have been significant at the p=0.05 level, and getting none of them significant at that level is pretty unlikely without suppression. And it’s obvious why there would be suppression; no one wants to give ammunition to the enemy.
Separately, it feels like it doesn’t really distinguish ‘justified priors’ from ‘unjustified priors’. If you tell me that a psychology experiment found evidence of psychic effects, I will basically just not believe it. But hopefully if you transported my mind to a universe where psychics were real, I would believe the corresponding studies in those universes—because I had seen things like dowsing being used by oil companies in that universe. This is, from the perspective of a new study, ‘my prior’, but that prior is built out of all of the evidence that I’ve seen before. It’s kind of fair to call my position on psychics a “trapped prior” but it feels more fair to call it “a mountain of evidence”.
Maybe better to assume good intentions? (And even if someone is biased or motivated by “impure” motives, we try to turn this into a high quality discussion if we can?)
Fair point, although I wasn’t assuming any bad intentions, more like a hard-to-explain emotional intensity that seemed out of character for someone whose writing I am familiar with. But perhaps expressing my genuine surprise was not constructive—thanks. I removed this intro from my post.
I think that if you ask anyone who knows me in-person they will tell you that I’m an unusually emotionally intensive person. My writing is also usually very emotionally intense but it tend to go through getting feedback from like 20 people who tell me to remove all of the excessive language and to tone it down before publication, so it ends up sounding normal. Comment do not go through this kind of process.
Thanks for engaging. I hope we can figure out what our cruxes are.
On points 1 and 2: I still think you haven’t addressed my counter-argument to points of that nature, which I’ve raised (1) in the post itself, (2) in my previous reply to you, and (3) in this comment. To reiterate some of it: you use sleep research to support some of your points, mostly in your review of Walker’s book but also in Theses on Sleep, and it’s not clear to me why research that shows that sleep deprivation is not as bad as people think is admissible evidence to you but research that shows that sleep deprivation is not harmless is not. I was, and remain, skeptical that you have a consistent and rigorous standard for what research you think is admissible and what research you think is not.
...
Notice that my post does not argue against anything that is invalidated by 4, 5.4.1 or 5.4.2.
As I elaborated before in my post, 5.4.1 pretty much just shows that sleep deprivation doesn’t make you very acutely sick or something, which is (1) not something I’m arguing against, (2) not something the sleep literature is arguing against, (3) not a surprising claim, and (4) not a novel claim.
Concerning 5.4.2, we don’t know how well people in the show would be doing in a rigorous assessment of cognition not subject to learning effects, such as the psychomotor vigilance task or other similar tests. I appreciate that the show is evidence that people are able to perform some tasks even while very sleep-deprived, but again, that’s not something I or the sleep literature are arguing against, and it isn’t novel or surprising.
...
Uh, whatever manic people are doing, it’s putting them at hospitals and making them unable to functionally recover even 2 years later at incredibly high rates, so they really aren’t a good example of a group that sleeps very little consistently without breaking down.
...
Also, notice that I could have written a post just like yours, with the same kinds of evidence that you use — not bringing up any meta-analysis at all — but instead arguing that sleep deprivation impairs cognition and can worsen mood. The kind of evidence that you use does not asymmetrically favor you. You may have a high prior that all anecdotes about sleep deprivation impairing cognition are untrustworthy, and all anecdotes denying that are trustworthy, but that too isn’t asymmetric; why can’t I claim that it’s the other way around because there’s a psyop making people wrongly believe that there are no harms to sleep restriction, fueled by bosses that want their reports to spend more time working and less time sleeping? (This is not something I actually believe, but I’m just pointing out that the kind of evidence that is consistent with your points is not asymmetrically so; you don’t elaborate on why we should expect people to be massively wrong about their experiences in that specific direction and not the other).
I don’t think you’re engaging with the substance of the argument.
The crux is:
This is incorrect. If you argue that “proper sleep is necessary for x” and I show you a counterexample where a person has x but doesn’t have proper sleep, then however many studies or arguments you give for why “proper sleep is necessary for x”, the argument will remain invalidated, which I claim 4, 5.4.1., and 5.4.2. (which you do not represent accurately in your comment—the participants were able to perform complex mental tasks and function in general at what appears to be ~full-capacity, not just “able to perform basic tasks”) all demonstrate.
I don’t recall having argued that “proper sleep is necessary for x.” [1]
I’ve argued that sleep restriction impairs cognition, is associated with negative mood and suicidality (not only positive mood) and causes overeating. Not that you need a certain minimum amount of sleep to perform competently at something. So I still don’t think that the kind of evidence that you bring up is asymmetric.
I’ll come up with an example to explain the difference. Suppose that, after 8 hours of sleep, humans complete a task with an average accuracy of 83%. After 6 hours of sleep, the average accuracy decreases to 71%. And even after a full night of sleep deprivation, accuracy is still 60%, substantially better than a dart-throwing monkey (which would have, say, a 10% accuracy at the task). Suppose that accuracy above 50% is considered acceptable. You point out that the average accuracy of the sleep-restricted/deprived groups is still pretty good, and argue that this shows that sleeping 8 hours per night, or sleeping at all the previous night, is not necessary to complete the task acceptably. And obviously, you’ll be right — but that wouldn’t be an argument against the claim that sleep restriction impairs performance on that task.
I edited my comment. Notice, however, that it’s still unclear how people in the show would be doing in a rigorous assessment of cognition not subject to learning effects, and, as far as I know, we don’t know how well they did on their tasks compared to their baseline performance.
Claims of that sort would not be very general, and their truth-value would obviously depend on what you mean by “necessary,” so I don’t think they’re that interesting. Also, in some tasks there’s probably a very big difference between acceptable performance and peak performance.
Does anybody know if there have been any sleep-deprivation studies that attempt to control for belief effects? I’m think about this sort of thing. The knock-on ramifications in either direction seem like they could be potentially significant. Among other things, belief effects could help to explain the swaths of contradictory studies around this topic.
This isn’t as important as my previous reply (in which I address your object-level arguments), but I wanted to perhaps note that most of your points 1.1 through 1.4 sound, to me, more like an attempt to generate an emotional reaction in the reader than a good-faith effort at pointing out mistakes you think I’ve made or investigating object-level disagreements (although I could be wrong). I don’t recall criticizing you for not having an MD or something, or publicly speculating that you have never thought about [important meta-level epistemological consideration].
I understand that the fact that I did not take biology or neuroscience classes in college is evidence that I would not have a good understanding of sleep research, but I think it is perhaps important to keep in mind that argument screens off authority here, and it sounds plausible that, a lot of the time, domain experts in the area would acquire knowledge in it the same way I do (by reading meta-analyses and systematic reviews, or textbooks based on those). They don’t have some sort of magical essence that makes them more knowledgeable than everybody else could become. They do original research, but not in every sub-area of their field. If I saw you explaining to someone why a massless particle can carry energy and momentum if it travels at the speed of light, and you were using the same arguments that convinced me why that was the case, I would not object simply because that (probably) wasn’t part of your formal education, as it was of mine, or because you didn’t talk to a physicist about it.
(I’m also a bit confused about why you are criticizing me for not talking to people in the field if, in your view, those people are mostly untrustworthy and just want to show that sleep deprivation is bad[1]).
Which, in my experience, was not the case — I was able to find many systematic reviews and meta-analyses claiming that sleep deprivation is probably not bad for some things. (For example, blood pressure, inflammation, and mortality, as I’ve pointed out in the post).
Domain experts usually do learn not only from public information. When it comes to the question of whether or not to believe a scientific paper domain experts usually learn the skills from talking with their collegues.
I had one bioinformatics professor who made a point of saying something in every lecture about how we shouldn’t just believe the literature and how there’s a lot of mistaken papers out there.
One of your disagreements with guzey is whether “X is true because a meta-analysis says so” is a reasonable argument.
Talking to people is a good way to understand how untrustworthy they are.
Just wanted to tune in on this from anecdotal experience:
My last ever (non-iatrogenic) hypomanic episode started unprompted. But I was terrified of falling back into depression again! My solution was to try to avoid the depression by extending my hypomania as long as possible.
How did I do this? By intentionally not sleeping and by drinking more coffee (essentially doing the opposite of whatever the internet said stabilized hypomanic patients). I had a strong intuition that this would work. (I also had a strong intuition that the depression later would be worse, but I figured I’d cross that bridge when I came to it, even though my depression was life-threatening, because I was cognitively impaired by my episode.)
It worked! It was my longest and most intense (most euphoric and erratic, but least productive) hypomanic episode, and I don’t think this is fully explained by it being later in the progression of my illness.
Did I “not simply break down?” I wouldn’t say that’s the case, even after iirc less than a week of hypomania and ~3 hours of sleep per night.
I would say that the urge to extend my episode was already an obvious thinking error from the hypomania.
I had the worst depression I had ever experienced immediately afterwards, and I would be willing to bet that, within-subject, longer hypomanic episodes in bipolar II patients are followed by more severe (more symptomatic/disabling, not necessarily longer) depressive episodes.
Generally, I would say that bipolar I patients with months-long mania are also “breaking down.” Mania is severely disruptive. Manic patients are constantly making thinking mistakes (inappropriate risks resulting in long-term disability/losses/hospitalizations, delusions, hallucinations). They’re also not happy all the time—a lot of mania and hypomania presents with severe anger and irritability! I would consider this a breakdown. I can’t say how much of the breaking down is because of the sleep deprivation vs. the other factors of the illness.
(Fortunately, I’ve been episode-free for 8 years now, save for a couple of days of hypomanic symptoms on the days I was given new anxiety medications that didn’t work out.)