“Why We Sleep” is riddled with scientific and factual errors (2019)
guzey.com
guzey.com
“Why We Sleep” Is Riddled with Scientific and Factual Errors - https://news.ycombinator.com/item?id=26684519 - April 2021 (151 comments)
Why We Sleep: A Tale of Institutional Failure - https://news.ycombinator.com/item?id=22844723 - April 2020 (52 comments)
“Why We Sleep” Is Riddled with Scientific and Factual Errors - https://news.ycombinator.com/item?id=22419958 - Feb 2020 (34 comments)
“Why We Sleep” Is Riddled with Scientific and Factual Errors - https://news.ycombinator.com/item?id=21546850 - Nov 2019 (58 comments)
Why We Sleep, and Why We Often Can’t - https://news.ycombinator.com/item?id=18798366 - Jan 2019 (80 comments)
Productive on six hours of sleep? You’re deluding yourself, expert says - https://news.ycombinator.com/item?id=15401397 - Oct 2017 (295 comments)
Sleep deprivation is increasing our risk of serious illness - https://news.ycombinator.com/item?id=15324195 - Sept 2017 (77 comments)
She's completely healthy otherwise (well, other than recovering from COVID just now).
If you're going to make money giving people health advice, you have an ethical duty to be truthful to a level so high that it's hard to criticize it. Guzey shows here that Walker is not at that level.
I've read the book, and don't think it said that. Have you read the book?
This is the kind of stuff that infuriates most scientists -- we go out of our way to draw strong conclusions based on weak evidence, but people who are willing to do so can usually capture the public imagination with clear narratives and scary anecdotes.
Even if the studies cited in "Why We Sleep" are correctly done, they're such extraordinary claims that they require extensive confirmation before they're to be believed. Laypeople don't understand this, so authors can cite "the scientific literature", sound authoritative, and still mislead.
How am I being misled if cancer is one of those potential issues? It makes sense in many ways. It’s like the old joke of when someone tells you that oxygen is slowly killing you. Most high school science teachers will tell you that fun fact, but you don’t lose sleep over it.
If there's no evidence, or only very poor evidence, then you're being misled.
> It makes sense in many ways.
Lots of things make sense, but are not proven (these are called "hypotheses"), or worse, not true at all ("fallacies"). Without proof, you can't tell one from the other.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6249821/
“Sixty-five studies from 25 articles were included, involving 1,550,524 participants and 86,201 cancer cases. The categorical meta-analysis revealed that neither short nor long sleep duration was associated with increased cancer risk”
I imagine if a similar analysis was done with rates of exercise, smoking, or hydration, we’d see a notable effect. My position now is that sleep is incredibly important, but perhaps not to the extent made out in the book.
I agree.
In general, people take criticism of this particular book, and extrapolate that criticism to mean that lack of sleep is not bad. We can all agree that sleep is good, but still disagree with the specific, wilder claims made in Why We Sleep.
First of all, "consistent sleep" and "health issues" are so vague that can even refer to common sense advice that you don't need a book to tell you, any doctor could. The book doesn't keep at that soundbite level, but gives specific ranges, arguments, and advice. That is what are misleading.
"How am I being misled if cancer is one of those potential issues?" Cancer is a potential issue of almost everything. If the chance of something particular causing it is existing but miniscule to the point of being irrelevant and someone overplays it, while pretending to give scientic advice and bogus numbers of what to do to avoid it, then you're mislead.
The same way "don't drink over two glasses of water per day, you can die of water toxemia" is misleading, even if drinking too much water (above a gallon in a span of a few hours) can indeed lead to water toxemia.
For example, book makes claims about what should be our sleep duration range, which are wrong and misleading. Second, the books makes claims about the effects of stepping out of that range that are also misleading.
For learning about how much you should sleep and what you can get if you don't, it's worse than "gym bro science"...
The way people defending the blog author use the term is as if the author is deliberately deceiving people for their gain. i.e. cynicism. That is not even remotely the case for anyone who read the book.
I also do not agree that it is misleading to provide general guidelines/recommendations and potential consequences. Not a single guideline was surprising or different than common sense. One has to read between the lines of their experiences and the facts presented. (i.e. if you ever worked at a startup or are a parent you'd have experience of lack of sleep and health issues)
The author is deceiving people through sloppy science, superificial verification and editing, and sensenationalist/alarmist key points, to create a catchy "clickbait" book and sell for his gain.
At least for me, I'm interested in how sleep affects my day to day performance since that's what matters most. I couldn't care less if Walker was wrong about how sleep affects lifespan or risk of cancer. A million things affect those two things.
The WHO didn't declare a sleep loss epidemic? "2/3rds of Americans don't get enough sleep" is false? That's what's wrong with this book? Not that part about how there are more driving accidents on the day we reset out clocks for DST? Not the part about how sleep affects traumatic memories in PTSD?
More generally, if you make a claim and someone claims you're entirely wrong, it should be surprising if they don't attack the parts you consider meaningful. If they wanted to publicly demonstrate your incorrectness, why wouldn't they attack the important parts? Either they can't, or they chose not to, but the latter doesn't match the apparent motivations.
Making a long list of everything they said that we don’t have evidence for isn’t really debunking anything. After all a broken clock may occasionally show the correct time. But showing some specific study was misinterpreted or wrong is far more direct.
Personally I find the claim that sleeping too much can shorten your life quite interesting.
> if he was wrong about "something meaningful", Guzey probably would have added it to his list.
I thought you were agreeing with the root comment that the things Guzey pointed out aren't meaningful.
> Any book of Why We Sleep’s length is bound to contain some factual errors. Therefore, to avoid potential concerns about cherry-picking the few inaccuracies scattered throughout, in this essay, I’m going to highlight the five most egregious scientific and factual errors Walker makes in Chapter 1 of the book. This chapter contains 10 pages and constitutes less than 4% of the book by the total word count.
1. There are many many more errors in the book, and the Guzey, not being very well organized, chose to highlight the fact that the author was wrong about cancer and lifespan (who isn't), and a couple facts not really related to the main point.
2. Despite claims to the contrary, Guzey is cherry-picking the few inaccuracies.
But the article we're discussing mentions this as a footnote, therefore I imagine that Guzey doesn't think it's very relevant either, or else he would have/should have led with it.
That's not the main takeaway from Guzey, I think. The takeaway is that there appears to be deliberate misrepresentation of the facts in order to spin a narrative, which is poor form from an academic.
In descending order of editorialization:
- other media
- other non-fiction books
- textbooks
- review papers
- research papers (reading these is discouraged for non-experts unless what you really want is not to see the forest, but to know every detail about the trees)
This is kind-of a meme talking point these days, but the few studies on this haven’t shown that the cause is lack of sleep, they only show some correlations with the time change (i.e., there could be other factors such as putting commutes in the twilight hours.) There aren’t that many crashes supposedly attributed to DST either (e.g. it’s claimed to be like 30 fatal crashes out of more than 35,000 annually), and the supposed temporary increase in crashes is dwarfed by crash rates in the summer anyway. If you care about car crashes at all, DST is a teensy blip. The big problems are 1:speeding and 2:drunk driving. Making even a modest change to reduce speeding would do many many times more good than eliminating DST.
One massive problem with the cadre of people claiming DST change is so harmful is that the majority of people experience social jet lag on a weekly basis. The one hour time change twice a year is dwarfed by the number of people staying up several hours late Friday and Saturday night all year long. (Full disclosure, I’m skeptical of many of the claims made about DST, and I’ve done deep dives into many references only to find very weak to non-existant evidence being used to support extremely strongly worded claims. The voodoo claims about circadian misalignment that permeate DST “research” are bogus.)
It’d be interesting to self-measure your sleep & next-day performance, that might be a lot more informative than someone else’s summary. It’d be interesting to discuss how to design some self-measure tests. What kinds of performance are you thinking about? Do you want to know the risks of activities like driving, or are you thinking about job/school performance?
This. That said, the amount of people I know who has successfully modified their sleep habits to increase everyday performance is magnitude lower than people who have successfully dieted, which is not much to begin with. I wonder what's harder.
the evidence suggesting links between inadequate sleep and long term development of dementia seems at least as important to me than day to day performance in the short term.
I've also found a lot of these points to be quite finnicky too, as [1] discusses
Furthermore, Walker is aware of all the concerns raised in the original article (it was released a long time ago, not sure if it has been edited since). I dimly recall that most, if not all, of the concerns raised by Guzey turned out to be true. Walker essentially got away with it anyway. There's also a debate between the two of them on some British radio station, I believe, though at that point I declined to listen as I knew I'd get frustrated as it was clear you can just get away with this stuff.
What's doubly weird for me is that I spent a year doing research at a prestigious sleep lab when I was studying, and honestly feel that Guzey's writing elsewhere is weirdly hostile to the concept of sleep. I'm extremely careful about it myself, and absolutely have massive performance degradation if I get anything less than eight hours. I,e. I don't even understand why Guzey is so opposed to pro-sleep messaging, but I think that's irrelevant when discussing Walker's conduct in writing the book.
Asking Guzey to focus on just one or two undermines the thrust of his argument. Walker doesn't commit to a few big lies, truth throughout the entire first chapter is conveniently twisted for the sake of 'storytelling' that I saw done constantly in certain academic circles. The disdain for inconvenient facts (and oh boy, some of the graph edits seemed fraudulent to me when I last checked) is unacceptable if you have any respect for your audience or honesty. And I believe there was also a fabricated WHO quote? I just don't see how there can be nuance to this - the only thing I can see Walker hiding behind is a flimsy excuse like "it isn't a journal article". I don't expect pop science to be phenomenal, but I expect an academic not to falsify graphs: https://guzey.com/books/why-we-sleep/#appendix-what-do-you-d...
#1: Walker claims sleep deprivation decreases life span, but so does sleeping too much.
#2: Walker says sleep deprivation is always bad, but sometimes it helps people with depression.
#3: Walker says that lack of sleep alone may kill you because it does for people who get "fatal familial insomnia" (FFI), but insomnia is a symptom and not the root cause of FFI.
#4: Walker says the WHO claims we have a "sleep epidemic", but Walker's citation does not actually show that and there is conflicting research showing we may actually be sleeping more.
#5: Walker said the WHO recommends 8 hours of sleep, but it's actually the National Sleep Foundation and they recommend 7-9
Here's Walker's response to some of this: https://sleepdiplomat.wordpress.com/2019/12/19/why-we-sleep-...
EDIT: I added points #3 and #4 after just to be more charitable and give a full summary instead of just the ones I felt were most nit picky.
#1 is the author using a linear model where it doesn't apply,
#2 is the author first promoting a super simplistic "more sleep = better" idea and then contradicting himself,
#3 is misinterpreting a symptom as a cause
etc.
#1 the premise of the book is that we (both individually and societally) are the only species intentionally depriving ourselves of sleep due to hectic lifestyle and work pressure, which cause undersleeping. It’s a factor we can change. Sleeping “too much” is associated with mortality for different reasons, which according to the book is because of deceases that cause you to be able to sleep more (which would be because the body needs more time to perform the physical recovery function for longer). Most healthy adults can't sleep longer than ~9h for an extended period of time. That, at least, makes a good argument for treating them differently.
#2 Sleep deprivation therapy may work well for some people with depression, and I don’t see the conflict in being open minded. Perhaps walker is a bit too reactive in his tone? He is intentionally a little provocative, and that may indeed be harmful to people of literal-mindedness. But what’s the alternative? Putting a caveat in every paragraph that YMMV? Requiring a test that you understand aggregates and distributions in order to buy a book?
#3 I remember reading the book and getting exactly that impression. If indeed FFI is a neurodegenerative prion decease (same category as mad cow decease) it’s deeply dishonest to say death is due to lack of sleep. That’s very misleading, no excuses.
#4 was a misattribution to the WHO when it was in fact.. the CDC. Welp, I mean, shit happens. This definitely falls under honest mistake.
#5 ok? Thanks for the correction, lol.
Ironically, this is why i really liked Freakonomics. All the studies they uses were just Levitt's studies, and you could just look them up and read them. I recently listened to a podcast criticizing the book and I couldn't even take it seriously, because you had a couple of laypersons who are politically aligned against the Chicago school and use that for the basis of their criticisms. I though it such a hilariously terrible "trashing" of the book that I couldn't stop shaking my head.
If Books Could Kill: https://open.spotify.com/episode/5wHpooGMRsSBrUHhQZbOZp?si=4...
I would prefer if Walker provided the citations for what this author is asking for, but the entire point of a popsci book is that it's not a scholarly article. Walker isn't a doctor, he's a PhD in neurophysiology. I think the criticisms are warranted, but I think it's glaringly obvious from reading that the book isn't a work of serious scholarship.
Doubly so when you are starting with economic article and make conclusions on social science. Economics in particular are pretty bad at reading what was already written about topic they are just discovering.
Peer review ensures bare minimum of formal validity. It won't check your data, nor try to reproduce the study, nothing of the sort. Peer review is a great tool, but it is not not stomp of "settled and actually captures the whole topic".
Most pop sciences books are basically: "I am an authority on this subject, let me explain the world to you." Which is all fine and dandy, involves a ton of faith in that authority, but there's something unique and interesting about a book that's like "here are my papers, let's go through them one by one."
Do I think all of the theories in freakonomics are correct? Of course not! At best they are wildly outdated studies, at worst they are completely wrong and harmful for people to blindly believe. But my point stands, for a pop science book, it's admirable for them to just hand you the data. It's right there. If you think there's a problem or you want to try and confirm the studies, you just can. That's how we should be doing things, and it's really not at all common.
Anyone who reads a pop science book for Truth doesn't understand science. The vast majority of "settled facts" are still debatable at the highest levels of academia, even in the hard sciences.
For any given book, you can construct a list of accusations like these, laden with quotes and references. Some of the accusations may be absolutely true (it's rare for a book to be 100% accurate), or superficially seem to be true but actually more nuanced than that in a way that is not trivial to explain, some may even be fabrications. It's hard to tell for an outsider.
Regardless of the accuracy of such a barrage of accusations, it takes an inordinate amount of time to respond to the criticism, as it will typically contain of several dozen points each of which require a lengthy response. And by that time it doesn't matter, the accusation will have already been accepted as true.
If you're going to publish something like this, you should at least notify the author well ahead of time and give them ample opportunity to explain what you've found and construct a response.
This in particular: "it takes an inordinate amount of time to respond to the criticism"
Ain't it. If you are going to present yourself as an authority on a segment of human knowledge, then you should necessarily EXPECT to receive challenge when you haven't actually done your own homework. Nobody who writes a book while acting as an authority should be incapable of showing their work.
And since they're all aimed at the first chapter, it's not cherrypicking.
I don't find it to be egregious
See also: the gish gallop: https://rationalwiki.org/wiki/Gish_Gallop
> The Gish Gallop is the fallacious debate tactic of drowning your opponent in a flood of individually-weak arguments in order to prevent rebuttal of the whole argument collection without great effort.
Interesting all of the criticisms are individually weak arguments. Like you’re saying no argument is strong?
See how pointless this is?
To argue against a book, you really only need to argue against its central theme. To combat a dozen accusations, you need to reply to each and every one to avoid being wounded by them.
No, you don't get a pass just because your "central theme" is acceptable.
If someone writes a book about a scientific topic, with a "central theme" that is uncontroversial but uninteresting (e.g. "sleep is good"), but otherwise riddles the book with factual and scientific inaccuracies, then they need to fix the problems. If they don't, they're writing science-themed fiction, at best.
At worst, someone can do real harm by dressing up nonsense as science.
If the author doesn't respond to such an inquiry within ample but reasonable time, then you can go loading up the shotgun with accusatory birdshot and take it to the court of public opinion.
That's the last resort.
You're right that it's excatly like nutrition advice or politics in that there's a 80%+ chance whenever someone sells a hugely popular book in this field that most of it is just made the f*** up.
Other sleep experts have asserted that the book is so awful as to constitute research misconduct and what we should do is give posts correcting bad science the attention they deserve and hold shoddy scientists and public intellectuals accountable.
I think its similar to how people will completely write off a book/movie/person for _one_ thing. I get some things have more weight than others and its hard to trust a book if you find one error... but I've seen people say entire books (technical ones) are "trash" because they disagreed with one or two issues.
Seems like a modern phenomenon. Or maybe just rose tinted glasses... but it seemed people could disagree or even point out flaws while still being on the same "team."
Interesting. It looks like it’s making an argument but if you read closely it’s nothing but insinuation and an opinion about the correct way to have disagreements so as to avoid scandal in front of the plebeians.
I don't personally think the article was in any worse taste than the book. Just because someone cites studies doesn't mean they know how to relate data. There's two entire websites dedicated to this sort of fallacy, here (HN) and Reddit. Pop-sci is the most annoying part of the 21st century to me.
I'm not convinced anyone actually understands sleep. We know too little and we can die. Too much, and we can die too. "Just enough" is the answer but as far as I can tell no one can quantify what this even means. I sleep enough to be rested. This seems to be >= 6 hours a night. The book would insinuate that I'm going to die of all sorts of terrible diseases as a result. The book itself isn't good science and I don't think a criticism of such a broad-strokes painting of a complicated subject needs to be any more kind.
To address your greater point if there's any place where you can throw a rock and find bullshit it's nutrition and health "science". In fact, the industry is so rife with bullshit it's often difficult to tease out actual actionable data. More people need to directly criticize these people in such a fashion until they simply go away.
How the hell does this work?
https://astralcodexten.substack.com/p/sleep-is-the-mate-of-d...
The kind that would scare someone like Sagan...
A scientist of what at where?!
And for every border line irresponsible "left wing" guest they have, they have 10 "right wing" ones. Ye, Glenn Greenwald, Alex Berenson, Bari, Taibbi, etc. etc. Perhaps right wing isn't the right label for this new emerging ideology. It seems to be a mix of cynical grifting (ironic given Taibbi's last book) and the politics of grievance and rejection by elites (therefore war must be raged against them) mixed with culture war BS (these days mostly anti-trans rhetoric). This cocktail of mystery seems to mostly live on the right of the political spectrum.
But when you dig a little deeper, most of these individuals were always right wing, they just played along with left wing dogma to be able to succeed in their media roles. Not unlike Trump, knowing that if he wanted to succeed in Hollywood and Manhattan real estate he'd have to vote democrat visibly. But that was never who he was.
> We're talking about a guy who spearheaded the "masks 4 all" push back at the beginning of COVID.
Also, what is left wing about the idea that masks help prevent the spread of respiratory viruses? I also thought that fell into the category of basic science literacy.
I generally go out of my way to say this whenever given the chance, especially in contexts where this is unlikely to be the majority opinion.
I'm sick of cynical folks who don't understand that Lex, Joe, etc. are putting themselves out there and are willing to be wrong, but sincerely and genuinely believe that it's better to move the conversation forward than to sit idly by. Their sincerity is not paper thin, in fact it's thicc af. Youngin language aside, there's so much cynicism and echo chambers out there and these people are actually trying to fight that. It's commendable what they do.
There are some that want to ban the spread of misinformation. There's value to this. Some misinformation is harmful, like what we've seen about vaccines. People have died from COVID needlessly as a result of vaccine scare-mongering, and this is terrible! If there are people out there who are knowingly spreading misinformation they should be prosecuted under some sort of "you-cant-yell-fire-in-a-crowded-theatre" statute.
IN ADDITION to that, there needs to be good faith discussion of the issues. Sticking to the vaccine example, people have legitimate questions. What's a vaccine? How does it work? How is it tested for safety? Are there any side effects? What's mRNA?
People like Fridman and Huberman attempt to have these discussions, and get into details, and explore the alternative arguments and reasons for supporting and rejecting them. Rogan I'm not sure about, I don't watch him much and what I've seen looks more like casual "shooting the shit" than a directed exploration of an issue, but as far as I can tell he seems to be acting in good faith and isn't one of those people covered by the second paragraph above.
The point I'm making is that when people are acting in bad faith and spreading misinformation, we should absolutely take action and, in your words, make it easier for the populace to reject a malicious actor outright, but the people we're talking about generally acting in good faith.
https://www.nytimes.com/2017/10/10/books/review/snooze-micha...
I mean, this Mathew Walker guy has been thoroughly vetted for accuracy by the diligent fact checkers from places known for integrity, like CNN
https://amp.cnn.com/cnn/videos/tv/2020/10/15/matthew-walker-...
or NPR
https://www.npr.org/2017/10/16/558152847/why-we-sleep
More seriously, though, of course it’s true that smart and confident sounding people are going to dupe Joe Rogan. He doesn’t even pretend to be any sort of intellectual or rational truth-seeker. It is more concerning that such lousy kind of “expertise” as exhibited by Dr. Matthew Walker is more than good enough to dupe mainstream media organizations as well, but to anyone paying attention, this should not come as any sort of surprise: public trust in experts has been in free fall for more than a decade, and really accelerated in last 3 years. The thing you should ponder about, which you won’t, is that your first instinct here is to slander your ideological opponents, using slurs like “tech bros”.
In general the media has very low science literacy and loves someone credentialed, self-assured, and with an erudite accent and story to tell that can easily fill airtime.
Acknowledging that junk science has managed to platform itself on some of the podcasts you mentioned, for myself I'd say that the long-form format at least allows for a greater possibility that the b.s. will out itself over 1.5-2 hours, versus a show like TED Radio Hour that is happy to bundle up a few nice sounding stories into something they can slip a Blackrock ad into.
https://hubermanlab.com/dr-matthew-walker-the-science-and-pr...
Huberman has always been kind of interesting to me; his list of topics immediately makes him look like a bro-science peddler, but the guy kicks off all of his podcasts by talking about how he's a Stanford Neuroscience professor, and he reads all the journals himself - not exactly daring someone to oppose him, but you have to admit he's willing to risk some really high credentials by putting himself out there like that.
Of course, like nearly everyone else, I don't go read the studies myself and go hunting for math mistakes and misrepresentation, so it's still all a big appeal to authority.
Sometimes I write comedic science fiction with pseudoscientific argumentation from absurd angles so I don't hold the above against the authors, except they seem to think it should be taken seriously?
The assumption is that errors in written work are evenly distributed (which is such a ridiculous assumption that I have a hard time not being dismissive) Otherwise there would be an evident concern that they cherry-picked a part of the book that had more egregious errors than other parts.
it's an assumption without any data however, so it isn't that great an argument.
Here is a more charitable way to interpret the author’s argument: if first 4% of the book contains substantial number of severe factual errors, it is highly likely that the remaining 96% of the book will also contain many other errors of similar severity.
This argument no way hinges on your imputed assumption that errors are uniformly distributed. Suppose, for example, that there are four times as many errors in the first chapter as there are in other chapters. Then, instead of 125 severe factual errors, we’ll only expect 30 of them or so. How exactly does this detract from the author’s critique of the book? Is book with 30 severe factual errors no longer “riddled” with them, in a way a book with 120 of them is?
Look, you invented the argument the author did not make, and beating it would not even detract from the author’s point if he actually had made it.