_Matthew Walker's "Why We Sleep" Is Riddled with Scientific and Factual Errors_
It may be true that the book is based on some weak studies but anyone can cherry pick a few individual sentences out of their context and from that write an essay about how wrong someone is (essentially the first 4 points that don't relate to a data error are summed up this way).
In fact, even some of the studies that person is referencing don't agree with the theses presented.
As one real basic example: The author of that blog is disingenous when they take the sentence:
> the shorter your sleep, the shorter your life span
Out of context from the rest of the page: https://archive.org/details/why-we-sleep/page/n9/mode/2up
Within context it's clear the author is speaking about specifically getting too little sleep as short sleep. In that context, shorter doesn't mean what is implied by the blog author's response (less in hours).
It's not just weak studies. Guzey provides multiple examples of demonstrably false statements in the book. There is also evidence of flagrant data manipulation. For example, rather than reproducing a chart from a study, Walker presents a modified chart with the data removed that contradicts the claim he wants to make. This calls into question the quality of the entire book and Walker's integrity as a researcher.
At the same time, I think Guzey's analysis is often brought up in order to dismiss the core claim of the book, which is that most people should be getting more sleep. I'm willing to believe that in fact people should be getting more sleep, and I'm also willing to believe that 90% or more of the scientific research referenced by the book is valid. It's just unfortunate that the book seems to be sloppily (and borderline fraudulently) written and researched.
> It's not just weak studies. Guzey provides multiple examples of demonstrably false statements in the book. There is also flagrant data manipulation. For example, rather than reproducing a chart from a study, Walker presents a modified chart with the data removed that contradicts the claim he wants to make. This calls into question the quality of the entire book and Walker's integrity as a researcher.
I did not see any examples of actual research being misrepresented. Where does this happen?
Throughout the post, Guzey is effectively saying: "Walker's book says X but a study says Y" without mentioning the study was released after the book, sometimes years. This is literally the first study linked to that suggests there's no correlation between sleep length and longevity (2018 meta analysis, book released 2017), which is a point the author isn't making in that first section, when viewed in context. If the argument is it's poor communication on Walker's part in the book, sure, I'll agree to that. That's not what Guzey is arguing though; rather he's arguing a statement that doesn't seem to be made. To put it plainly, these two statements are very different:
- If you do not get enough sleep, there is a correlation to worse health effects, worse health outcomes, and less lifespan
vs
- If you don't sleep a minimum number of hours, you will have a shorter lifespan
Walker appears (in context) to suggest the former, while Guzey is arguing against the latter. One of the first problems with any study (let alone meta-analysis) in sleep has to contend with is the fact that the individual data points may require a different number of hours of sleep to be properly rested, and that sleep quality can vary greatly. Simply counting the hours is insufficient.
Really, maybe there's a bigger discussion to be had around the reproducibility crisis as a whole and what it means for people who author books. The problem with books and subjects like this is that they can be an in-time snapshot of knowledge, which are often incorrect when further reviewed (see: Power posing and related). Maybe ethically social scientists should be barred from publishing.
Beyond that, there are plenty of examples of Walker making strong claims without citing sources. For example, there is no citation for the claim that sleeping less than 6 hours per night doubles risk of cancer, the existence of fatal insomnia (a neurodegenerative disease) doesn't prove that lack of sleep will kill somebody, the WHO never declared a sleep loss epidemic, etc.
> Of the 112 athletes studied, 64 athletes (57%) sustained a total of 205 injuries; 48 athletes (43%) were not injured. Of those who were injured, 42 athletes sustained >1 injury (38%). Table 1 includes a summary of the injuries by anatomic location.
And then:
> Sixty-five percent of athletes (56/86) who reported sleeping <8 hours per night were injured, compared with 31% of athletes (8/26) who reported sleeping ≥8 hours per night. Figure 1 shows the likelihood of injury over the 21-month period of reporting, based on hours of sleep per night.
And then the graph is provided which shows the likelihood of injury very strongly correlates to lack of sleep. With a sample size of 86, that 15% difference in the <5hrs group is basically 1 person. We don't know because we don't have the study's raw data that I can find (quickly, at any rate) but 3/5 people reporting < 5 hrs of sleep seems plausible, given that the < 5 hrs group is exactly 60% and given the other percentages for other groups present on the graph.
You can choose to believe dropping the < 5 hrs group is malicious manipulation I suppose but we're probably talking about a sample size of 5/112 for that particular data point. Would it have been better to show the full chart with the raw numbers of participants? Absolutely.
Ironically, Guzey then goes on to indicate the following:
> As an aside, that 9 hours of sleep column is based on exactly 1 child being injured out of 6 children who reported sleeping for 9 hours.
Whenever I see someone being this disingenuous (mentioning the detailed numbers when advantageous, leaving it out when disadvantageous) it makes me question their motives.
Absent knowing the real raw data, I have to consider that Walker's omission on the graph is still in line with the findings of the referenced study:
> In conclusion, lack of sleep and increasing grade in school appear to be associated with increased injury risk in an adolescent athletic population. Encouraging young athletes to get optimal amounts of sleep may help protect them against athletic injuries
Guzey's omission seeks to undermine this point. Perhaps criticism should be levelled at the study itself if that's the goal. Certainly sample size is probably an important factor, or other limitations identified by the study's authors, such as:
> A limitation of this study was that it was conducted at a single academically demanding, urban, private, combined middle school/high school, and the findings may not be generalizable to other populations in different geographic, socioeconomic, and educational settings. Further, the data are based on the self-reported survey responses of adolescents. Another possible limitation is that this study examines the adolescents’ reported chronic sleep deprivation. Acute sleep deprivation, such as getting minimal sleep on a more limited basis, may also put the athlete at increased risk of injury but would not have been captured by this survey as the average amount of sleep might be sufficient. In addition, the athletes were not asked about their dietary intake or specific medications, which may have an effect on their sleep or injury risk. We also did not account for napping in this population as a source of additional sleep. Several authors have shown that daytime napping can improve performance of both athletic and cognitive tasks.
Neither kind of omission are good approaches to presenting science and IMO raw numbers should be as available as possible for all of this kind of data.
[1]: M. D. Milewski et al. “Chronic lack of sleep is associated with increased sports injuries in adolescent athletes, Journal of Paediatric Orthopaedics 34, no. 2 (2014): 129-33. at: https://journals.lww.com/pedorthopaedics/Fulltext/2014/03000...
I still think the book has merit as an accessible way for people to learn more about sleep.