Why We Sleep: A Tale of Institutional Failure
statmodeling.stat.columbia.edu
statmodeling.stat.columbia.edu
“Knowing” incomplete, false, distorted, or misleading facts is often much worse than having no information, or having never even considered a topic.
If I read fiction, it may imply some fundamental lens on the world, but I think my brain is better able to recognize it as an opinion, or subjective.
But if I internalize the “truth” of dietary principles or basic psychology, I will make real choices and have my perceptions shaped to believe I am seeing the results of these choices ... and it is often completely wrong.
This can be subtly or devastatingly negative for my success and quality of life.
I think this problem is under considered. In recent years I have started realizing that most of what I read literally makes me dumber.
It’s a thought that has more implications than I think most people want to consider. Especially people whose identity is connected to being someone who “knows” things.
I prefer having a few wrong concepts in my toolbox than none at all. As the Gates quotes in the article suggests, it's good practice to be vigilant and run a basic reality check before absorbing an idea as part of your worldview, but that you might encounter wrong ideas is not a reason to stop reading accessible science books. I don't need to read papers on economic theory if I can pick up a basic book by a respected author and at least end up with some additional concepts, ideas and a good understanding what at least some experts believe is the truth. That doesn't mean I need to believe everything they say and if it matters you should neither rely on one journal's output nor on one author's book as the ultimate truth.
I can read a book about dieting or psychology or evolutionary theory and I should be aware that I won't end up as an expert, but at least I'll very likely be better informed.
"Why we sleep" has lots of good content, generally coherent information, etc. As the article here shows you shouldn't believe everything at face value but you will learn something. I'd love a reaction by the author to the claims here, but even if he says "I was wrong to do this" it doesn't invalidate his whole book or all his knowledge on the topic. He might have taken it out to avoid confusing laymen that all like to imagine themselves in the "under 5 hours" category. That doesn't make it sound in a scientific way but it does if his main aim is to get a message across that sleep is grossly undervalued.
There's lots of science on the need for sleep and what the article here says that there's no proof that sleep under 5 hours is harmful is simply not true.
Edit: url & minor fixes. Also I hope to eventually progress with this, using my http://onemodel.org project.
Edit: ideally anything (or any source of info) could be assigned transitive trust levels. Dietary science would get a low one in my system at this time. ;) Physics, math, and epidemiology would probably get much higher ones.
Question would be whether you are better off at the end. Did your journey through these concepts help you improve your own understanding and action?
Half joking. I'd rather be stupid, get my sleep, and feel great. ;)
I think readers mostly enjoy the book because it’s a reminder of sleeps importance in a world where spending time on the internet and working long hours can seem even more important.
It’s good you decided to write off a whole genre, I’ll enjoy taking a light hearted approach an a open mind to reading similar publications in the future.
I can imagine much common knowledge is difficult to proof and the various alternate Nobel prizes are great at rewarding some of this important but hardly recognised work of "proving the obvious".
Like, this kind of exchange is exactly why academics try to avoid randos from the internet. They tend to seize on one point, declare victory, and refuse to change their minds. And when the academic doesn't grovel in compliance, they declare academia to be a failure. As far as I'm concerned, UC Berkeley responded perfectly.
I mean, obviously it wouldn't pass a "peer-review" analysis, but that is exactly what happens every time in science pop readings. Either we eliminate every science reading that is not strictly papers, or we accept some misinformation here and there.
I respect Andrew Gelman, but in this case I think he jumped to fit this into a pattern where it doesn't belong. I've looked into all the criticisms of Why We Sleep (which, again, covers hundreds of studies), and there's no p-hacking, there's no fraudulent data, there's no piles of studies retracted, and there's no crisis of replication. Literally the worst criticism is that one bar of a bar graph is removed, in a way that doesn't even change the point of the single sentence referring to it. There are simply better targets out there than sleep.
I fundamentally disagree. What else is throwing away data that disagrees with your argument?
1. https://www.reddit.com/r/slatestarcodex/comments/fo3ess/why_...
I still don't think that trimming a graph in a way that doesn't even change the conclusion of the sentence that refers to it (having less than 8 hours of sleep increases injury risk, and it's still true upon included the omitted bar, which barely even changes the average) is worth trashing an entire 300 page book. Academia isn't about following a set of rigid rules, as if it were some endless high school project. It's about finding the truth, and I've seen no evidence that this was harmed at all here.
> I still don't think that trimming a graph in a way that doesn't even change the conclusion of the sentence that refers to it … is worth trashing an entire 300 page book.
You're strawmanning. The trimming is simply the most obvious of many errors found in a single chapter. There are other known errors as well as (probably) unknown errors.
That said, I think we should be forgiving when people make mistakes as long as they own up to them. Neither the author, the publisher nor the university have demonstrated much of a commitment to research integrity in this case. That's my main complaint.
Do any of these errors reverse that information? Is sleep actually harmful? Should we in fact be loading up on caffeine and alcohol and sleeping 3 hours a night? If not, then what's the point?
The example there is:
Tom: All cats are animals. Ginger is an animal. Therefore, Ginger is a cat.
Bill: You have just fallaciously affirmed the consequent. You are incorrect. Therefore, Ginger is not a cat.
> Logician: Here is an example of a syllogism. The cat has four paws. Isidore and Fricot both have four paws. Therefore Isidore and Fricot are cats.
> Old Gentleman: My dog has got four paws.
> Logician: Then it is a cat.
> ...
> Logician: Another syllogism. All cats die. Socrates is dead. Therefore Socrates is a cat.
> Old Gentleman: And he's got four paws. That's true. I've got a cat named Socrates.
> Logician: There you are, you see...
> Old Gentleman: So Socrates was a cat, was he?
> Logician: Logic has just revealed the fact to us.
Substitute "data" for logic in this context I guess
- 5h sleep - 60% injury likelihood
- 6h sleep - 75% injury likelihood
Now maintain the conclusion that the graph supports the idea of the likelihood of injury being higher the fewer the hours slept per night. Of course such manipulation will not make your book a bestseller.
Studies [0] show that the likelihood of your science being bunk increases with your desire to turn them into bestsellers and make more money from them.
[0] Too lazy to manipulate a bar chart to show whatever I want it to show because I don't actually aim to turn this comment into a bestseller.
Like I agree that removing the bar is bad, but Walker's takeaway is pretty much the same as that of the original paper from which the graph is taken - the paper states that predictor for injury is sleeping less than 8 hours, which Walker points out as well.
> First, an enormous literature dedicated to the treatment of depression with sleep deprivation has found that people with depression frequently benefit by not getting a good night’s sleep.
> Second, Walker directly contradicts himself in Chapter 7 by acknowledging that there are cases when a good night’s sleep is not helpful after all...
> Finally, although Walker states that “sleep deprivation is not a realistic or comprehensive therapy option”, a review chapter of sleep deprivation in the book Sleep, Neuronal Plasticity and Brain Function published in 2014 reads: "[C]onsidering its safety, this technique [sleep deprivation] can now be considered among the first-line antidepressant treatment strategies for patients affected by mood disorders."
There is no one-size-fits-all advice for sleep, it would seem. For some folks, yes, too much sleep is harmful, and yes, sleep deprivation therapy benefits them.
Moreover, the point is that scientific literature should be peer-reviewed.
[0] https://guzey.com/books/why-we-sleep/#no-a-good-night-s-slee...
The example cited here, where people on 5 hours of sleep actually have slightly less chance of injury than 6, is by far the most egregious one, but even it doesn't change the conclusion of the sentence where it appears. At most, it would require a footnote: "also, something weird is going on in the tiny 5 hour group".
If the additional data falsified the conclusion, that would be one thing. But it doesn't, so why keep harping about how it wasn't shown?
Is it okay for me to make money by distorting science? At what point does my behaviour cross from "finding the best presentation for the data" into "misleading the public"?
This isn't just one book either. It's a whole industry. People know that some magazines have comprehensive fact-checking, and they used to expect a level of fact-checking in pop sci print books. They didn't know that pop-sci books have almost no fact-checking.
https://slate.com/technology/2016/08/why-doesnt-anyone-fact-...
https://www.cjr.org/analysis/journalism-book-fact-checking-j...
https://www.theatlantic.com/entertainment/archive/2014/09/wh...
Even the greatest popsci book of all time, a brief history of time, is famously a tome almost no one ever reads fully[1]. Every popsci book I've taken up I can never finish because half of it would be repetitive BS and I'll come out of it not taking anymore takeaway points than what was in the article in Forbes promoting the book. The motivations for publishing popular science books by professors are probably to blame here, on top of the fundamental research ethics problems that plague academia nowadays.
1. https://www.washingtonpost.com/news/morning-mix/wp/2018/03/1...
Yes, actually. The point of writing an entire book is to present enough evidence for the reader to come to the author’s conclusion on their own.
The important part of reading a book, the part that has a chance of altering your behavior in the future, is spending time considering the subject matter from various different angles. That’s what gives you the ability to recognize similarities between situations described in the book and situations that appear in your life. This ability to see the things that happen to you in a new light is the real value these books provide.
What a great quote. I'll be stealing that thanks.
It’s a popsci book, not big Rudin.
He doesn’t say If you wake up early stay in bed, actually the advice given was once you’re up, get up, don’t snooze.
The comment to which I replied, in part: "Do any of these errors reverse that information? Is sleep actually harmful?… If not, then what's the point [of arguing about whether Why We Sleep contains errors]?"
I interpret this comment as: "Why We Sleep may contain errors, but I don't care; sleep is obviously good for you. I know how much sleep is necessary for me, and am capable of ignoring whatever Why We Sleep says on the matter of how much is required."
My response is that we should care that Why We Sleep contains errors. People are going to read that book and use it to determine how much sleep they need, whether or not the parent commenter has done. The fact that Why We Sleep contains errors obviously doesn't mean we should get less sleep - the greatest idiot on Earth may say that it's raining, but that doesn't mean it's sunny outside - but it should make you at least a little sceptical of its arguments that you should get more sleep, because that sleep comes at a massive cost.
Suppose for the sake of argument it is the case that I require seven hours of sleep a night, and the additional eighth hour confers little additional benefit (or even confers harm - it doesn't matter to my argument). Suppose I habitually sleep seven hours, but then I read Why We Sleep and am terrified into sleeping eight hours a night. Now I am spending a full 1/24th of my life - about 4.1% of it - on extra sleep that is doing very little for me except robbing me of consciousness.
Perhaps it is the case that for you in particular, there is enough benefit from that eighth hour of sleep to outweigh the cost (and I assert that it's definitely worth everyone individually doing this little experiment on themselves, because people differ)! Perhaps it is the case that for most people, there is enough benefit - but we don't know, and we can't know simply from Why We Sleep, because it contains errors.
(The first part of the comment to which I replied was "The general message and key takeaway I got from the book was that sleep affects your health in various ways and getting a sufficient amount of higher quality sleep is better than not doing so." This is undoubtedly true: so obviously true, in fact, that there is no need for anyone to read a book to learn it.)
It's a natural process of healing and rest, like.. huh?
Just read the paper, and the association is pretty clear there: fewer hours of sleep means more injuries. P = ~0.006 rr=.8 per hour of additional sleep. That’s not a huge effect size, but I find that pretty believable from my lived experience. Lol, yes, p values, but does anyone think less sleep makes them more coordinated? I don’t allow myself to drive long distances on less than 6h sleep based on my prior experience having tried.
For a lay audience explaining the data for the left most point would be hard. Why is 5h of sleep less injurious than 6h? Could be random chance (only 160 students in the study). Could be that those players are worse (coach pulls sleepy kid off the field).
I doubt the most uncharitable interpretation of Walker’s trimming is correct (people who sleep 5h/night are better at sports and less likely to be injured).
This particular criticism rings hollow to me.
https://news.ycombinator.com/item?id=18244061
I mean, none of this is surprising.
Specifically, note that your description of the comment is longer and contains more detail/clarification than the actual comment it describes!
Like someone is on welfare and the welfare includes free health care insurance. Then they get a job, they do not get welfare anymore, but then they need to pay for insurance, so with the job they have less money than before.