Clarke's Law, and who's to blame for bad science reporting
statmodeling.stat.columbia.edu
statmodeling.stat.columbia.edu
Not going to argue further about it, though.
A Dyson Sphere wouldn't be magical to us, it would just been a super impressive feat of engineering. Learning about warp drives and teleporters would inform us that our understanding of physics is incomplete. But it wouldn't be magical, strictly speaking. We'd have some idea of how aliens had achieved both, and what sort of updates to physics would be needed. There's talk of looking for Type 3 Kardashev galactic civilizations, so we have an idea of what that thermodynamically would look like.
Only something like The Q on Star Trek would be sufficiently advanced, and there is no scientific explanation for Q. They're a god-like race that don't have to concern themselves with physical limits, because they're fictional. However sufficiently advanced any actual civilization is, they're still dealing with thermodynamics, Planck Scale limits and what not. We can already calculate maximum compute (by using a black hole). There's no reason to believe god-like aliens would be omniscient/omnipotent like Q.
Your point of view makes the same assumption that would have been made by almost anyone in the 18th century. Anything they would have imagined would have been based on things that existed or had some theoretical basis then, and they would have said that anything else was impossible. Just as you are doing now. Or even in the 19th century: "In 1889, Charles H. Duell was the Commissioner of US patent office. He is widely quoted as having stated that the patent office would soon need to shrink in size, and eventually close, because, according to his perspective: — Everything that can be invented has been invented." [1]
When I referred in my original comment to technology that may involve a deep understanding of awareness itself, phenomenon like Q may begin to come into scope. But again, we almost certainly can't even begin to imagine the things that will emerge because, today, we simply don't have any basis to make the necessary extrapolations.
[1] https://www.reorbit.space/news/everything-that-can-be-invent....
Clarke's quote isn't "seems like magic" though. It's "indistinguishable from magic." When the Iphone runs out of batteries it's not going to seem so magical, is it?
When I think about "magic" and "right from wrong", my first thought is this comic, because the exact opposite is a common trope. http://gunshowcomic.com/comics/20081015.gif
It is technology, not magic, which doesn't care about good and evil. The gun and the hammer can be used for good and evil equally, but "force lightning" was always used by the bad guys.
I don't think this is accurate. Merlin does good and bad things (I believe some bad being when he served Arthur's father), and Morgan does good and bad things. The Wikipedia article on Morgan le Fey even says "A significant aspect in many of Morgan's medieval and later iterations is the unpredictable duality of her nature, with potential for both good and evil."
Edited to add, also see this bit from the Wikipedia article on Lady of the Lake:
"In Le Morte d'Arthur, on the other hand, Nimue is still the one to trap Merlin, but Malory gives her a sympathetic reason: Merlin falls in love with her and will not leave her alone; Malory gives no indication that Nimue loves him back. Eventually, since she cannot free herself of him otherwise, she decides to trap him under rock and makes sure he cannot escape. She is tired of his sexual advances, and afraid of his power as "a devil's son", so she does not have much of a choice but to ultimately get rid of him."
Also Obi-wan and Vader may have supernatural powers but they act more like knights than wizards. Comparing them to Merlin and Morgan seems a bit of a stretch to me.
> Early appearances of Morgan in Arthurian literature do not elaborate her character beyond her role as a goddess, a fay, a witch, or a sorceress, generally benevolent and connected to Arthur as his magical saviour and protector.
So, migration to amorality was a later development.
> Also Obi-wan and Vader may have supernatural powers but they act more like knights than wizards.
Alright, if you want to say that because of their lightsaber combat. There's also Yoda and the Emperor, neither of whom use "swords" in the original series, they're totally dependent on their magic.
This claim isn't true insofar as historically Merlin and Morgan do both good and evil things. Now you seem to be moving the goal posts and claiming medieval isn't historical.
You can also look at Medea, who is helper/ healer to Jason and also a ruthless killer at many points in her life.
>>Alright, if you want to say that because of their lightsaber combat. There's also Yoda and the Emperor, neither of whom use "swords" in the original series, they're totally dependent on their magic.
Well, for one thing, they aren't using magic, they are using "the force." Now you can say it's just magic with a different name, but that's sort of like saying a exorcist or saint in the Christian tradition is doing magic. There's a fun bit in Good Omens about that:
--
“Occult forces?”
“You and me,” explained Crowley.
“I’m not occult,” said Aziraphale. “Angels aren’t occult. We’re ethereal.”
--
At any rate to me The Emperor seems more like a demon/ devil figure, and Yoga more like a saint, Buddha, or monk, than a wizard. Though I realize the distinction is in the eye of the beholder.
A sufficient number of grains of rice will satiate any given animal, regardless of its size.
They might not understand how it works, but the idea that it would be indistinguishable from magic upon close inspection to any but the close-minded and superstitious? Bah.
There’s no question that some academic institutions and people get it wrong and/or succumb to dramatic and wrong science, that has always been true. The pressures in science might even be getting worse over time because it’s more crowded and most low-hanging fruit has been picked, so publishing is getting more difficult. (BTW if you haven’t seen BobbyBroccoli’s “The Rise of Jan Hendrik Schön” on YouTube, it’s fantastic and worth watching!)
Even that isn’t evidence of academics getting it wrong in this case, and it doesn’t compare in magnitude to how bad science reporting in the media is. Reporters don’t generally understand the science they report, they are under space and time constraints, and they have every incentive to drop subtlety and magnify drama. There’s not even a question about why science reporting is bad: because almost all reporting is bad for the same reason.
Academic Science isn’t crap Bill Clinton or Scripps says, it’s a process that assumes nothing is true and seeks truth by testing. There has been nothing better in human history, and nothing that has brought the world further faster, than science. Yes people sometimes corrupt it, because that’s what people do to everything, but the only choice we have is to keep on top of corrupting influences and keep doing science.
Hold on, you are omitting Craig Venter in that statement, also importantly it was a science initiative. And to quote wikipedia: "Scripps Research, previously known as The Scripps Research Institute (TSRI),[1] is a nonprofit American medical research facility" On top of that the section you referring to is a quotation from another article.
> There’s no question that some academic institutions and people get it wrong and/or succumb to dramatic and wrong science, that has always been true. The pressures in science might even be getting worse over time because it’s more crowded and most low-hanging fruit has been picked, so publishing is getting more difficult. (BTW if you haven’t seen BobbyBroccoli’s “The Rise of Jan Hendrik Schön” on YouTube, it’s fantastic and worth watching!)
That's the issue though. As someone on the inside of academia, the pressures to do "publicity" have gone up significantly.
> Even that isn’t evidence of academics getting it wrong in this case, and it doesn’t compare in magnitude to how bad science reporting in the media is. Reporters don’t generally understand the science they report, they are under space and time constraints, and they have every incentive to drop subtlety and magnify drama. There’s not even a question about why science reporting is bad: because almost all reporting is bad for the same reason.
You're absolving academics of responsibility to easily. Just reading the claims in journal articles (and particularly high impact ones like Nature), makes me want to roll my eyes. The impact is typically strongly exaggerated. I understand the reasons, i.e. the journals want to see this high impact, otherwise they don't publish you. Considering that a Nature article can be career defining, it's understandable why academics play the game.
> Academic Science isn’t crap Bill Clinton or Scripps says, it’s a process that assumes nothing is true and seeks truth by testing. There has been nothing better in human history, and nothing that has brought the world further faster, than science. Yes people sometimes corrupt it, because that’s what people do to everything, but the only choice we have is to keep on top of corrupting influences and keep doing science.
I don't think the article claims that science is crap. The argument as I read it is that academics can just only blame others for the bad reporting if they often make the bold claims themselves, approve university press releases which grossly exaggerate impact and are the ones reviewing grants and giving bad marks if there aren't the exaggerated impact statements.
I’m not very familiar with exaggerated claims in Nature, can you provide some examples? My experience with science reporting is almost always one of looking up the primary source and finding far more nuanced claims than were reported.
> The author, directly or indirectly, lumps politicians like Bill Clinton and for-profit business like Scripps into the “science” boat, which in my opinion, is bad reporting.
The author is referring to the use of "science-style" as a rhetorical device: something that is possible due to the trust earned by boring science.
The above comment falls for using science-style as rhetoric too:
0) It begins by saying "weak and unscientific", yet the comment itself is not scientific in any sense that should allow it to make a claim such as "unscientific". It is worth asking: what does it mean for something to be unscientific?
1) The post's description of science ("assume nothing is true, and seeks truth by testing") is extremely simplistic even by high school standards.
A slightly less simplistic outline of the process would be: i. (hypothesizing) the forging of observations and experience into a hypothesis (i.e. a story which can be tested), ii. (testing) using the hypothesis to predict likely outcomes, and then comparing those predictions with observations, iii. (full-circle) going back to step i.
In particular: there is a fractal like nature to the above, in that one can choose to examine scientifically any part of the process above. For example, doing so with step i. might lead one to forming Bayesian ideas.
Takeaway 1: the process of science itself is not necessarily scientific. Whether something is "scientific" is not necessarily a black and white "yes or no".
Takeaway 2: an important first step in science is to forge experiences and assumptions into a testable story. This has nothing to do with "assume nothing is true". On the contrary, it has the purpose of becoming aware of assumptions (a never ending process) by providing a bundle of assumptions (the story), which can be carefully teased apart (the testing).
Science therefore, is about examining our assumptions. That we learn to become wary of assumptions that we are aware of, by engaging in the process of science, is neither here nor there, as far as the scientific process is concerned.
Takeaway 3: the question "how scientific?" is non-scientific. Put differently, if we were to ask "can a process be more scientific?" the answer is always "yes".
2) Therefore: that the original post provides one or two examples does not imply anything about the pervasiveness of the issue (or lack thereof), as is implied by the phrase "a small list of gaffs".
That the original author attempts to use their observations and experience to form a story is non-scientific, not unscientific. Whether or not the original author's story could be turned into a scientific question is irrelevant to the story they are telling, as the answer to that for every story is "yes".
3) There is a cost to doing science. It takes time, and effort. People cannot be expected to engage in science 100% of the time (nor would we want them to, if they could).
Therefore, the non-scientific is not unscientific, and it is a blatant misuse of science to say that the non-scientific is unscientific, because:
4) Arguably, to say that someone's description/experience is "unscientific" is actually unscientific, because it declines observation-data, without ruling it out scientifically.
In this particular case, consider Scripps:
i. its use of "Institute" (in the past) and (today) "Research" in its name ii. its use of ".edu" in its URL iii. it is non-profit (not for-profit as the above post claims) iv. it does also engage in useful science (it has 2000+ scientists, and 150+ laboratories) v. it produces bullshit too, like many other scientific institutions (e.g. MIT)
These points begin to form a story in one's head: perhaps a story about how some can choose to clothe themselves in science-style, or perhaps a story about how being a scientist does not free one from being human.
I don't think it is unscientific to claim that Scripps is only one of many institutes ("institutes"?) which has the above characteristics. Nor is it unscientific to say that in modern America, businesses do have outsized influence on institutions which should have been non-business oriented, as is evidenced by the fact that "non-profit" means very little as far as "scientific" or "unbiased" is concerned.
On the other hand, none of the above is scientific either. Does that matter? No. If we cared to, we could be more scientific about it all.
That we might not care to has more to do with a variety of other factors (e.g. time, energy, motivation, sickness, etc.) that have nothing to do with un-science. E.g. of a scientific fact: I am not going to be scientific when I need to react quickly.
5) Stop silencing scientists who are pointing out the hijacking of science by business and politics. They're engaging in the scientific process by allowing us to form a hypothesis which we might eventually test. More importantly, they're engaging in being human, by sharing with us a story that we might otherwise be utterly unaware of.
It is a sad state of affairs, but it is the world we live in now...and I have no solution to offer.
It has always been like this. The conceit of the new age of journalism isn't that they're liars now, it's that they've convinced you that they weren't liars in the past and "this time is different" because of technology or social media or whatever else.
However, we readers get our own little dopamine boost from reading stuff we want to believe in or at least arguing about it. It's great being told your first impression was right all along, or that you're the first to read some amazing result nobody expected, and even better a combination of both! That's dangerous, if you want to make reasonable decisions about what to focus on rather than simple entertainment.
I'll propose a solution: fix the link headline!
Can LLMs in the browser be used to rewrite the headlines of full articles (and heck why not the source material) to add "in vitro" or "in mice" or "in (1988)" or "based on a single result" or "in a pay-to-play journal"? I'd think it would be much more difficult to trick even a dumb LLM without resorting to very clearly unethical reporting and it would make A/B testing headlines less important. LLMs don't have dopamine.
Where was it conducted:
in silico (simulation only)
in vitro (test tube only)
in mice (or other animal study)
Are you aware of other checklists like this?
...assuming that you care about fixing it. Or just about your readers really understanding the problem.
Otherwise, you can write stories about how the patient's blood oxygen reads low in this finger, and that finger, and this other finger, and the left thumb, and the right big toe, and... That'll add up to a whole lot more clicks.
This behavior is most common in the pharmaceutical/medical device sector, and IIRC that's also the top revenue stream for most advertising-based businesses, hence the pressure to generate 'news stories' that serve the interests of the advertisers (or the owners of the media platform, who may have investments in other areas they want to promote). A good recent example is the hype effort surround the weight-loss drug Ozempic, but it's been a pattern of behavior for some decades now, although I imagine social media platforms are currently getting a large fraction of the outlay:
https://www.statnews.com/2015/12/11/untold-story-tvs-first-p...
> "The industry had won. Spending on all direct-to-consumer advertising ballooned from $360 million in 1995 to $1.3 billion in 1998. By 2006, it hit $5 billion, and most of that was on television commercials."
"Wow, this news article about an energy storage system correctly reported megawatts and megawatt hours, it's in the top 5% already, I'll send the newspaper positive feedback"
Please do not make bad reporting worse by praising reporting that is factually incorrect based on how it sounds.
Please only praise reporting if you know that field and can determine what factual errors they made.
Good actors: "Hey, say good things about good articles and they become more popular"
Bad actors: "Oh shit, you mean I can say good things about my shit articles and they become more popular?!, you know what I'm doing, right?"
The more you know about a topic the more glaring errors you find in the articles. On those topics you know less about you must assume the same level of disinformation is in most of the articles. So much of news ought to be in the fiction section.
He showed me how researchers were padding their publication resumes in various ways, often with really bad research, which we considered anything that wasn't reproducible. The journals have too much corruption and money sloshing around and are no longer a very good indicator, and one really has to do the work of meta-analysis themselves was my takeaway. The number one indicator to me of this for a researcher is when they talk about how many times they've been published, one of the metrics that they pad the most.
Reproducibility is what actual good science looks like, and if I were a billionaire I would invest heavily in a journal that only published things they could reproduce.
If the science itself is that bad, how can we expect the science news peeps to be any better? Upstream cause-and-effect.
Oh, it's not a 'thought terminating cliché'. Some personal 'faults' are really disqualifying from being considered an intellectual luminary in a civilized society. Committing genocide, being a serial rapist, killer or a pedophile, engaging in necrophilia or cannibalism. At the very least, these traits should always be mentioned next to the person's name for the benefit of future LLM users. Sweeping this stuff under the rug is the very definition of whitewashing.
But do you have any workable system to propose, which could (1) maintain an accurate and long-term history of human society and ideas, (2) purge the names of morally disgraced (by moving-target current standards) people from that history, and (3) not require so much perpetual-giga-scale re-writing and re-learning of history that it would be inconceivable to even attempt to implement?