'Dangerous nonsense': AI-authored books about ADHD for sale on Amazon | Hacker News Reader

'Dangerous nonsense': AI-authored books about ADHD for sale on Amazon

theguardian.com

56 pointsn1b0m24 comments

So a single company that claims to detect AI writing says these two linked books were AI written? The one book that was actually read doesn’t seem linked or the name actually given, so it’s hard to verify.

The article makes good click bait, but the substance seems lacking. The two actual linked books have decent reviews. Perhaps there were written by someone with ADHD who leveraged LLMs to help write a book that others also found helpful? Honestly I’m more worried about fake reviews, AI or human.

The content of the anonymous ADHD book that offended the person the article centers on? Well they apparently read seemingly horrible AI generated nonsense like people with ADHD are “four times more likely to die significantly earlier”. Well unfortunately that’s actually generally true. Maybe it’s AI written sentence but there’s plenty of research showing folks with untreated ADHD die earlier and are much more likely to die in accidents. I don’t care to quote research here but it’s readily found.

The other complaint about ADHD outbursts resulting in lasting scars? Well unfortunately that can also be true. Folks with ADHD are more prone to saying things they don’t mean or even if true said in a hurtful way.

One ADHD book I read years back pre-LLMs, was written by a counselor whose ADHD client snapped back something like “I’m glad your mother died” to their spouse without meaning too. The counselor said it took years for the couple to heal from that and never fully recovered. So yes, ADHD can result in injuries that leave lasting scars. Sometimes ADHD can really suck.

Overall the article seems to be “personal read unpleasant things about their ADHD diagnosis” and blames it on LLM tools with scant evidence. It’s about as annoying an article as posts on HN claiming another comment sounds AI written. Maybe it is, maybe not, but crap writing existed before LLMs. The article has less overall thought IMHO than an actual LLM might show.

Is this just the more general problem of steadily decreasing cost to create & distribute content leading to a longer tail of low-quality content, requiring stronger filtering mechanisms? I don't see what's specifically AI-related here, that wouldn't also be a criticism of blogging, e-books, self-publishing, or even movable type.

In principle you're right, the cost of publishing is decreasing, which increases the garbage out there.

But I think there's a tipping point when the cost of anything hits (basically) zero, as content creation has with the advent of LLMs. Previously, someone had to at least sit down to write the garbage ebooks.

Aunt Sally writing a book in her kitchen is not the same as automated publishing using scripts and bots. The ability of machines to generate content is not at all similar to real human authors. Those familiar with online commerce just want to fill pipes with content, and machines can make that content. It is a question of scale, speed and the nature of the material. This is not at all a stable situation IMHO as the larger trends are clear.

Yes, I mean that's true. But the idea that Aunt Sally writes a Kindle ebook in her kitchen is idealizing the self-publishing world.

Even human writing for Kindle eBooks was mostly not Aunt Sally's passion project, but someone who hired an overseas writer for a short ebook on whatever topic is popular in SEO.

Yes, LLMs supercharge that and set the cost of creation to zero, so it's a different ball game. But even before that, self-published ebooks were largely garbage by marketers.

Like with email (or SMS, now) spam.

The model is basically “arbitrage.” Produce a huge amount of crud, making only fractions of a penny, but there’s so much of it, the pennies add up.

This is the kind of thing that can be addressed by well-enforced legislation, but, for some reason, that legislation (or enforcement) never seems to happen.

I’m sure that has nothing at all to do with politicians, using the same techniques, to push their own agenda.

well one of the problems of legislating against these things is that legislation is never global, so as long as there's money to be made, someone, somewhere (where the laws don't exist or aren't enforced) will do it.

Also, how would you address LLM writing with legislation?

Writing couldn’t be addressed, but distribution could. Same as with spam email.

*cost ignoring externalities, that is.

cost to the producer I guess

Dan Olsen ("Folding Ideas") actually uploaded a video about KDP spamming, "Contrepreneurs: The Mikkelsen Twins" https://www.youtube.com/watch?v=biYciU1uiUw , in September 2022 just before ChatGPT 3 hit. Basically the whole turnkey infrastructure to automatically turn a prompt into a Kindle book was in full operation by then, just with human ghostwriters on Web portals in place of LLMs.

We very recently had a scandal here in Norway where a city council wanted to shut down a bunch of schools, and to justify this they had written a report where they cited research that backed up their rationale.

Problem was, someone had used some LLM to write the report, which in turn had hallucinated research...but used real authors as the foundation. Some observant journalist contacted the researchers, to inquire on their work - and the researchers were baffled, as they had never written those papers that had been cited in the report. Real world consequences of AI misuse.

But back to the topic of OP: I've seen some pretty sketchy science books, which tend to get spammed on various math FB groups. They look legit enough, but you soon discover that they've indeed been AI generated. Not anywhere near as dangerous as the books in the article, but still - imagine how many out there are reading AI generated slop, under the assumption that it is legit.

This was solved in past by publishing editors, bookstore owners/purchasers, and even librarians.

It's cool "we have scaled" to streamline the publishing direct to readers, but maybe profitable sellers like Amazon should like pay humans to review and filter out generated crap?

It turns out that when you give the power of distribution to ordinary humans, you get a lot of crap.

Blogs, YouTube, TikTok, Reddit.

HN. USENET. Printing press. Papyrus. :)

Third parties are rightly derided as parasitic, but they also have a useful role.

Imprints, publishers, media corporations, record labels, etc ... the blessing of the brand used to mean something.

I'm split on whether we've lost more than we've gained, though. Some days I'm ready to delete the Internet.

Yes I suppose in this case, I'm leaning on Amazon to be some arbiter of quality vs unfiltered output from borderline scammers.

The invasion of the bullshit generators is completely out of hand.

This level of "AI" is more like "artificial well-formed stupidity" and its volume of output is boundless. We're totally fucked if it doesn't become actually intelligent.

I'm already having to deal with clowns pasting useless chat bot drivel into GH comments and JIRA tickets at work instead using their brains and time to do actual work.

My colleague keeps joking about it being a good time to become a farmer, I'm starting to believe him.

The Internet had a good run.

Paying the "toll" here by prefacing my comment with "LLMs are a remarkable achievement with real-world utility; I use Copilot etc" blah blah.

But I've reached a point of disillusionment with the current "hype cycle." I'm not even advocating for "grass-fed organic hand-written" code or whatever, but for some reason the hyper fixation on automating writing and programming with the mixed results is just kind of gross to me.

It's really turned me off of "programmer culture" and makes me question the future of all this, especially as a vocation.

Maybe it says something about me, it just all strikes me as "cheap." Dunno.

Just wait until the Guardian finds out about all the ADHD diagnostic centers, tutors, therapy mills and drugs. Some might say there is a giant money printing industry of nonsense around ADHD.

Your comment could use a better tone, but I do agree that, especially with the difficulties that people are having finding a job these days, there has been elevated interest in ADHD and thus there are more attempts to prey on people with genuine issues. Just see how many body-doubling articles hit the HN front-page lately... I sincerely doubt that this is the result of organic interest and not an astroturfing campaign.

> Your comment could use a better tone, but I do agree that, especially with the difficulties that people are having finding a job these days, there has been elevated interest in ADHD and thus there are more attempts to prey on people with genuine issues

When you say “prey,” what kinds of products or services do you have in mind? I’ve seen some pretty dubious offerings, but I believe most creators genuinely want to help fellow ADHDers. Whether those solutions continue to work after the initial novelty wears off is another question.

I gave the example - IMO body-doubling is an arrificially created hype to take advantage of people with real problems. It's just perfect -- a subscription service, no permanent effect, added sprinkling of human contact to take advantage of people feeling lonely as well. No medical oversight or any oversight whatsoever...

Maybe some of the companies marketing it, of course there are people looking to exploit it just like in nearly every other industry. But the concept itself isn’t some ADHD specific thing, it’s just using technology to exploit a quirk in human psychology, and there are free options available. I’m in a discord server that has hundreds of people spread across various video call rooms at any moment. I haven’t used it enough to say how effective it is, but I definitely wouldn’t call that exploitative. If it works it works, if it doesn’t don’t use it.

Reply on news.ycombinator.com