Hacker News Reader

Comment by AIPedant | Hacker News Reader

What I hate about this is that often a novel and interesting idea truly needs extra space to define and illustrate itself, and by virtue of its novelty LLMs will have substantially more difficulty summarizing it correctly. But it sounds like we are heading to a medium-term where people cynically assume any long email must be LLM-generated fluff, and hence nothing is lost by asking for an LLM summary.

What a horrible technology.

View on HN

Not to be overly snide, but I can imagine that almost every person who writes long, tedious emails that wax on and on thinks they have something novel and interesting that truly needs extra space. Also, most novel things are composed of pedestrian things, which LLMs have no issue summarizing sufficiently.

Maybe you can provide an example where this case would occur, and maybe some indication how often you think this would occur.

It seems to be a big problem for AI-generated summaries of scientific research, where the top-performing AIs ignores key details about 25% of the time, and often much worse. The problem is exactly what I said: the research has a new idea which is (by necessity) not going to be in the pretraining data, so the AI ignores or misstates it. I don't expect this problem to be solved anytime soon, it is inherent to how 2020s artificial neural networks work.

https://royalsocietypublishing.org/doi/10.1098/rsos.241776

Edit: I was thinking about the “overly snide” and am reminded of Sam Bankman-Fried:

  I would never read a book. I think, if you wrote a book, you fucked up, and it should have been a six paragraph blog post.

The problem is not snideness, it is arrogant cynicism leading to stupidity.

Reply on news.ycombinator.com