LLMs Are Complicated Now | Hacker News Reader

LLMs Are Complicated Now

ianbarber.blog

213 pointsmatt_d76 comments

It's the bitter-lesson to feature-engineering lifecycle.

When a technique or technology is new people are making massive gains by just applying it to some use case, or gathering more data for training, or giving it more resources.

As time goes on those "bitter lesson" gains start to hit the shallow part of the logistic curve and companies have to start investing more and more effort into engineering for each small, incremental gain.

Well put, thanks.

I got a very different message from this, actually much closer to the problem of incumbent advantage.

The known-good thing has been heavily optimized for performance, making it much harder for new technologies to prove that they are better. This is similar to the problem of gas vs electric engines - we had a century of optimization and ecosystem development around gas engines, which creates an uphill battle for electric motors even though they are (eventually) superior on every way /except/ having that massive ecosystem.

The problem isn't as bad here, because software is much more flexible than hardware, and scaling laws give a reasonable way to try things out at smaller scale before going whole hog.

what is the known-good thing? The whole point is that LLMs were not optimized at all, they got better results than older ML algorithms just because they are able to use all of the GPU, where older algorithms are designed for 10yo GPUs and can't make use of modern GPUs. But now you do in fact have to optimize, to the point that transformers look a lot more complicated than "attention is all you need."

the known-good thing is whatever was previously used for the application you're using llms for in any given case. This is often the structured application of human intelligence.

I assume the choice of phrase "bitter lesson" is intentional irony (since the original concept is that you get better results by just scaling up and not trying to be clever with domain-specific knowledge)?

I assume that the bitterness of the bitter lesson is not for engineers but for subject matter experts. I can only imagine how it would feel to discover that your decades of hard-earned expertise don't amount to a whole lot when it comes to domain-specific ML modeling, compared to simply throwing more compute at the problem.

Maybe the ultimate bitter lesson is that entropy always wins in the end.

My read of GP is that there is a cycle economy between "bitter lesson" style domain-general scaling and domain-specific adaptation once the scaling plateaus for the latest tech.

Specifically the you get better results with techniques that can scale up to the amount of data you have. People often think of it as a "just brute force it" but for the lesson to apply you do need to come up with how you're gong to get the data and how you're going to use it.

Why didn't this author compare Llama 3 with GLM 5.2 (released 1 week ago) which is a more standard attention based LLM? To compare 2 separate families of LLMs and then pointing out that they are different is not a surprising result and detracts from the point the author is trying to make.

https://sebastianraschka.com/llm-architecture-gallery/?compa...

If you look at it, the diagrams are very similar, but the main differences are that the feedforward is replaced with a MoE (router to multiple feedforwards) and the model has a different attention implementation.

It’s written by AI.

[[citation needed]]

I am a professional writer and have been for over 30 years. (I do not use any form of LLM ever.) This means I read a lot. This also means that I have 30+ years of experience of readers not understanding what I wrote, or not getting further than the title, or not getting the main message, or inverting it in their heads, or inserting their own message and then complaining when I diverge, and an endless list of Ways People Do Not Get It.

I am also a trained TESOL teacher. Ability to capture gist is a skill we test for and measure, and many, maybe the majority, of native speakers don't have it and don't know.

In recent years I constantly see people going "this is written by AI" and I have yet to see a single of of them able to coherently prove their point. It's all just feelings and hunches.

So I am calling you on this:

How do you know? Show your working. Demonstrate your case.

I don’t think TFA is written by AI.

But AI written pieces do have a certain feeling. A sort of saccatto in the succession of ideas that does not feel natural. They emphasize certain points, and you as a reader, you just wonder why is that. There is the “This thing, not just that thing”. There are also the three successive propositions (mostly in one sentences) to accentuate an idea and “Negation. Strong positive idea in the same direction”.

In general try reading one (vocally) to yourself and it will feel really weird.

Just like em-dashes, some people have always done these though. Why are they penalized with immediate AI slop witch hunts? The LLMs didn't come up with these tics out of thin air.

>The LLMs didn't come up with these tics out of thin air.

LLMs were trained with a lot of synthetic data to transform them from a complete this text into a chatbot, I suspect that this tons of synthetic data that forces the LLM to answer questions into a specific ways also forced them to have this "synthetic/robotic" language. Claude users would have noticed the "belter and suspenders" phrase just started popping out after an update and I am sure is nto because lots of developers used it in their blogs and Anthropic scrapped those blogs in that update.

You have to start with the reason there’s a witch hunt. I love reading. I read books (novels) almost every day and I’m almost always perusing a textbook pr an article for my jobs and my hobbies. The signal/noise for entertainment or information was fairly high, then come LLM tools.

You start to be interested by the title of an article or a book cover, and then you start reading it and it’s just vapor. Nothing tangible to be gained. It’s like buying something expensive and finding out a cheap trinket under the wrapping.

After a couple of times, you will develop a certain kind of heuristics for this kind of texts. It will not be perfect and will have some false positives, but that’s the only way to keep your sanity.

I mean I get it. It bugs me too. And yes, obviously most of the text that people point at as being AI slop is in fact AI slop. At the same time, you also have the other set of asshats that just go respond with things like "AI slop!" to everything when they can't actually be certain. Like I said, some of these AI tells are things that actual real humans do too.

I like your vapor term. For me it's about the content anyways. If it's just some sort of vapid, pointless drivel then I'm not going to like it regardless of who or what wrote it. And in my experience text that strongly correlates with AI tells also strongly correlate with having sparse substance and lots of fluff.

In other words, if people don't like something, just don't read it. Shouting at people for AI generated text just makes you look foolish if the text is not in fact AI generated. And the person shouting "AI slop!" has no way to prove it other than vibes.

I've realized that even when humans write that way, I also stop reading. Manipulative writing always shuts down my interest in reading it. At least when the LLM does it, it's a byproduct of training. When a human does it's intentional.

Yeah see that makes a lot more sense. If one doesn't like the writing style or content, just don't read it. The reflexive "this is AI slop!" complaints have themselves become a caricature. If something was written by AI but one liked the writing style and content, who cares? But regardless of the authorship if it's vapid and annoying, then that's different.

> ust like em-dashes, some people have always done these though.

Everytime someone claims that they have always written like this I grab a pre-2022 post of theirs and five both to a few SOTA chatbots and ask "did the same writer author both these texts".

Thus far I have never gotten a "likely" response.

If the author truly did not use an AI to write something, then it is more likely that theybhave spent so much time conversing with their LLM than reading human authored material that they now sound like an LLM.

This specific article, though, doesn't look anything like LLM output.

PS. Isn't it odd how all LLMs have converged on the same speech patterns, patterns which resemble almost no human authored material outside of high-pressure sales techniques?

I've used pseudo-em dashes going back to the 90s. In that I'll use hyphen (-) a lot. So for me, an actual em dash would be a tell, but if I just have a normal hyper in between two thoughts, it's not.

And yes, I agree that most people who light up on AI tell scans are indeed using AI. That's not my point.

It feels manipulative. The strong language always seems like it is trying to convince you it's right even when there is no need to, such as a factual presentation. Of course if you look at popular pre-AI blogs you can see strains of the same thing, so it's no surprise that llms blend together all of those "most persuasive" methods.

I’m not sure if it is written by an LLM, but anything being called “load-bearing” (formatted that way and all) sets off my alarm bells

Hopefully you don't work in construction!

Amusing :) but I specifically mean in a SWE context. It’s used constantly by LLMs, Claude especially. Genuinely so curious to see certain phrases come up so often, beyond what I would’ve expected for how often they’re used by people

Claude's writing style is at least as distinctive as any human's personal style. It has a long list of favorite words, verbal tics and common structures. On top of that, LLM writing is often bad in a very particular way: it's weak on actual things to say, but with an overheated style.

Some days, I spend over 4 hours a day reading walls of text written by Claude. If I couldn't recognize Claude's default "voice" by now, something would be wrong. It would be like a Hemingway fan not being able to recognize Hemingway. Except more so, because Claude's writing style is getting worse from version to version, descending into self parody.

On the statistical side, Pangram's model identifies AI-authored text with a 1-in-5,000 false positive rate, measured against hold-out texts from before 2022. My "ear" also agrees closely with Pangram. If I think something sounds AI written, Pangram virtually always comes back with "AI, confidence: high."

That reply was AI written.

Lol, no. I've always sounded like that, and there are decades of my writing online.

Also, FWIW, Pangram scores my writing as entirely human.

Claude's writing isn't easy to identify because it uses em-dashes and bulleted lists. Claude's distinctive style goes much deeper than that.

I think it was a joke

This reply was AI written

Claude's default voice, yes. But I'd assume a lot of people have learned to prompt it to something other than its default style. IMO it is good practice to have a style guide to feed in with the prompt.

> On top of that, LLM writing is often bad in a very particular way: it's weak on actual things to say, but with an overheated style.

This point is interesting because it raises the question of what "LLM writing" actually is. If it is expanding a smaller prompt into a larger article then yes, by construction the information density is low. But it can also be used to take a semi-coherent stream of consciousness and turn it into something readable and the people using it that way might already have started to slip under the radar.

This is a lot like how the criminals seem especially stupid because the ones who get caught are disproportionately the stupid ones. The easily detectable LLM writers are going to be the lazy ones.

One thing I think that helps is that for anything more worthwhile than message board posts like this I use Claude to review my text, make suggestions, and iterate on structure with me. But I'm the one writing the bulk of the text. I'll take some of its suggestions verbatim, but only if I genuinely like it better than anything I came up with myself.

The end product is something much more polished than anything I'd writ eon my own, but still comes off as being genuinely from me. At least that's what people have told me when I've asked.

I suspect people who think they are getting away with this are far more obvious than they realize.

> The easily detectable LLM writers are going to be the lazy ones.

To an extent, true. There are a lot of lazy ones though. And even for those who take steps, it sometimes leaves enough of a trace to at least raise the question.

I often run my writing through scans for AI tells. The number of things it flags that are just my own personal vocal tics, that I've had for 40+ years, is amazing.

In other words, correlation != causation

> It has a long list of favorite words, verbal tics and common structures.

Go on, then, let's see the list.

You need to start using LLMs a lot and then you will know how we know.

Edit: You know how you can recognise someone just from their gait while they walk towards you? I would struggle to describe that for an individual person but it doesn't mean I can't identify them from that alone.

> You need to start using LLMs a lot and then you will know how we know.

$SWEARWORD no!

I am an AI vegetarian: I won't touch them ever except for unavoidable ones (defaults on some search engines, e.g. when I use a web browser on its default settings – my own disable that junk) or for machine translation, which remains their only really valid use case in my book.

I want Scrabble rules for HN AI challenges. If someone finds an AI-generated comment, the commenter has violated HN guidelines and the comment should be deleted. But if the accusation is wrong, there should be a penalty for the often massive disruption the accuser has caused to the discussion.

(As of now, that four-word low-effort comment has generated over a thousand words in response, none of which improve this article's discussion.)

I like this idea. Wildly impractical, but I am tired of every article followed by accusations, "Smells like AI!!! I can tell by the pixels"

Grammarly and GPTZero say 0% AI.

Highly doubtful

I am _very_ familiar with Claudish, and to some extent, the other AIs' writing styles. This article is human-written and features human writing quirks.

The very first sentence

> Back in 2022 and 2023 there were two big branches of machine learning happening at Meta.

is unmistakably human. That's not how a LLM would phrase this sentence, and if it did, it would have put a comma after 2023.

You can just prompt an llm to make causal grammar mistakes

Leaving out that comma is not a grammar mistake. The comma slightly changes the feel of the sentence, but it's not wrong to include or omit it. But yeah, I agree with the other commenter that AI would be less likely than a human to omit the comma.

An adverbial clause at the front of a sentence should have a comma? Or is that not what it is?

Adverbial clauses at the front of a sentence don't always have to have a comma. It's often taught this way to new English language learners or in style guides that want to keep things as unambiguous as possible because including the comma in that case is never wrong. But it's also sometimes optional.

Here is the Chicago Manual of Style's Q&A on commas: https://www.chicagomanualofstyle.org/qanda/data/faq/topics/C...

One of the answers cites section 6.34 of the Chicago Manual of Style as follows: "Although an introductory adverbial phrase can usually be followed by a comma, it need not be unless misreading is likely. Shorter adverbial phrases are less likely to merit a comma than longer ones."

For the example we're discussing, "Back in 2022 or 2023", my personal instinct as a university-educated native US English speaker would be to include or omit the comma primarily depending on how much emphasis I want to put on the timing of my statement. I also know that I tend to write overly long sentences with too many commas, so sometimes I'd intentionally counteract my own tendency and limit the complexity of the sentence by removing a comma like that if the sentence still seems good without it. Other times I'd split the sentence into two, but my sentences rarely get as staccato as AI-written ones.

> If you look at it, the diagrams are very similar,

The page links to the same site you do. No wonder it is similar -- the source is the same!

The source is the same in the original article too. He is using a different diagram from the same site on the right to justify his point on how much more complicated things have become.

Yeah, not a great apples-to-apples comparison.

I think the point stands: MoE, a myriad of complex attention approaches, shared layers, you name it. And making it all work together well is a huge trial-and-error pain even for small models, never mind getting to efficient hardware utilization.

The author is correct, the model architecture is now much more complicated. You can see this if you use llama.cpp and follow the project. The earlier models were always fully implemented. Yet with more contributors, as of today tons of latest models only have partial implementation. DeepSeekv3.2 isn't fully implemented, same with KimiK2.6, GLM5.2+, DeepSeekv4 has no implementation, MiniMaxM3 not supported yet, Hy3-preview no implementation. The latest models are just bare bones to run with lots of support missing for the advanced features.

indeed, there's even a (pretty solid) custom server just for DS4 https://github.com/antirez/ds4

-- works very well on high-RAM Macs

I don’t k ow what I’m doing wrong. Everyone says ds4 is faster than a lot of models around the same size, but I’m getting 2t/s with DSv4 vs 12t/s with Minimax 2.7 (16Gb 5080 + 16gb 5060ti + 128gb ram).

ds4 is optimized for systems with unified memory. it works best on apple silicon with 96GB+ of RAM

ah! ok, now that makes sense. Thanks for that.

The architecture is not much more complicated. And llama.cpp not implementing something is more likely an issue of its business model and financial incentives than it being due to raw complexity.

> Why didn't this author compare Llama 3 with GLM 5.2 (released 1 week ago) which is a more standard attention based LLM? To compare 2 separate families of LLMs and then pointing out that they are different is not a surprising result and detracts from the point the author is trying to make.

The entire point of the comparison is that LLMs look vastly different today than before. Comparing more similar LLMs would detract from the point I thought the author was trying to make.

It is misleading the reader since most current LLMs look the same as before. It is cherry picking an example to make a point when it's not necessary at all to make the argument he is trying to make.

> It is misleading the reader since most current LLMs look the same as before.

But most of them do not? They do look vastly different from the earlier incarnations of GPT and Llama.

Can you provide an example? I provided an example of a week old model looking mostly the same.

- with all due respect, from a ux perspective, could you kindly add a page where i can see just the titles of all your blog posts

- https://ianbarber.blog/blogroll

- https://ianbarber.blog/archive

- https://ianbarber.blog/blog

- https://ianbarber.blog/posts

- none of the above links work

- i really dont want to scroll 200 pages just to see what your blog articles are

lol yeah I guess the best move right now is to fetch their /feed and iterate through <post>s

There is https://ianbarber.blog/feed

firefox directly downloads it, i cant see what it is without downloading a file

For someone like me who's never done any hands-on work in ML, the blog is really hard to understand. Whoosh, over my head.

But, I think the underlying problem is that we don't understand how this sh*t works. So, it's just an empirical, iterative mess.

Like physics in the the years shortly before relativity and quantum mechanics.

My read was: "The first 90% of the work takes the first 10% of the time, and the remaining 10% of the work takes the other 90% of the time." And that we are now squarely in the remaining 10% of the work.

> Claude Telenovela

Nice, hadn't seen this one before.

Reply on news.ycombinator.com