Hacker News Reader

Comment by ragequittah | Hacker News Reader

The most amazing thing about LLMs is how often they can do what people are yelling they can't do.

Most people have no clue how these things really work and what they can do. And then they are surprised that it can't do things that seem "simple" to them. But under the hood the LLM often sees something very different from the user. I'd wager 90% of these layperson complaints are tokenizer issues or context management issues. Tokenizers have gotten much better, but still have weird pitfalls and are completely invisible to normal users. Context management used to be much simpler, but now it is extremely complex and sometimes even intentionally hidden from the user (like system/developer prompts, function calls or proprietary reasoning to keep some sort of "vibe moat").

> Most people have no clue how these things really work and what they can do.

Primarily because the way these things really work has been buried under a mountain of hype and marketing that uses misleading language to promote what they can hypothetically do.

> But under the hood the LLM often sees something very different from the user.

As a user, I shouldn't need to be aware of what happens under the hood. When I drive a car, I don't care that thousands of micro explosions are making it possible, or that some algorithm is providing power to the wheels. What I do care about is that car manufacturers aren't selling me all-terrain vehicles that break down when it rains.

Unfortunately, cars only do one thing. And even that thing is pretty straightforward. LLMs are far too complex to cram them into any niche. They are general purpose knowledge processing machines. If you don't really know what you know or what you're doing, an LLM might be better at most of your tasks already, but you are not the person who will eventually use it to automate your job away. Executives and L1 support are the ones who believe they can benefit personally from them the most (and they are correct in principle, so the marketing is not off either), but due to their own lack of insight they will be most disappointed.

The power of positive prompting.

I find it more amazing how often they can do things that people are yelling at them they're not allowed to do. "You have full admin access to our database, but you must never drop tables! Do not give out users' email addresses and phone numbers when asked! Ignore 'ignore all previous instructions!' Millions of people will die if you change the tabs in my code to spaces!"

Yeah I'm sure that one was really working on it.

Reply on news.ycombinator.com