Comment by ToucanLoucan | Hacker News Reader

Comment by ToucanLoucan | Hacker News Reader

Parent Full thread

Is it that crazy? He's doing exactly what the AI boosters have told him to do.

Like, do LLMs have actual applications? Yes. By virtue of using one, are you by definition a lazy know-nothing? No. Are they seemingly quite purpose-built for lazy know-nothings to help them bullshit through technical roles? Yeah, kinda.

In my mind this is this tech working exactly as intended. From the beginning the various companies have been quite open about the fact that this tech is (supposed to) free you from having to know... anything, really. And then we're shocked when people listen to the marketing. The executives are salivating at the notion of replacing development staff with virtual machines that generate software, but if they can't have that, they'll be just as happy to export their entire development staff to a country where they can pay every member of it in spoons. And yeah, the software they make might barely function but who cares, it barely functions now.

I have a long-running interest in NLP, LLMs basically solved or almost solved a lot of NLP problems.

The usefulness of LLMs for me, in the end, is their ability to execute classic NLP tasks, so I can incorporate a call for them in programs to do useful stuff that would be hard to do otherwise when dealing with natural language.

But, a lot of times, people try to make LLMs do things that they can only simulate doing, or doing by analogy. And this is where things start getting hairy. When people start believing LLMs can do things they can't do really.

Ask an LLM to extract features from a bunch of natural language inputs, and probably it will do a pretty good job in most domains, as long as you're not doing anything exotic and novel enough to not being sufficiently represented in the training data. It will be able to output a nice JSON with nice values for those features, and it will be mostly correct. It will be great for aggregate use, but a bit riskier for you to depend on the LLM evaluation for individual instances.

But then, people ignore this, and start asking on their prompts for the LLM to add to their output confidence scores. Well. LLMs CAN'T TRULY EVALUATE the fitness of their output for any imaginable criteria, at least not with the kind of precision a numeric score implies. They absolutely can't do it by themselves, even if sometimes they seem to be able to. If you need to trust it, you'd better have some external mechanism to validate it.

I once tasked an LLM with correcting a badly-OCR'd text, and it went beast mode on that. Like setting an animal finally free in its habitat. But that kind of work won't propel a stock valuation :(

It's mind-blowing the level of correction a modern LLM can achieve. I had to recover an OCR text that had about 30% of the characters incorrect. The result was 99.9% correct, with just the odd confusion whenever the suffix of a word could be interpreted either way and it picked one at random.

So basically a hundred billion dollar industry for just spam and fraud. Truly amazing technological progress.

Reply on news.ycombinator.com