Hacker News Reader

Comment by fnordpiglet | Hacker News Reader

It’s fascinating how little people really understand how LLMs are impacting businesses. Almost any task that couldn’t be done because it was too expensive to hire thousands of humans to sift through things is being outsourced to LLMs. I find it remarkable that people don’t realize how amazing these things are at classification and structuring of complex stuff with some alacrity, often far superior to human agents. They scale arbitrarily, work 24x7, and have very low failure rates compared to poorly trained high turnover humans. They generally do a good job identifying when they can’t classify something and delegate to human review. Are they perfect? No, but they’re considerably less error prone than a staff of hundreds of humans. Is it lucrative? Absolutely. I’ve seen this now at five major megacorps and I have to believe it’s going on at most.

View on HN

Agreed. I thought the analysis in this article was exceptionally poor.

For example, there is a small section handwaving away the potential impact on the search engine market, saying that ChatGPT has been out a year and hasn't made a sizable dent in Google.

But the thing is, the way ChatGPT interface currently works and the way it is marketed, most people don't even think of it as a "Google replacement" in the first place - except perhaps some techies and folks on HN. But it's not hard to o imagine advancing tech (so LLMs can be more frequently updated) and a rethought UI (and I'm not just talking about plugging in some "ChatGPT-in-Bing"-type interface, either) that could radically change where people go when they want information.

> there is a small section handwaving away the potential impact on the search engine market

I would assume lot of IT folks might have moved to ChatGPT to ask IT (programming) questions instead of searching Google and lot of these folks also use Adblockers which blocked statcounter's trackers so the stats might be misleading by a bit.

That’s also not considering people who “ask ChatGPT things” instead of googling and don’t consider it the same as search

Can you elaborate?

I think you're right that LLMs are very good at classification and structuring, but that's not how I see them used. The experiments I have witnessed resemble "hail Mary" attempts to replace human agents with a magic chatbot that would know it all and speak plainly. Those experiments almost always fail, because the bot hallucinates too much and/or because the training data is poor and not detailed enough to provide efficient fine-tuning.

For example: a helpdesk handles incoming support requests for an internal, proprietary application. Support agents write the request down, then spend time investigating the problem and possible solutions, based on the symptoms. There is a weak link between the symptoms reported by the users and the actual underlying problem, so this takes time, esp. for newly hired support agents.

One approach would be to build a semantic search engine to speed up the investigation phase, which would positively affect the productivity of the agents. But that's not what's been tried: the company hoped to get rid of support agents completely, and replace them with a bot that would interact with the users and solve their problems on the fly.

When that doesn't work, everyone's disappointed and tends to simply give up.

I agree with the approach of augment-not-replace human agents. For example there’s a contact center management company using Langroid[1] (the Multi-Agent LLM framework from ex-CMU/UW-Madison researchers) in exactly this way: they use a Langroid RAG agent to come up with top 3 possible answers, and the human agent then picks a good one. Interestingly, they index past successful conversations of experienced human agents, and the Langroid agent’s suggestions are informed from these as well as company docs.

[1] https://GitHub.com/Langroid/Langroid

So, imagine you need to evaluate applications for some line of credit by reading the documents associated and classifying the businesses in a taxonomy of business types. Depending on volume this could be millions of documents. You can prime an LLM context with the taxonomy and examples of each, then evaluate each document in a clean primed context asking for a classification if it’s clear and a request for review if it’s not clear. This works exceptionally well in practice, almost double the precision and recall of humans.

Businesses are replete with similar relatively trivial classification problems that often require armies of low paid low skill workers but are fairly crucial to the ongoing operations of the business. Chat bots are the default thing people gravitate to with LLMs because they’re the example in chatgpt. But that’s a really uncreative use of the tech, and the domain they’re exposed to is pretty open ended leaving them subject to hallucination, prompt injection, etc. A more constrained domain with more business impact is where I see meaningful applications happening.

From a personal perspective… I have a family photo collection going back decades that I used to put effort into tagging and describing, before realizing that automatic classification models would come eventually. After years of waiting, it feels like we’re close, but I’ve tried LLaVA-1.5 and it was underwhelming. Are there other more specialist models I should try? Must be open source and easy to invoke from a shell or Python script.

Try Fuyu-8B or CogVLM. Failing that, there’s always the GPT-4V API. There are also other non-LLM solutions for image captioning / tagging (e.g. what the smartphone vendors do), but I’m less familiar with those.

+1 on CogVLM.

https://github.com/THUDM/CogVLM

Recent discussions: https://github.com/ggerganov/llama.cpp/discussions/4350

It is the best open source vision language model out there that I'm aware of that's most comparable to gpt4v. Beats the pants off Llava1.5 and variants like bakllava.

There's a demo here http://36.103.203.44:7861/

i have been watching this subthread in hope that something interesting might come out of it and you just delivered. this demo is amazing. i am now dying to apply this to my photo collection.

on that note, some time ago i came across an image management tool that uses machine learning in some form. i don't have time to look for it now, but i just wanted to mention that someone is working on something like that.

>> I’ve seen this now at five major megacorps and I have to believe it’s going on at most.

I sense a potential misaglignment with the article headline that:

    The industries AI is disrupting are not lucrative

While "major megacorps" may indeed be casually using AI for eg "classification and structuring of complex stuff..." I have no problem understanding why your averge mom 'n pop store might not even have problems suitable for this particular type of number-crunching

I think this article makes some interesting points, but it's not exactly on the money (pun intended)

Quote:

    What are AIs of the GPT-4 generation best at? It’s things like:

    - writing essays or short fictions
    - digital art
    - chatting
    - programming assistance

The author lists these points yet fails to realize that these four activities succintly sum up the major part of Internet-as-we-know-it. Add media management for streaming and we're basically there. Also:

    The issue is that taking the job of a human illustrator just. . . doesn’t make you much money. Because human illustrators don’t make much money!

This is direcly seeing things from the wrong perspective. On average illustrators may make little money individually due to a huge over supply in their market, but still those that depend on their services would probably not agree to calling these services "cheap" or that this part of the budget was neglible. This is not about wage competition at the producer stage - it is about cost savings at the consumer stage.

As for the question:

    How precisely will AI capture a portion of the $300 billion movie and game market?

...the author tries to brush it off referring to some strike at one particular place called Hollywood. Well, that market is larger than any particular place, and... it is evolving too. And, at the core of it is exactly the services that LLMs excel at, be it storytelling or hallucinating.

Clarifying my point it seems as if the author makes the claim that extremely lucrative fields are in fact not.

Last, I should add one particular multi-billion dollar industry in which AI seems to have quite some potential: Crime. Specifically, but not limited to, fraud.

Somewhat of a hot take. Normal classification algorithms output a confidence weight, LLM's don't. This means you can only use LLM classification if you don't care at all about managing your precision and recall metrics.

This means LLMs are useless for 90 percent of real business requirements. (Business actually wants some sort of binomial regressor, not a plausible-looking hallucination. Plausible-looking hallucinations are the domain of spam and copywriting.)

Auto Label can do it even with closed-source LLMs: https://www.refuel.ai/blog-posts/labeling-with-confidence

Also, your 90% number is a 'plausible-looking hallucination', which is mildly funny.

>Explicitly prompting the LLM to output a confidence score is the least accurate

They don't show the whole prompt they use for this so I am sceptical of the comparison that recommends their service which does the token level accuracy. In my short testing you can drill down some acceptable accuracy for specific domains if you lead it over some general "ask yourself" questions then "finally, output a % confidence"

I couldn't care less about the specific implementation.

My parent comment said "LLMs are useless for classification, because they can't give confidence scores". I think i have shown that to be demonstrably false.

My understanding is these are more “confidence in the tokens selected being appropriate” rather than the more fundamental “confidence the classification chosen is accurate given training data,” where training data is samples of the distribution being classified rather than language corpus. They seem similar at first blush but the first is more about appropriateness of phrasing and how esoteric the semantics are, and the latter more descriptive of some statistically observable process - which is something LLMs are literally incapable of reasoning about directly.

You can fine tune an LLM to output a particular token for category X. This way, you get 1) fast inference (just need to generate a single token) and 2) confidence metrics (just use the logprobs for the token that was chosen). Gets me to 93% accuracy with a 7B model, trained with only 15 examples per category (around 10-15 categories in total, I think).

The reality is LLMs aren’t competing against normal classification algorithms, as the tasks have no algorithm that approaches the observed precision and recall of LLMs. They’re competing against humans, who also have no metrics other than observed precision and recall. When you observe their performance vs humans at these sorts of tasks they’re generally superior, especially as the quality of the human work is very low - poorly trained, poorly paid, and poorly incentivized people doing very boring work do very badly. LLMs do very well.

The savings can be substantial even tho the people are poorly paid and trained, because you need so many of them, and need a complex organization built around them to source, train, manage, and observe all the humans - of which there can be many thousands with high turnover. Their function is actually pretty important and the high failure rates cause material impact to the business in terms of loss and risk. Improving these dimensions substantially upgrades the business over all.

The truth though is we are really early in the development cycle. These sorts of changes will play out over a decade. In that time I expect the technology to improve in many dimensions - power, capability, ease of integration, ease of observing, ease of fine tuning/alignment to task, applicability to other functions, etc.

As you have seen it in action: how much more additional profit is in being able to do those task now? How much RoE increase is there? (For the megacorps).

In some cases as much as hundreds of millions of profit, in some cases its existential- one situation they needed to spin up a complex business process that would have required an entire organization of likely thousands of humans to comply with a regulatory requirement they were found in violation of and the only way to comply was to have humans review millions of documents and web sites manually to classify them. With a fair amount of fine tuning they were able to build an LLM infrastructure that did this role with the backing of a 20 person review and escalation team. They couldn’t have spun up the equivalent traditional org on time and would have had to had a potentially ruinous public regulatory issue. The cost savings on one hand is hard to gauge, particularly because it would have greatly impacted their culture and organization to add so many humans so quickly. Anyway it’s amazing how many ways these things can be used.

Thank you! So it is rather an incremental change in profit, not transformational (when it isn't life or death of the corporation/a business model requirement)?

I think I would say this. It’s a technology that’s on radars for about 12 months, is immature in all aspects, and has already transformed large swaths of the business operations to be much cheaper, scalable, and precise. I’ve never seen that in 30+ years.

In some cases it actually is transformational as in the case I mentioned that it talked directly to a core business problem that would have killed the org. In others it’s incremental, but at the scale of megacorp there everything is incremental and something that moves margins by 5% or something is earth shattering for them. It unlocks businesses they would have been too hard to comply with requirements, it stream lines operations, etc, which for many businesses are monumental changes.

What happens in 5 years when models are much more powerful, interfaces more clear, and ability to customize easier?

Thanks. I think the big unlock is corporate structure and setup - some classic reasons for having one firm might very well disappear.

Like what

Just imagining some random ideas here and the use case come instantly. Imagine you have a user feedback system that has thousands of submisisons and you want to know how many are talking about a particular feature. How many are speaking positively about the product, how many seem to be asking for support rather than asking for feedback.

Being able to sort through thousands of items for basically free which would have been so laborious previously that you would not have done it. And its a use where a small amount of error does not matter.

Construction business

Lots of government contractors have junior analysts-type jobs whose work entails responding to government RFPs. I know of one specifically who found ChatGPT was great at these responses - it still took further human review and editing, but so did the initial human drafts. Just one example.

How big is the market of "junior analyst-type jobs" globally?

Will disrupting or obliterating that industry make an impact on our lives? Will it add anything to the bottom line of OpenAI?

> How big is the market of "junior analyst-type jobs" globally?

Hence the "Just one example" sentence I ended my comment with. There are tons and tons of others. The Washington Post did a whole long article months ago about people who were laid off _already_ due to ChatGPT. Folks like marketers, copywriters, data analysts, medical transcriptionists, etc. There are a large number of jobs out there that used to need, say, 3 or 4 people, where AI could bring that down to 1.

A lot of people are just in denial. It's some psychological mechanism. They have to call it a stochastic parrot, they have to think of the LLM as something that's only a toy at worst and at best equivalent to google search. They're wrong. This thing can code too. It won't replace a human coder yet, but it's part way there.

It's only a matter of time before it's fully there.

It won't be fully there until it gives a confidence interval and has weights to manipulate precision and recall. These are baseline non-negotiable features.

Unfortunately the way LLM's are architected right now means they will never be "fully there".

Why does it need things that are absent in humans?

Sure, those things could make some things easier, but why are those things that humans don't have suddenly "non negotiable" when a machine also can't do them?

Legal issues?

Those things aren't absent in humans. We spend trillions of dollars to make sure humans are qualified and aren't just bullshitting plausible-sounding words. (In fact, that's really the whole point of "education", which some people spend 20 years being subjected to.)

> confidence interval and has weights to manipulate precision and recall

Neither of which are supplied by education, and I am unaware of any humans basing able to deliberately (let alone precisely) alter their own perception on the scale between precision and recall — look at something, it's "obviously" X or not X almost immediately after you know what the category X is, even when you're wrong. Apart from the very first few encounters it doesn't even matter how much of a noob or expert you are in the field of X-recognition, your confidence is the same.

Worse:

> qualified

Given how well ChatGPT does on standardised tests, doing better than many actual humans even despite its many flaws and limitations, it should be clear that the qualifications are not good enough to do what you're expecting them to do.

> and aren't just bullshitting plausible-sounding words.

That's demonstrably how humans work (at least when it can be tested, perhaps people who need split brains are weird): all the indications are we do a thing first and then come up with a justification after.

(And then we have people like Boris Johnson, 2:1 BA from Oxford, with a disconnect between reality and the words leaving his mouth that would be comical except he actually became Prime Minister in real life and not just a TV comedy blending 'Allo 'Allo with The Thick of It).

There's research on estimating confidence from neural net activations in LLMs.

That’s absolutely false.

1) statistical intervals are mathematical artifacts of our techniques that describe the samples observed and trained on. They aren’t ground truth observations of the underlying process or populations. We put too much weight in them.

2) you can absolutely observe precision and recall from online performance and compare that directly against human performance on known labeled data. From that you can determine which has the better error rates. That is entirely sufficient for almost all practical use cases.

3) obviously it would be better if we could derive confidence of a classification, but given the fact LLMs aren’t directly reasoning or optimizing the statistical properties of some mathematical problem they will never have the same character as say regressions or other statistical techniques that are some form of mathematical optimizer. They’re just solving the problem in fundamentally different ways.

4) it’s not clear to me statistical optimization is a universally superior technique. The reality is many problems are better solved with an abductive reasoning technique like LLMs exhibit, and humans absolutely use when classifying. There are lots of awesome features such as the ability to inspect residuals and confidence intervals, and they’re generally computationally cheap. But for all that their absolute utility in the real world is fairly limited, especially when considering complex non linear tasks with huge latent spaces that are unobservable.

Not true. Not even humans have that level of manipulation. We employ humans despite the lack of these features.

That means these baseline features are 100% fully negotiable. They're negotiable all the time because we employ humans that DON'T have these features. Thus LLMs don't need the features either.

Either way, we don't understand LLMs well enough to even predict whether future modifications will or will not have the features you claim. Such a hardline claim that it will never "fully be there" is illogical. Nobody predicted that transformers could lead to LLMs, nobody can predict what LLMs will lead to.

Humans absolutely do have these features. (Implemented in a slightly buggy way, yes, but that's not relevant in this context.)

It is a stochastic parrot.

The error is deriving human ideas of 'quality' from that.

The stochastic parrot might well be better at your job than you are.

if it's a stochastic parrot then you are too. You just have a better predictor.

Obviously what I and everyone means by stochastic parrot is that it's not intelligent. It's wrong. It is intelligent. At worst it's as intelligent as a mentally retarded/schizophrenic or insane human. But even a mentally handicapped human still displays a level of intelligence.

Reply on news.ycombinator.com