Comment by Animats | Hacker News Reader

Comment by Animats | Hacker News Reader

Parent Full thread

"OpenAI will have to continue to raise more money than any startup has ever raised in history, in perpetuity, to survive."

As of 2022, Uber had supposedly raised a total of $25.2B in funding over 32 rounds. So they may be ahead in the money-drain business.

"There is no way to justify the current level of investment, with local models being freely available."

The business problem OpenAI faces is that their systems aren't good enough that users can trust the results. Slightly crappier results are also available at much lower cost. Unless OpenAI can definitively fix the "hallucination" problem, and at least return "Don't know" when appropriate, this isn't going to work at OpenAI's price point.

Chatbots are about as good as low-end outsourced call centers. They can sort of help with programming. The systems that generate pictures can do some impressive things. LLMs produce better blithering than most bloggers. It's really impressive. But, absent a major theoretical breakthrough, that's not enough to fund OpenAI.

One big difference is Uber only has a few peers and became instrumental before the realities of market size caught up to them.

But every hyperscale data center owner is investing for themselves, major companies capable of the R&D are investing in their own science, and the market for LLMs is slow to materialize relative to the hype. That's money that might have gone to OpenAI if their situation was similar to Uber's.

> The business problem OpenAI faces is that their systems aren't good enough that users can trust the results.

It depends what you're doing with it. Answering arbitrary questions is very different from document summarization or from mapping arbitrary questions onto a list of canned questions to then answer.

It also doesn't have to be perfect. Instead the cost of an overall system (AI + error reporting & handling) just has to be cheaper than that same overall system using whatever the AI is replacing.

If it makes errors 3% of the time, and people from Mechanical Turk make errors 2% of the time, its usefulness depends on whether or not those 50% more errors cost more than the cost savings from paying a model provider rather than paying humans.

Humans hallucinate too though. A lot. A good tip for system design is to assume the human operator goes off the rails at some point and does something absurd and nonsensical.

Hallucinations don't need to be fixed in one go as much as improved progressively. There is some magic threshold where they'll stop being an issue for specific tasks, or alternatively they simply become more reliable than humans. The problem will die with a whimper.

I think you have a fundamental misunderstanding of the problem here if you equate human hallucinations with LLM hallucinations.

What's the misunderstanding? The hallucinations I've seen from AI tend to not be that dissimilar to when humans are guessing. They will guess that a function like that exists. The difference being that humans are capable of checking their work and see if this actually is true by referencing some existing source of truth.

An additional difference is that humans know what they dont know. LLMs will happily and confidently make up nonsense.

I'm not entirely cynical on the value of LLMs, but I've yet to see one say "I dont know", or "I'm not sure, but here's my best guess".

> I'm not entirely cynical on the value of LLMs, but I've yet to see one say "I dont know", or "I'm not sure, but here's my best guess".

I've used LLMs to build form autofilling based on unstructured documents. It correctly does not answer fields that it doesn't know, and does not try to guess anything. It has been pretty much error-free.

It's all about your prompting. Without explicitly being given guidance on how not to answer, you're right, they will never say they don't know.

If you've never actually seen that happen, I encourage you to experiment more with LLMs; there's lots that can be achieved with the right prompting.

>A good tip for system design is to assume the human operator goes off the rails at some point and does something absurd and nonsensical.

And your assertion is that when this happens, it's because the human operator is hallucinating?

No. It's that humans will sometimes do the same thing that we've decided to (mis-)label as hallucinating when an AI does it.

I just saw an announcement that Musk has launched the largest GPU compute cluster in the world in Memphis TN. How did he fund this?

I wonder at what point Grok might become a serious competitor to OpenAI and Anthropic?

IIRC A proof was just published for the inevitability of hallucinations in the kinds of statistical models we use.

Reply on news.ycombinator.com