Hacker News Reader

Comment by VerminOctopus1 | Hacker News Reader

Not to take away from this or its usefulness (not my intent), but it is wild to me how many pieces of software of this type are being developed. We’re seeing endless waves of specialized wrappers around LLM API calls. There’s very little innovation happening beyond specializing around particular niches and invoking LLMs in slightly different ways with carefully directed context and prompts.

View on HN

I see it a bit differently - LLMs are an incredible innovation but it’s hard to do anything useful with them without the right wrapper.

A good wrapper has deep domain knowledge baked into it, combined with automation and expert use of the LLM.

It maybe isn’t super innovative but it’s a bit of an art form and unlocks the utility of the underlying LLM

Exactly.

To present a potential usecase: there's a ridiculous and massive backlog in the Indian judicial system. LLMs can be let loose on the entire workflow: triage cases (simple, complicated, intractable, grouped by legal principles or parties), pull up related caselaw, provide recommendations, throw more LLMs and more reasoning at unclear problems. Now you can't do this with just a desktop and chatgpt, you need a systemic pipeline of LLM-driven workflows, but doing that unlocks potentially billions of dollars of value that is otherwise elusive.

>pull up related caselaw

Or just make some up...

At the token layer an LLM can make things up, but not as part of a structured pipeline that validates an invariant that all suggestions are valid entities in the database.

Can google search hallucinate webpages?

>doing that unlocks potentially billions of dollars of value that is otherwise elusive

What's more, it unlocks potentially new additions to the 206 legal cases where generative AI produced hallucinated (fake) content.

https://www.damiencharlotin.com/hallucinations/

How is something that cant admit it doesnt know, and hallucinates a good innovation?

Modern LLMs frequently do state that they "don't know", for what it's worth. Like everything, it highly depends on the question.

The application of a new technology to new fields always looks like this. SQL databases become widespread, there's a wave of specialized software development for business practices. The internet becomes widespread, and there's a wave of SaaS solving specialized use cases.

We are going to see the same for anything that Claude or similar can't handle out of the box.

Think of it this way: before the internal combustion engine people used animal power, steam power, human power, wind power, etc to move cargo, passengers, and even specialized loads like water pumps for the fire brigade. Then with internal combustion they did those things faster and at greater scale. That wasn't innovating on the ICE itself, or solving new problems. But it was still useful. Of course they also eventually did innovate on the ICE, and they solved new problems with it(heavier than air flight, for example) but it took awhile.

I suspect it's jumping on the hype train. Especially since its from a big Uni. Funding in research is all about marketing and latching onto the right keywords (just like VC really) so the most successful researchers are those who can market themselves effectively. Whether this tool is actually any good is secondary to whether it achieves the real goal of getting future funding for it's author.

> We’re seeing endless waves of specialized wrappers around LLM API calls.

AFAIK, doing proper RAG is much, much more than this.

What's your technical background if you don't mind me asking?

I’m a software engineer in the biotech space. I haven’t worked with RAG though, maybe I’m underestimating the complexity.

I work at a top three lab. RAG is just Mumbai magic. Throwaway. Hi dang.

What is a top three lab?

We know they don't work at OpenAI or Anthropic, but beyond that have no information

By that argument every SaaS is a db wrapper

Reply on news.ycombinator.com