LLMs are a lot more like a generalized processor than people are admitting right now. Granted you can talk to it, but it becomes significantly more capable when you learn how to program it -- and thats where the value will be added.
I don't know if you mean, like, LoRAs and similar (actual substantive changes), but the vast majority of "learning how to program" LLMs (accounting for the majority of startup pitches as well) is "prompt engineering" - which, as the meme goes, isn't a moat. There's a skill to it, yes, but if your singular advantage boils down to a few lines of English prose, your product isn't able to control a market - and VCs are (rightly) not interested unless you have the possibility to be a near-monopoly.
But no one would say that now, thats ridiculous. There is a sufficient degree of prompt engineering that is already defensible, I'm already doing it myself IMO. You'll see very sophisticated hybrid programming/prompting systems being developed in the next year that will prove out the case.
For example 30 parallel prompts that then amalgamate into a decision and an audit, with 10 simulation level prompts running chained afterwards to clean the output. These types of atomic configurations will become sufficiently complex to not be just for 'anybody'.
It's pretty effective on complex problems like Spam, Trust and Safety, etc. And the applications of these sort of reasoning atomic configurations I think are unlimited. It's not just 'talking fancy' to an AI, its building processes that systematically improve reasoning to different very hard applied problems.
But overall, hasn't that theme been true for like... all tech ever? You have to set up and build your own innovation path at some point.
They are limited to applications in which the latency slo is O(seconds), knowledge of 2021-present doesn’t matter, and you’re allowed to make things up when you don’t know the answer.
There are, to be fair, many such applications. But it’s not unlimited.
But in general many configurations are possible, but also you need to refine the personas a great deal to get it to work well.
Sure, so you ensemble some results. You're back to the classical "hyperparameter" problem though that's faced ML for a long time-- what those personas are, what those subsequent prompts are, etc. require a fair amount of manual verification and tuning. And the search space is extremely vast.
Not to mention that something like this is likely to be very unperformant.
So... prompt engineering? They're an extremely inefficient processor though and very prone to error (despite what synthetic benchmarks may show).
I think this is very much like the CCD sensor, that Kodak couldn't envision using because it was "so expensive, slow and low resolution."