This is a bit of an outdated take as of two days ago. Dear lord things move fast these last few years. Some of this is still relevant. Fine tuning Jev once available could address certain concerns.
(Very excited as I got an invite email for TypeSafe today! I don’t have time for all the little experiments I want to run with Jev and Astra combined!)
I see a bunch of people saying that there were already similar solutions in this space. Maybe true but it definitely feels like the missing primitive for working with llms. Can immediately see how it can be deployed in real applications in a way that the autoregressive chat approach can’t.
Also, I think that’s what you’d like the article to be about. Not that you’re unfounded, I’m sure others are thinking the same.
But to further dissect that, having any article be just about the cons and not including pros is in its nature, reductive. There certainly are pros (I have no stake in this race for the record) and model use choice should be seen as situational. To seek a silver bullet is itself reductive and thus, something we should avoid as it’s ultimately a fools errand imo.
Anyways, this article is plenty good. Thanks for the writeup OP.