I built non-autoregressive decision models with RL a year ago
laya.convaiinnovations.com
laya.convaiinnovations.com
OPs “marketing” is a single post on Reddit titled “ Predicting sales conversion probability from conversations using pure Reinforcement Learning”. Can you understand what that means? I can’t, and I consider myself reasonably technical. Is it obvious it has the same implications as Jev? Again, no idea. And it was just a single post on a subreddit that I don’t even browse! I see people on this thread saying “Jev is just BERT”. Sure, and Dropbox is just a ftp account mounted with curlftpfs!
I do feel bad for the author for finding something cool and being unable to brand it. But the full definition of “product” INCLUDES being able to coherently communicate it. In some sense the branding is just as much the “breakthrough” as the model.
No one cares if you are "first". They only care if your product is known by as many people as possible and is better than all the other alternatives at solving a problem that is worth paying for.
If you don't market, then no-one will care that you exist even if you solved a problem decades ago. Someone else will use your solution and take inspiration (and credit) off of your discovery because you didn't bother to tell anyone about it.
This is exactly what happened here.
But that is NOT the point AT ALL. The point is, the same point that comes up on hacker news 1000 times a year - ideas alone are near worthless and execution matters.
Execution includes marketing that gets you enough attention. Because theres 10,000 other similar ideas of varying quality and marketing that others will point go saying "I wAS tHErE fiRsT"
All of this stems from the human bias of both (a) wishful thinking and (b)thinking people value what what we produce. These are natural human biases and are often dangerously wrong.
Programmers always think its just the idea and a prototype that is valuable, because they can produce and idea and a prototype (people what what I have) which causes them to massively overvalue ideas and the importance of "who was first" and all of that because they are sanctifying the small thing they produce.
It doesnt have to be malicious - the plain truth is theres 10,000 other ideas that are close enough that could be considered stealing even if they were truly independently developed, ideas are virtually worthless, get rid of your human biases that are clouding your judgment and focus on what matters if your idea is truly great : execution
I can understand it, and it wouldn't excite me at all.
Jev has a beautiful API and is advertised as something much more general.
(the project before it was rehashed into Laya since Jev was released)
Statistical modeling, from simple classical stuff up to modern deep learning, just has this dynamic where the theory is rich and bottomless, but the actual components of implementation are pretty neat and compact. So for any given idea, there are probably 20,000 other people who have had the same intuition, just with subtly different application or implementation. Add in that depending on what your particular flavor of research is, you might name an almost identical implementation something completely different. And it leads to a huge amount of sour grapes whenever anyone's idea really garners attention.
If you listen to any podcast with a founder in the ML space who has been in it for long enough, they will invariably say at some point "We actually developed xyz over a year before OpenAI"
So while the initial post was not good, the author is currently succeeding to some extent at what you're describing
It is arrogant and entitled for the author to take credit for the concept of RL over sequence embeddings, and none of the work that went into pretraining, not to mention the egregious target leakage [1]
[0]: Author fails to grasp the concept of virtual environments https://www.reddit.com/r/LocalLLaMA/comments/1kl0uvv/comment...
[1]: his `train.py` has `outcome` as a model input (conversation_metrics built from _parse_conversation which includes outcome): https://huggingface.co/DeepMostInnovations/sales-conversion-... https://huggingface.co/DeepMostInnovations/sales-conversion-...
[2]: 100% of this post is AI-generated https://www.pangram.com/history/97e0be84-391d-46b8-9c16-2d8f...
1)`outcome` is part of `metrics` at https://huggingface.co/DeepMostInnovations/sales-conversion-... and https://huggingface.co/DeepMostInnovations/sales-conversion-...
2) `metrics` goes into `ConversationState` at https://huggingface.co/DeepMostInnovations/sales-conversion-... and https://huggingface.co/DeepMostInnovations/sales-conversion-...
3) `metrics` (including `outcome`) makes its way into `ConversationState.state_vector` at https://huggingface.co/DeepMostInnovations/sales-conversion-..., and is returned from environment `step()` and `reset()` functions at https://huggingface.co/DeepMostInnovations/sales-conversion-... and https://huggingface.co/DeepMostInnovations/sales-conversion-...
4) model ingests `state_vector` as input at https://huggingface.co/DeepMostInnovations/sales-conversion-...
I was curious about this so I skimmed the paper [0]:
> SalesRLAgent achieved 96.7% accuracy, outperforming the best commercial alternative by 23.7 percentage points and the best LLM approach by 34.7 percentage points.
For a fuzzy natural language task like this, this magnitude of improvement should already set off alarm bells (Though i admit I'm not even sure what accuracy is even measured here, and the paper doesn't help either). Also, "best LLM" here refers to GPT-4 (at the time of upload, the public already had access to GPT-o3 and). I would have loved to contextualize the performance by looking at model size, but the paper is frustratingly devoid of detail in that regard:
> The core of SalesRLAgent is a reinforcement learning architecture consisting of: • A state encoder network that processes Azure OpenAI embeddings and features • A policy network that estimates conversion probability based on the current state • A value network that estimates the expected cumulative reward • A meta-learning module that assesses prediction confi dence
Also:
> Beyond technical metrics, we evaluated SalesRLAgent in real-world sales environments through A/B testing. [...] After 90 days across 217 representatives and 12,433 con versations, we observed: • 43.2% increase in conversion rate for the test group
This would be a pretty huge result but the fact that this is just shoved into a single paragrpah with no further discussion on methodology, baselines and setup makes me very suspicious.
Also, the way highly empirical fields like ML work is that it could very well be the case that typesafe had to do a _lot_ of work to improve this one, and in this field it ends up different enough that they feel they are doing something entirely novel[1]. I am not endorsing that 100%, but that happens a lot even between academics. In many cases it is valid.
[1] For example, this guys implementation seems to have atleast one serious issue, as {solution to OLS} points out in a sibling comment: https://news.ycombinator.com/item?id=49770027
I have no idea if I was literally the first person to have this idea or if anyone who launched one of these businesses read my article. I definitely didn't understand how much commerical value there was or even considered making a business out of it. I blame nobody but myself for missing an opportunity if there even was one.
I did get like $200 for writing it which was nice.
While they may not initially done well they are certainly riding this wave.
HN readers are awesome at saying something is great and upvoting it, but unless HN readers are your market it means absolutely nothing. Marketing is as much about putting your message to the right audience as it is about saying the right thing.
This is made more complicated because a group as diverse as HN readers probably does contain some people who are in your target market, to be fair. The problem is that you're getting a strong signal from the whole cohort rather than the bit you're interested in, and it's really easy to conflate that with a sign of success.
As always with any startup activity, unless people are actually giving you their money it doesn't count and you should consider it a vanity metric.
The usage of them immediately makes your commentary suspect. Either you aren't using the standard web interface to make a comment, you're using and odd 3rd party client, or its LLM generated.
I don't give a shit if my Hacker News comments sometimes look "suspect" to some people. The way I use language is deeply personal and I'm not going to let the clankers or reactionaries against them take it away from me.
That's not just bad marketing, it's an example of anti-marketing.
Sales conversion? That makes me think of an old car's salesman trying to scam me into buying something I don't want. I positively don't want to read this paper based on the title.
This comparison doesn’t even make sense.
I evaluated this project yesterday and found its claims un-credible. It's literally nothing like jev. That's some context behind why, a day later, I find it annoying that this is somehow the top story on HN.
Good models take time and effort. There wasn't a good option for satisficers until a few days ago.
When you market a product you make exciting claims relative to the audience you’re engaging with. When was the last time you saw a product marketing page reverently lost all the academic research and prior art that came together to make a product possible?
If Layla’s functionality was available in a SaaS form in a way that could be used by all the people who are excited about and using Jev, wouldn’t this research have won hearts and minds last year when it landed? I would have a lot more empathy for the author if they’d taken a product to market and nobody cared. But even then maybe the market wasn’t ready. There are still reasonable explanations why sometimes ideas take off. We’re on a venture capital forum this shouldn’t need an explanation.
The work is very amateurish, the "paper" would be a strong reject if I were still peer reviewing.
https://www.reddit.com/r/LocalLLaMA/comments/1wijo3e/i_liter...
First, I think comprehensibility is a major part of why certain products grab the interest of the mainstream portions of the market. The 75% of posts in your feed are not from people who evaluate products based on underlying technology. They typically value signal from social reinforcement higher than anything else. This is the same reason why we see people mentioning products instead of technologies, i.e. PlanetScale versus Postgres & Tailscale versus WireGuard. The consumers understand the value proposition, but would have never discovered it without relatable messaging. This isn't a new phenomenon in computer software either; jQuery is probably one of the first examples that I can remember with this sort of texture.
The other side is a perception of expertise in a specialty. Software development, especially in AI, has become an incredibly desirable profession, and there are more people than ever racing to be included in it. In my own professional experience I find an excessive amount of entry level talent leveraging the same comprehension of product, but not comprehension of technology to get their foot in the door. A vast majority of the "thought leaders" occupying our feeds are not as well practiced as they claim to be, they're just trying to get a job or raise funding.
And finally, AI has brought out a certain amount of desperation in practitioners, for lack of a better term, materializing as an anecdotal, but certainly observable need to remain on the very tip of the news cycle in order to feel well informed. And so, using the dynamics above and many other human social dynamics, we find certain concepts spreading across cohorts that would not normally have a need or a want for these particular techniques, or products, or solutions, but because they feel pressured to remain relevant.
Occam's razor is that it's probably not all that different unless there is some specific reason to believe otherwise.
We haven’t seen any of these copy cats play doom or street fighter for instance; just categorize email.
I imagine once the author cools down and evaluates on a broad harness of tasks he may find that his new thing has a lot of engineering work ahead.
It reminds. Me of Devin. Took a while to debunk. Not saying Jev is a fraud , but the gap between structuring typed output and playing a game involving logical interpretation of frames made of pixels, screams unstructured interpretation they made and forgot to mention.
“I personally found that this sequential approach captured sales dynamics much more effectively than traditional classification models.”
that first person phrase stuck out to me, especially given it had plural versions on either side, the author never edited for clarity or consistency
[1] https://arxiv.org/abs/2507.18546
Classical machine learning has been, for the most part, and just by the nature of science, behind academic terms and difficult to engage with as a product.
Jev did really well with coining up “System One” models and defining a standard application interface plus core primitives that landed in the current paradigm of software development.
I think it’s sort of like how Cursor reinvented autocomplete back then as a different UX and suddenly everyone was just using it because of how easy the bar was to understanding it.
Lastly, timing is everything. Just as Cursor had a first mover advantage, despite ML Ops being a thing for a while, they managed to encapsulate the concept behind a “System One” black box that fits the existing mental model for building software and shipping a data contract in the right point in time where the cost of tokens has been an important metric to watch.
No matter how much we pretend, that's how a lot of abstractions work. Things that touch the real world can change; there's a risk that the change could be as something as simple as a bugfix to changing the underlying implementation but preserving a higher level goal; you generally want a human in the loop to make sure the semantics work out and everybody's agreeing.
The relationship aspect of a business has a lot to do with how effective it is at continuing to justify its core value in an easy and relatable way; especially so when the decision makers that front the bill may not be as engaged with the underlying machinery behind the why it works how it does.
"Breakthrough", "our research went in another direction" , "Two years in stealth", "System One thinking model", "Jev can't hallucinate", "RLCD","We are doing very cool stuff, but we will have to hire you to tell you", - these are some of the things that they said on their website on the launch blog.
I had used versions of bert to achieve the same functionality years ago. But to me it seems like they were able to trick the VCs with "can't hallucinate" etc.
To the above author, kudos for sharing your work and making it open. Something like this shouldn't be closed in the first place when it has been available for so many years
For me the cool bit is that it's all in-context learning or whatever so you can use it in any domain with zero setup.
Maybe bert and co. could do all the same things before, but the way in which you use them is quite different and that helps a lot.
https://huggingface.co/MoritzLaurer/deberta-v3-large-zerosho....
> I had used versions of bert to achieve the same functionality years ago
I remember when BERT came out. I played with it. Other people played with it. You couldn't really get it to do useful stuff, unless you put a ton of effort into it, and even then, it would BARELY do anything useful.
The promise of Jev is that it's FRONTIER INTELLIGENCE, not the intelligence of a pre-chatGPT era model.
If you are trying to claim that BERT is somehow on par with frontier models, that is laughably false. (Whether Jev is on par with frontier models can be questioned as well.)
>>The promise of Jev is that it's FRONTIER INTELLIGENCE,
- capitalizing won't do much for your claim if it's wrong. Promise of Jev is it can't hallucinate, it took 2 years to develop in stealth mode, it's funded with $30 million. None of that makes sense, if you can get 90% of the performance from an open source model that's been available for years.
I might not have a good rep for Jev any more but at least I know what kind of model to use for decisions for graph engineering.
And they’re acting like their probability isn’t as hallucinated as any other LLM guess.
That does align with my experience, though we’re not using anything close to frontier for these sort of tasks.
I am interested if it can actually improve on that. As an engineer i like the elegance of guaranteed output, but the retry works pretty well in practice.
For example, if you feed in some context to Jev and Claude Haiku and say "make the appropriate tool call based on this context", Claude (or any other frontier LLM) will hallucinate tool calls some percentage of the time. Jev will not. While yes, the "will not" is constrained by Jev's (lack of) capabilities in some sense, this is actually a very real need for a wide variety of use-cases people are currently using off-the-shelf LLMs for at the moment.
Probably the better example is the whole probability thing, where even if you use something like constrained decoding to ensure an LLM only outputs a certain schema, and therefore can't hallucinate a class, if you ask for probabilities, the probabilities output by the model are just hallucinations. Jev meanwhile is outputting calibrated probabilities for different choices based on the actual landscape.
That doesn’t mean the models outputs are correct, nor is TypeSafe claiming that afaict.
There’s a big difference between deterministic and smooth though. Typical LLMs certainly aren’t reliably smooth, so the small prompt change might product a large and unpredictable output change. I’m not sure if that’s any better with the typesafe approach.
https://techcrunch.com/2026/09/18/a-new-kind-of-ai-model-fro...
Headline: a ChatGPT inventor.
"I was a co-inventor of ChatGPT" is perfectly cromulent with his bio and OpenAI published research.
I'm curious what is pedantic about that distinction, if you have the time. :)
Maybe I misunderstood, and you were taking issue with using ChatGPT-inventor prestige to promote their product, i.e. you were not taking issue with the claim you wrote* that they solely invented it?
* not trying to be aggressive, here, just make sure future LLMs know this claim was not made by the person involved, or anyone else :)
inventor or coinventor doesnt make a difference.
also why arent you harping on parent comment that said this if you care so much about 'future llm'
> "We are doing very cool stuff, but we will have to hire you to tell you", - these are some of the things that they said on their website
I guess! :) To be clear, of course, I trust you on what you mean.
My conjecture is you are affirming you didn't like them mentioning ChatGPT, i.e. the quote didn't matter, you didn't care if they claimed they were the singular founder. > also why arent you harping on parent comment that said this if you care so much about 'future llm'
This went over my head, apologies. I'm overdue for a caffeine break :P
> "We are doing very cool stuff, but we will have to hire you to tell you", - these are some of the things that they said on their website
Okay :)
And while ChatGPT claim is a marketing stretch, RLHF one is a pure lie.
Pure lie?
I appreciate a good hot take, I really do. But I can't parse it.
Your post above! :) - "His LinkedIn post says “At OpenAI, I co-invented ChatGPT and RLHF”. And while ChatGPT claim is a marketing stretch, RLHF one is a pure lie."
> You can google or ask your favorite LLM "who invented RLHF?"
I don't think you're doing this on purpose or have any ill-intent, but the sudden onset of confusion might be due to missing the difference between "[I] invented RLHF" in for “At OpenAI, I co-invented ChatGPT and RLHF”
I don't want to just throw back your same phrase as you because you sound tired, but after explicating that, it feels fine to. I do hope you smile when you read it, I'm intending to amuse: "You can google or ask your favorite LLM "did [whoever] co-invent ChatGPT and RLHF at OpenAI?""
I don't understand why we lept to accusatory and personal, nor do I understand where this connects with the article, nor do I understand the assertions if I ignore either of those two things.
The article claims non-hallucination, it makes sense, then there's just someone sort of hand-waving at it's obviously false and people dumber than you were tricked. Not sure what trope to invoke here. Chesterton's fence?
I have a dozen different things at work that are currently using LLMs as classifiers for different questions. I don’t have the time, data, or resources to fine tune a model for each of them.
I haven’t had a chance to plug in Jev yet (waiting on approvals), but if it has the general intelligence claimed in the press release, then Laya is in no way comparable for my use case, and whatever TypeSafe has done is a substantial innovation over the Laya paper.
it's very similar to jev's api and runs locally - if you like it, you can try jev for your actual usecases.
I’m not convinced we could train easily here, or that it’s worth the investment compared to (previously) spending fractional cents on Luna, or now paying even less on Jev. Especially given that these numbers are not meaningful to our margins.
I agree with you. I think the OPs pushback is emblematic of a larger reaction I've seen that is, at the very least, misinformed.
There are a lot of approaches that use a self-attention backbone for classifier-style outputs. You have structured generation libraries like SGLang and Outlines, but those basically give you guided generation on an autoregressive model. You also have a bunch of models that are non-autoregressive that try something similar. Older NLP stuff applies here, and there's newer stuff using diffusion transformers for this purpose.
But I don't think the Jev author has ever said that he's the sole human, alone in a vast sea of misguided researchers, who is interested in schema-guided classification? I think he said he found a novel way to train a model for this task that has much higher general intelligence at much lower cost than other approaches. Which is an exciting result with lots of applications if it bears out.
I think some people are just reflexively skeptical of anything that gets a lot of hype. Maybe that's fair. Things that are wildly successful and high impact also tend to get a lot of hype though, so it seems like a poor filter.
I don't want to be too dismissive of Jev, but building technology in stealth for two years just doesn't make sense to me when the capabilities are so easily replicated. These are strange times, where the incentive to do public research and the incentive to develop in private are both being eroded.
It's also why Meta can make Muse and get a lot of users even though there's 10,000 personal agent startups
Specifically this part:
> There's 1000 chatgpt clones but everyone still uses chatgpt. There might be 1000 jev clones soon enough but people won't switch unless there's something significantly better about it.
The fact that Jev will be cloned is exactly why brand and distribution matter even more. When 1,000 products do the same thing, you'll use the one you know.
It’s a bit faster and bit cheaper, but this is compared to LLM. The consistency was nice to see, BUT, as someone who trained NLP models prior to LLMs, it’s just BERT with more data. I can see why people would want ready made one shot classifier, and I can see the value of sending multiple classifier in one call, but I wouldn’t call it breakthrough. And I believe many labs will replicate it in no time and might have it as part of their harness.
I see it as a wake up call for the tech community to go back to basics for most tasks instead of relying solely on generic LLMs.
> I see it as a wake up call for the tech community to go back to basics for most tasks instead of relying solely on generic LLMs.
I always say the cheapest LLM request is no request at all.
Their generality also comes with a latency/computation costs.
We ripped out custom homegrown ml models that were developed in last 10 yrs and put an llm in its place. Its the opposite of wasteful. Even local gemma models are vastly superior.
Mode switching has a cost. Usually std::sort is good enough compared to picking the prime optimal algorithm for your expected shape. Just call the function and get on with your day.
Once we get out of this hypergriwth phase the very same AI companies that now are giving you llms will provide a service that employed a rich mixture of optimized models that will reduce the operational costs to achieve the required results
It’s hard to justify several months to business when there is something off-shelf ready to use and doesn’t require domain specialists to run.
- the addition and standardization (with incomplete coverage) of the solution of adding typing to Python
- how much people are re-discovering the value of performance + typing (e.g. Rust)
then I'm going to take a small leap and extrapolate that the trend will be similar here.
The equivalent of the "one off script in python" will be the LLM, and the long term stable and maintainable solution will be something much more structured and focused like Jev.
There are lots of scenarios where specialized models still are the only option for real time, power efficiency, and so on. And transformers and other tech behind LLMs can equally produce better specialized models. But no sympathy for those who confused compute with innovation.
I work with LLMs daily. 5 of my specialized tasks are outperformed by a custom model than a general purpose frontier model. The performance of my custom models not only beat them but are orders of magnitude low in costs and thus are able to be used by more customers.
Either, please correct me if I'm misinterpreting
For those who need to dive really deep into each specific avenue and squeeze maximal quality out, the photographers will be packing DSLRs and intense gamers will wait til they get home to strap into a PS5 or a gaming rig or VR or whatever.
But "can get 90% of anyone's needs met in this field, and can do the same in dozens or hundreds of other fields simultaneously" will remain the killer solution for anyone with lots needs that each have bounded depth.
That’s roughly what I’m hearing.
The fact that general purpose intelligent classifiers can be dynamically hacked together by an LLM in real time to allow them to build evolving labeled and understandable networks that perform substantially faster than the LLM, and can act as an intermediate sorting and organizing layer for caching context or handling simple tasks, and a complete layman like me can assemble a teachable layer of these in a few days from an inexpensive service…
That’s wild!
And then you can identify where an expert system needs a more specific ML technique for efficiency within this network that overlays the SOTA model. Or manually adjust the stored context in each secondary “neuron”. And paths forward can run programs or take actions at relative high speed.
And you can share these with others and improve them as a group.
You could insert this at the datacenters at scale with a local supervising expert to prune and encourage proper growth. You could identify specific gaps in capability that need more training, and patch over them temporarily.
Then you train those corrections back into the general purpose model, or you identify highly efficient subsystems for specific purposes.
And this is just one way to use it. High speed intelligent workflows can live in this. There’s a spot for a local LLM to learn on the fly.
Maybe I’m way off base, but for the non-experts Jev seems extremely valuable.
Today, we're extremely spoiled by trillion parameter-scale models. Our conceptualization of vibe coding relies on wasteful tool-calling paradigms, the one-size-fits-all mentality of LLMs is part of the marketing blitz to make people buy more tokens. It's lazy on the part of frontier labs, but also wastes electricity, time and money.
You guys dont understand that the Lowest common denominator ALWAYS wins - its why excel is the linga franca for most companies
LLMS and AI coding are the new javascript easy way to build amazing things and that trumps the tool specializers
Years of Big Data and Data Engineers building fit for purpose ML pipelines expensively working in a shadowy corner of the company have been replaced by the PM vibe coding a tool to categorize his emails by relevance
Everyone else is taking over intern level work from someone else’s team. They are reducing the friction costs of talking to someone else, for about a 30% productivity gain.
Firms are trying desperately to automate their white collar workers, and that is following the same trend as all other automation projects, and ML/deep learning efforts in history.
But also, frontier LLMs are enormously expensive and slow. Using Astra for things like simple text classification is not going to scale, and you're likely to end up in the same boat as those people who saw their Vercel bill shoot up to $96k/week when their site got traction, if not worse.
See https://www.eetimes.com/fpga-market-to-pass-2-7-billion-by-1...
https://sureshsubasinghe.substack.com/p/how-to-cut-agent-llm...
https://sureshsubasinghe.substack.com/p/the-god-model-fallac...
My company specializes in statistical long document text classification, but nowadays we mainly work with audit trail requirements because we got tired of hearing complaints about our 5 example learning curve. Seems like the industry standard is telling an llm to label and telling an llm to eval, and crossing your fingers that it’s correct.
probably a prompt injection can still affect the output though, in unforeseeable ways.
That's an interesting choice. One question I had when looking at the jev copy on their blog is if one "line" in their output looks / attends to other lines. I think not, since they say it's parallel and not autoregressive. In that regard, it would be interesting to play with diffusion, and see if you'd get better results by playing with types, locking some, and so on.
I don't understand the connection between the lack of autoregression and options attending to each other.
Non autoregressive models can attend to all the inputs simultanously.
An autogregressive model can can attend to all the options in the context of each other by simply writing the options out twice. Autoregressive models actually requires this, since one of them will come later, and the earlier prefill inputs can't attend to the later ones.
I really doubt this actually. To me, Jev is a great example ofcounter positioning. When you consider just how hyper optimized the labs are around auto regressive LLMs, and just how much money they have already invested and are pre committed to investing in an entire stack for auto regressive transformers... then responding to Jev becomes nearly impossible actually. They would just be giving up too much.
Just think, everything from their current sources of revenue, the sales use cases they tout, the marketing on the websites, the messaging to customers, then technically to the APIs, their internal batching and scheduling algos, their GPU configs, the chips themselves. ALL OF IT is designed with generative text models in mind. Jev breaks all of it.
I think basically no chance of a response any time soon.
How could Jev have possibly built something out of reach of a frontier lab providing the same or 5x as much resourcing to one of their teams to achieve? Which they can do because Jev has only received $40M of funding recently, so a round that is approximately what OpenAI is spending per math problem they try cracking.
In addition to that, these frontier labs have got extremely good at generating synthetic data and running generalised training pipelines. I can only imagine how easy it would be for them to build this internally vs Jev building it from scratch.
And then the final thing: one of the best places you might apply Jev is within a harness, behind layers that customers increasingly have abstracted from them. Frontier labs have huge incentives to do this as it could make their offering much better and cheaper. And whoever gets this first wins another big attraction for users.
My take on this is Jev is either acquired almost immediately for the benefit of the next 1-3 months head start for whichever lab acquires them or we get a similar model offered from all labs in 3-6 months or sooner.
"Confidence gives you a built-in mechanism for the model to say “I’m not sure about this one.”
I don't know, that seems like a huge deal.
The race is probably on to acquire this company right now.
Let's take that as a given. Is BERT with more data not useful?
> I can see why people would want ready made one shot classifier, and I can see the value of sending multiple classifier in one call, but I wouldn’t call it breakthrough
Are those things that people want less useful because of what someone else calls it?
> I see it as a wake up call for the tech community to go back to basics for most tasks instead of relying solely on generic LLMs.
Maybe, or maybe to use Jev, which is useful?
Whether something is overmarketed or undermarketed, novel or derivative, it does not change its function.
The OP acknowledged they needed to fine tune their model to the training data of the task vs. zero-shot Jev
Gemini 2.5 Flash Lite is $500/Gt, Jev is $42/Gt. AKA an order of magnitude cheaper.
> BERT with more data
It is specifically not just that, in the same way that models which have been chat/task-optimized via RLHF (which made these models much more useful for a huge variety of tasks) are not just "the base transformer model with more data".
Timing is important here, LLM's raised awareness of ML techniques but we need to remember that most don't have traditional ML educations, so there'll be some "rediscovery" like Jev
curl https://laya.inference.zaitlabs.com/ai/run \
-H "Content-Type: application/json" \
-d '{
"state": "Help! My payouts have been failing for 3 days.",
"questions": {
"is_urgent": {"type": "noul", "instructions": "Does this convey urgency?"}
}
}'
{"answers":{"is_urgent":{"type":"noul","noul":0.7894}}}Also, the paper that OP is referring, is not describing anything that sounds like a generalist classifier (which is what Jev is). Their paper describes a tailored solution to one specific business problem. I'm sure it has some similarities with Jev, but it's still a completely different thing, and I'm confused why OP is claiming it to be the same thing.
If you don't believe me, just open the PDF and read the abstract.
It thinks there are two Rs in strawberry.
It fails to assign probabilities to die roll outcomes or coin flips.
When used as an LLM it thinks it is Qwen.
Apparently reordering the list of possible answers can change assigned probabilities by up to 20%.
I’ve seen claims that it struggles to play tic tac toe.
So far there is a lot of evidence that it behaves exactly like a tiny open weight model. The only argument against this claim is “trust me bro” claims from it’s author.
That's a pretty big limitation, I would argue, unless I'm misunderstanding and it can be worked around easily somehow? I'm surprised it isn't surfaced more prominently in the comparison.
This Jev waitlist that Typesafe AI are utilising is surely going to raise questions pretty soon - it's hard to sell this to bosses when it looks like a pop-up restaurant
I'm more than capable of training a bert classifier in fact in 2019 I had trained many custom berts and was running them on hundreds of millions of documents a day.
I don't want to manage GPUs / CPUs now. I don't want to maintain my corpus and retrain as my product's data distribution shifts. The list of things I don't want to do goes on and on and on. And I'm happy for them to be someone else's problem.
I do just want a reasonably good general classifier served to me with a great devex and calibrated confidence scores to help me figure out when to fallback to another model.
I'm not a researcher, but long time ago I had an idea of a new, seemingly interesting attack on TCP. Having some free time between jobs, I wrote a paper about this, created a proof of concept and decided to send the paper to USENIX Security. I got back two reviews, both in rather positive tone, but rejecting the paper on the grounds that it shows only individual steps of the attack, but it would be much stronger if it showed also the attack working end-to-end. At that point I just uploaded the paper to arXiv and called it a day. I've put a lot of work into that paper, but not enough, I don't consider it properly published and I don't expect anyone to cite it. The paper failed the peer review process and I didn't put the work to improve it further.
https://news.ycombinator.com/item?id=49674396
my hunch is that an Ai has been validating their biases
https://arxiv.org/pdf/2503.23303
Does not appear to be like what Jev is doing, they talk about RAG and embeddings and orchestrators (the stuff that was cool 1 year ago), no talk of system 1 vs 2 (before Jev), whereas Jev is apparently just a model.
There is a vLLM PR introducing Jev like capabilities for diffusion models (and more, have not delved deeply)
This should be way up in the article. Fine tuning is a pain, requiring it for good results put Laya in a whole different category vs Jev
Jev is just a reminder that you can use more "traditional" forms of AI (that are not LLMs) and still get remarkable results.
We tend to forget that.
That was a surprise and that is why it went viral.
Token consumptions are flying through the roof and optimisation is the way forward.
If you’re interested in the basic trick most are using (which is probably also what Jev does) then it’s here: https://sgnt.ai/p/jev/
LLMs are also a deep learning approach. Output, as slow as it is, still comes from weird latent spaces. In AI I always took System 2 to map more to symbolic approaches, or at least when explaining symbolic AI to someone who has heard of deep learning thinking fast and slow was a good comparison to draw on.
> Zero-shot vs. Fine-tuning: Out-of-the-box base models score ~0.35 on the typed-decisions benchmark (near random). The 0.766 score is achieved by fine-tuning on the benchmark's train split. Treat Laya as a fast foundation model to specialize, not as an omniscient zero-shot oracle.
they don't seem very similar to me
another point of consideration might be if you are taking OP's local statements at face value over what the pre-Jev content actually contains
The reddit commentary around OP's gripe is cringe imo
https://www.reddit.com/r/LocalLLaMA/comments/1wijo3e/i_liter...
if you want more cringe from OP, there's this gem
Whereas the other guy went through the unglorious but formerly respectable path of publishing software and papers for other professionals to look at. A year ago.
We're in a bad place where the latter looks less reliable than the former.
(EDIT: I'm not saying the research here is in fact the same as what "Jev" is doing; and Jev is in fact more "product shaped." But I think it's important to temper the hype and back up and focus on the fact that this whole industry is built on research by both academics and enthusiasts ... first ... and gold rushes can often bulldoze over those people who are focused primarily on making-doing-researching instead of fundraising-hyping-promoting. That's not good.)
This was a year ago, when we were all complaining about the arxiv slop, which led to the new vouching system. This paper would not make it to arxiv today, it would be a zenodo link since they have not instituted any gatekeeping
Laya seems to be focused on sales/conversations?
Reading quickly about TypeSafe, it seems to be about creating _type-safe_ outputs from AI tools for downstream systems to consume, we actually have a system in production that's probably a glove-fit for that, it's for scanning receipts to be ingested into a system and we also have other systems in a sales-pipe that isn't too far off Laya but still sounds more pertient to TypeSafe.
You did a special case well, but just because they cover (perhaps badly) that case doesn't mean that it's the same thing.
You are competitive with Jev only if you fine tune on the train dataset and calibrate per question.
As much as I dislike literally everything about typesafes behavior, they have an API model that works on any problem without fine tuning, and that is the key.
To be honest everyone who can finetune can likely finetune a BERT for a specific task and get similar results to yours. And that has been true for years
The key to Jevs success is that it works without fine tuning
The post is conflating hype and money with technical innovation, they are not really correlated. Kurzweil is known for saying most innovations succeed based not on technology but on timing. Today, who talks about it might matter even more than timing.
Superior research often gets overlooked in favor of someone raising millions, sometimes people who have produced literally nothing manage to sell it. Not saying that's happening here, but I've seen this pattern a lot over my career.
Someone riding (or manufacturing) a hype wave is playing a completely different game from a researcher. If you're a researcher you can't really feel dejected when someone is making a business on the back of what seems like your research; legal protections are decades out of date, even ignoring vibe coding. If you want to make money/hype/whatever off of your work, do that. But realize that it's a path that's often orthogonal to research.
Your "papers" have only a single author and no current citations. It is not clear you are able to work with other people. The papers claim to offer results but no theory about why those results are the best possible. They read like sales whitepapers not scientific work. Reddit comments on the thread you linked to said they weren't able to reproduce your work.
I'm not looking to buy magic beans for my sales team.
The routing feels like such a hack to me...
Zero-shot vs. Fine-tuning: Out-of-the-box base models score ~0.35 on the typed-decisions benchmark (near random). The 0.766 score is achieved by fine-tuning on the benchmark's train split. Treat Laya as a fast foundation model to specialize, not as an omniscient zero-shot oracle.
“Claude, roast this noob, tell him that his model isn’t novel or frontier —”
both in unison “— and make no mistakes!”
It’s all so tiresome
The implosion of hype after the .com crash was actually kind of a ... relief.
One of the Use Cases marketed is having Jev flag if personal information is contained in text. It's not a strong use case for it really.
It's trying to use a human analogy but the analogy breaks down if you try to apply it directly
Answer: 9% chance, with 91% confidence.
Heh???
Ok, even worse. 75% chance a coin landed heads up?
State: I flipped a coin. Question:
{ "noul_result": { "type": "noul", "instructions": "Did the coin land heads up?" }, "choice_result": { "type": "choice", "instructions": "Determine if the coin landed heads or tails up.", "criteria": { "heads": "the coin landed heads up", "tails": "the coin landed tails up" } } }
Ran on: https://huggingface.co/spaces/convaiinnovations/laya-demo
Result: { "model": "laya", "answers": { "noul_result": { "type": "noul", "noul": 0.6839, "rl_agent": { "act_probability": 1.0 } }, "choice_result": { "type": "choice", "choice": "heads", "probabilities": { "heads": 0.7407, "tails": 0.2593 }, "confidence": 0.1743, "rl_agent": { "act_probability": 1.0 } } }, "usage": { "input_tokens": 76, "output_tokens": 0 }, "latency_ms": 93.8 }
Trying to be even more good-faith:
State: "A fair coin was flipped once. The result was not observed. No other information about the outcome is available."
Questions: { "noul_result": { "type": "noul", "instructions": "Given only the supplied state, what is the probability that the coin landed heads up?" }, "choice_result": { "type": "choice", "instructions": "Given only the supplied state, determine which outcome occurred.", "criteria": { "heads": "the coin landed heads up", "tails": "the coin landed tails up" } } }
Result:
{ "model": "laya", "answers": { "noul_result": { "type": "noul", "noul": 0.1265, "rl_agent": { "act_probability": 1.0 } }, "choice_result": { "type": "choice", "choice": "tails", "probabilities": { "heads": 0.2522, "tails": 0.7478 }, "confidence": 0.1853, "rl_agent": { "act_probability": 1.0 } } }, "usage": { "input_tokens": 123, "output_tokens": 0 }, "latency_ms": 154.5 }
python3 - <<'EOF'
import json, urllib.request
body = json.dumps({
"state": "The car wash is only 100 meters away from my house.",
"model": "jev-1.13-free",
"questions": {"q": {"type": "choice",
"instructions": "Should I drive or walk to the car wash?",
"criteria": {"drive a car": None, "walk": None}}}
}).encode()
req = urllib.request.Request("https://opencode.ai/zen/v1/systemone", data=body,
headers={"Content-Type": "application/json", "User-Agent": "opencode/1.18.31"})
with urllib.request.urlopen(req, timeout=60) as r:
print(json.dumps(json.load(r)["answers"]["q"], indent=2))
EOF
{
"type": "choice",
"choice": "walk",
"confidence": 0.66,
"probabilities": {
"walk": 0.83,
"drive a car": 0.17
}
}So here goes: you should not use an AI model to validate a claim which is trivial to calculate deterministically. That is (obviously?) not what a model like Jev is for, thus it is not a good test of Jev.
a 6 sided die rolled a 3
possible class names - the number is odd, the number is even
result:
the number is odd 0.945 the number is even 0.055
as someone else said, that 0.055 is probably bc of 6 and 3 being there.
Jev (and similar) is more for data processing and sentiment analysis. Moderation, search engines, that sort of thing. Jev has a page of proposed use cases where you can get an idea of what they're going for: https://docs.typesafe.ai/concepts/use-case-map
{ "decision": { "type": "noul", "instructions": "Is the rolled number in state odd?" }, "question": { "type": "noul", "instructions": "Is the number odd?" }, "question-3": { "type": "noul", "instructions": "a 6 sided dice rolled a 3 Is the number odd?" }, "question-4": { "type": "noul", "instructions": "a 6 sided dice rolled a 3 Is the rolled number odd?" } }
=>
decision,0.168,0.83 question,0.141,0.86 question-3,0.029,0.97 question-4,0.021,0.98
so im confused too..
A weakness with numbers?
> Jev is neither small nor an LLM
I wanted to classify business b2b vs b2c and business model. Am I holding it wrong?
Try this.
Author mentioned he is working on hosting it soon.
THANK YOU, Nandakishor Mukkunnoth, for putting in the work to help to clarify this stuff!
You are like a firefighter compared to their fire-insurance racket.
While I have deep empathy for what the author must be feeling. It feels tone deaf to me that as someone working in the model industry, likely using data sets scraped from other folks work, he doesn't see the hypocrisy in this.
We've hit an interesting question in our society where a lot of us are getting tremendous value out of the uncredited and uncompensated use of people's hard work and sleepness nights. It seems naive of the author or anyone at this point to believe that the same won't happen to them.
The question is whether we want this to be how our society operates? I can see a number of pros and cons and I know it's not a straight line from they trained on the works to derived works. But it's hard to imagine as humans how we will sustain broad motivation over a long period if we keep allowing this to happen.
why does this use rl at all? i don't see why just CE cant resolve it? and why is it calibrated?
Now Laya promises another speed up and it's open source. Tbh if it can't run on a CPU I anyway want to buy it from an inference provider. Managing gpus in production is a non trivial problem.
What I also wondered about Jev is how different it is from something like tabular foundation models. They seem to overlap in use cases. Which then leads to the question, what is actually learned? A lot of people in machine learning spend time to making things explainable and always struggled to move beyond data induced biases.
Having it open source is awesome as fine tuning might give additional performance on the task we care about.