Hacker News Reader

Comment by lewisjoe | Hacker News Reader

Can someone ELI5 me why Jev class models matter?

Note: Not the technical side, but as an end user of LLM APIs.

Basically compared to standard LLM models it is an order of magnitude cheaper and faster. (Note: I didn't get to actually try jev yet, just looked at demos/specs/pricing etc)

You can definitely do similar things with say hosted LLMs + a lib like outlines , or with API models and the proper output validation layer, but again way slower and more expensive.

And on the opposite side you can train dedicated classification models that will be even cheaper and faster to run than jev. But, well, you need to train them (costly, time consuming, and data might be hard to come by depending on target). Here you are a nice zero-short system, that can handle complex/messy data out of the box.

Thanks. How reliable is the world knowledge? Say, I use it as a delegation router for picking the best model for a task - how can I be confident it's doing that with enough intelligence?

When I use traditional LLMs I get some sense from the flagship-ness and regular usage. How do we get such confidence for Jev like models?

The Jev alternative, laya is only 334m parameters and you can post train it as the workflow is open source. The downside is that laya is only 1024 tokens, and Jev is ~32,000 tokens. I think for many tasks 32k tokens might be too small. You could probably design a task complexity router with something like qwen 0.8b with similar speed and a much larger context window (260k token). As always, "it depends". 0.8B is almost as fast as Jev without any of the limitations.

I was wondering about the same and found this video very helpful. It may not be truly ELI5, but close enough: https://www.youtube.com/watch?v=tTnUcSj-QPA

That helps, thanks.

Reply on news.ycombinator.com