Jev and System One Models: Calibration Beats Accuracy | Hacker News Reader

Jev and System One Models: Calibration Beats Accuracy

kartikpansuriya.com

The way he proposed to deal with "How urgent is this customer support message?" is "a score over ordered levels".

I'm not sure that's a good way to use Jev for this use case. If I had this problem, I would ask Jev to answer several different yes/no questions about each message, and then use the probabilities as inputs into a logistic regression model that predicts urgency.

If you ask the model specific yes/no questions which can be answered reasonably objectively from the input, I think the answers are going to be more stable over successive generations of models.

e.g. if you ask 'Is the customer angry?' I'd expect that answer to have high agreement between models and between models and humans. But directly answering the 'is it urgent' question is much harder. (Although I suppose you can try to put the rules in the prompt.)

It seems to me that Jev cannot be usefully calibrated out of the box, in a fairly strict sense. Suppose I have a pull request and the state is the title, description, etc. The question is “Will it be merged?” (it doesn’t really matter whether it’s a choice or a “noul” [0]).

Now consider that two different projects may have radically different criteria for accepting a PR. And the two projects may have different probabilities for acceptance of a random PR from the distribution of PRs they get (i.e. the overall fraction of PRs that are accepted). So what could Jev possibly return that is “calibrated” for both? It doesn’t even have an “I have no idea” option because the output schema cannot distinguish between “I am confident that there is a 50% probability that the answer is yet conditioned on the state” and “there is no useful information contained in the state that I can extract and therefore you should assume that you posterior distribution is the same as your prior”.

For fun, I gave Jev some irrelevant state and asked it various questions for which the state was useless (I picked sporting outcomes), and it was 0-for-3 at giving yes/no probabilities that were particularly close to the obviously correct no-information answers or close to 0.5 in cases where the prior was far from 0.5.

This is a silly test, but I’ve personally encountered genuine production situations where the best classifier available (or at least the best one available at any cost remotely close to what it was worth) was, drumroll please, a constant. But it was a calibrated constant: we measured it! And there is no way to feed this sort of information to Jev. (Yes, I tried it. Even literally stating the distribution in the state does not work well, although it does appear to have some effect on the outputs.)

[0] Is “noul” even a word? I know what a binary classifier is…

Let me state it plainly --- this is not just a badly written article. It's super annoying and content free.

Calibration has to be measured on your dataset. Just because the probs sum to 1, does not make Jev or Jev-like models claibrated. For folks interested in digging deeper into calibration, studying ad click prediction models (where calibration is super important) is a good place to start.

> “Zero hallucination.”

This is the part that bothers me the most. How is it 0 hallucination, if the correct answer is not even the part of the options. There is no way to mark absentia or a way to know I absolutely cannot choose any of the options.

I am wondering, if anyones tried dead simple combinations of embedding with logistic regression to solve classifications problems?

Can’t produce text ergo can’t hallucinate I think is the marketing claim

Of course you can generate text. You just need to run it in an autoregressive loop as a sort of reverse of all the fun Jev-like papers that have come out in the last couple days. Ask it to predict the next letter in a string, then sample at your favorite temperature, then predict the next letter, etc. This will be quite expensive, and it may work terribly. I’m not personally inclined to try it. I am, however, curious whether it would work less horribly if you correctly guess what tokenizer the input uses and request a choice over next tokens consistent with the tokenizer in question.

It would be absolutely hilarious if you did this, asked it which model it was, and it gave a recognizable answer that wasn’t Jev.

See https://news.ycombinator.com/item?id=49778162.

I believe the entire idea is that you always give it an explicit option for the "none of the above" cases and then handle the follow up branch outside of Jev like you usually would

Spot on. A slightly less accurate but calibrated model inspires far more trust than a 'perfect' one that's consistently overconfident.

Humans don’t use “quietly” in normal parlance. Come on man, try harder.

Edit: Dude, you didn’t even use the thing you’re talking about? It’s on OpenRouter. Do better!

Edit 2: OP is a ~60 day old account, only other (positive) commenter is a ~48 day old account. Sus.

Reply on news.ycombinator.com