Hacker News Reader

Comment by _davide_ | Hacker News Reader

Agreed, it's a real issue, but it can probably be vastly reduced by having the schema in the system prompt and by giving the model an expectation of a fixed value: no decent modern would pick a prose ligament over a provided value.

To completely squash the issue, a few cheap LoRa iterations will do the trick just fine.

View on HN

Sure, you can fix that in a couple lines. Then a couple more lines for evaluating multiple questions on the same answer in parallel. Then a couple more lines for the confidence score (which is trivial to compute from all we have, but missing regardless). Then a harness to fine-tune an existing model to perform better on this specific task, and a collection of training data to use for that

I think we can all agree that Jev is not rocket science. It's a good idea executed well, with marketing that might have been a tad too bold

The confidence score is not trivial to compute. That is the whole point of the model. Even if you are using a proper scoring function such as NLL, it is not enough to ensure calibration in deep nets. So you have to do good post training to ensure it. These are all known techniques, but they are far from trivial, especially on large scale datasets.

Their docs at https://docs.typesafe.ai/confidence state "confidence is a statistic computed from the probability distribution the answer already gives you. TypeSafe computes it for you"

And further down "TypeSafe computes confidence from how the probability is spread across the options. All of it on one option gives 1.0; the more evenly it spreads, the lower the confidence. This demo uses (3 × largest probability − 1) / 2 to approximate confidence for three options."

So while we don't know the exact formula they use, it is just a function over the probabilities

I am open to the argument that this does not work well if you just plug in a qwen model instead of a model that is trained to output more statistically useful token distributions

> I am open to the argument

we agree then, that is the entirety of my argument. Getting a deep net especially one that is anywhere near even SLM size to be calibrated is tough, especially across domains. They claim calibration across a variety of datasets which is interesting.

For N options, it's (N x Max Probability - 1) / (N - 1). It's verified in this article: https://bernoulli.app/articles/is-jev-confident

It means confidence is just a converted max probability and not an independent signal.

Reply on news.ycombinator.com