Hacker News Reader

I couldn't care less about the specific implementation.

My parent comment said "LLMs are useless for classification, because they can't give confidence scores". I think i have shown that to be demonstrably false.

View on HN

My understanding is these are more “confidence in the tokens selected being appropriate” rather than the more fundamental “confidence the classification chosen is accurate given training data,” where training data is samples of the distribution being classified rather than language corpus. They seem similar at first blush but the first is more about appropriateness of phrasing and how esoteric the semantics are, and the latter more descriptive of some statistically observable process - which is something LLMs are literally incapable of reasoning about directly.

Reply on news.ycombinator.com