Comment by janalsncm | Hacker News Reader

Comment by janalsncm | Hacker News Reader

I really feel out of my depth because 2 out of the 3 methods here seem like they shouldn’t work?

> To evaluate comprehensibility quantitatively, we employ an LLM-as-a-Judge framework

This isn’t the worst idea, but it’s still a bit incestuous. Adding an LLM judge to check for hallucinations creates two new kinds of problems: false positives, where your judge hallucinates an incorrect fact, and false negatives, where the judge lets a hallucination slip by.

> We measure reproducibility through knowledge distillation. By fine-tuning a weaker model on the generated CoT traces, we use the downstream performance gain of the student as a proxy.

And my problem here, as a member of the GPU proletariat, is that this just seems incredibly inefficient. In other words, you’re going to generate a bunch of rollouts from your model then wait for the student to train? I guess if you have the compute to train a trillion params then maybe you don’t care.

LLM as judge / self-distillation is effective insofar as it can make models more reliably do things they are already capable of. But I agree that for pushing the frontier of what a model is capable of understanding and producing, incestuous is a good word and it's unlikely to scale far.

You can and should eval your judges. They're also typically easier to eval because often you have them emit categorical/structured data.

> LLM-as-a-Judge

Empirically, many problems look like they're easier to check than they are to solve. This seems like a reasonable way to bootstrap a little extra performance, with prior art in well-known DeepMind experiments. It's unclear if it works recursively (I imagine not), but the core idea is solid.

I’m not reacting to the idea of using an LLM as a judge in general. That’s a proven path.

I’m specifically reacting to using it to reward the chain of thought during RL training because models love to hack their rewards, learning any possible shortcut rather than the task we want them to.

I have my LLM agents fact check each other as a matter of course. They each regularly find things that the other missed. They are typically the same model (Opus).

Reply on news.ycombinator.com