Hacker News Reader

only if you bias your "random sampling of the distributions it gives"

Fixing the seed is still intentional bias. Or you could force it to always take the one token with the highest probability, but that is still biased sampling. Deterministic, sure, but intentionally wrong just to avoid a technically

View on HN

There's nothing inherently biased or intentionally wrong with greedy decoding. Why would there be? Let's say you're trying to predict what an expert doctor would say to a patient with cancer. You think there's a 99% chance the doctor would say "start chemotherapy" and a 1% chance the doctor would say "don't worry about it". If you have to pick one to output, you'll pick "start chemotherapy". It would be crazy to say the best thing to do is to roll a 100-sided die to determine your answer. A language model is literally in that exact situation.

Reply on news.ycombinator.com