Big labs don’t expose logits, obviously. So I was wondering, if you wanted to play with distillation at scale, what other signals could you get from these models?
Ha fun building this experiment: give Claude Code/Codex an enforced “workbook” and ask them to write down decisions, alternatives they considered, tradeoffs, etc., while working Just Normal model output, not hidden reasoning. Interesting to think about whether traces like these could be useful as a signal.