Hacker News Reader

even in pre-training, usually 30%-50% is code these days.

That would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.