Why train on scientific texts when half the papers are BS?
An AI that does that detection would be both wonderful and dangerous.
(That book abandoned its only interesting ideas and went totally off the rails a few chapters later IIRC.)
Still, it is a great question and part of me wonders about the whatifs of this evolution.