and anyone notice that the pace has broken xAI and they were just dropped behind? The frontier improvement release loop is now ant -> openai -> google
Happy to learn more about this if anyone has more information.
But scaling pre-training is still worth it if you can afford it.