Evals saturating faster is interesting. Do you think this is an indication of RL or training pipelines getting better over time at the labs, or is this primarily an indication of the data / RL-environment industry scaling up over time?
Likely both. Labs are getting faster at identifying capability gaps and building or commissioning work that can help develop those capabilities. Similarly the data industry is also getting more efficient with requests with faster turnaround times and better yields.