> another fun fact, according to Twitter rumors GPT-5 is already training on 25,000 A100s w/8 GPUs each, so 4.8m GPU/hours per day.

If true that is incredible. From what I understand there isn’t a function that relates the increase of training time/effort to performance. For example, twice the training doesn’t mean twice the performance. It could mean no increase or 1000x increase, no one really knows. Does anyone have any idea what to expect in terms of capabilities when this training is finished?