GPT-2: February 2019
GPT-3: June 2020
GPT-3.5: November 2022
GPT-4: March 2023
There were 3 years between GPT-3 and GPT-4!
GPT-3: June 2020
GPT-3.5: November 2022
GPT-4: March 2023
There were 3 years between GPT-3 and GPT-4!
But there's a light and day difference post-Nov22 than before. Both in the AI race it sparkled, but also in the funding all AI labs have.
If you're expecting GPT-5 by 2026, that's ok. Just very weird to me.
> I am pretty sure that GPT-3.5 should be thought of as GPT-4-lite, in the sense that it uses techniques and compute of the GPT-4 era rather than the GPT-3 era
Compute of the "GPT-3 era" vs the "GPT-3.5 era" is identical, this is not a distinguishing factor. The architecture is also roughly identical, both are dense transformers. The only significant difference between 3.5 and 3 is the size of the model and whether it uses RLHF.
actually i was wrong about when gpt-4 started training, the time i gave was roughly when they finished