Of course, it's nothing else. Who could possibly believe that OpenAI and others would dump billions into development and training and aren't smart enough to figure out they could also do it with $500.
> We add a vision encoder to Llama3 8B
https://news.ycombinator.com/item?id=39136472
Many companies also optimize for tools, like Python, that have boost productivity more than price/performance ratio. OpenAI had billions of other people's money. They might just keep using tools which worked before.
Lastly, there are tons of papers published on techniques that claim to reduce cost. Most of them aren't good. Their benchmarks aren't good. Even reviewing most of them is more time than a lot of AI researchers have. Those that make it to established communities usually have gotchas that come with the benefits. So, they could also simply miss a needle in a large haystack.
I think you're right that they'd be using whatever really worked with no loss in model performance. It's just that they might not for a number of reasons. The rational choice is for others to keep experimenting with those things in case they get a competitive advantage.
People upvoting the post??
Not really sure? But PT Barnum said there's always a lot of them out there.
Pretty sure they mean fine tuning though?
But even that is total tripe.
These guys are snake oil salesmen. (Or Sylvester McMonkey McBean is behind it.)