I think the weird thing about this is that it's completely true right now but in X months it may be totally outdated advice.

For example, efforts like OpenMOE https://github.com/XueFuzhao/OpenMoE or similar will probably eventually lead to very competitive performance and cost-effectiveness for open source models. At least in terms of competing with GPT-3.5 for many applications.

Also see https://laion.ai/

I also believe that within say 1-3 years there will be a different type of training approach that does not require such large datasets or manual human feedback.