we do finetuning too. your number one complaint of bad dataset, we solved it by creating a better dataset optimizer than what is available in the market today.
we have continual learning where you can train domain B on top of domain A and domain C on top of Domains A and B. with out catastrophic forgetting. you should try it out at modelbrew.ai , test it and compare.
There are increasingly many finetuning services which can be pretty useful for people that have some data but not the resources to train or run inference. This blogpost can help compare some of the current available services on cost, speed and user experience.
This is quite interesting. And with the Open source models getting better and better every day, we'll soon see wide adoption of people fine-tuning OSS models and use them instead of using proprietary models