That's why Alpaca is so exciting: it instruction-tunes LLaMA to the point that even the tiny 7B model (the one that fits on a phone) produces useful output: https://simonwillison.net/2023/Mar/13/alpaca/
If the cloud of uncertainty around commercial use of derivative weights from LLaMA can be resolved, I think this could be the answer for a lot of domain-specific generative language needs. A model you can fine tune on your own data, and which you host and control, rather than depending on a cloud service not to arbitrarily up prices/close your account/apply unhelpful filters to the output/etc.
Person A uses GPT3 to generate training data, and publish it on his blog, without representing it as human generated. Person A does not give permission for it to be used by Alpaca team.
Alpaca team comes along, scrape his blog, and uses it as training data, without permission from person A. Now this is fair use, so there is nothing person A can do to stop it, just like how Github scraped our code for Copilot without permission.
That would have the same licensing problems that they have though: that alpaca_data.json file was created using GPT3. But creating a "clean" training set of 52,000 examples doesn't feel impossible to me for the right group.