If you release source code under a FOSS license doesn’t that specifically mean that you explicitly consent to it being used in ways you might not be fine with?
The attribution licenses such as ALv2, BSD, MIT, etc. which are generally considered to be "permissive" still require attribution, and this condition is not upheld by popular LLMs.
Of course, the copied material has to be judged substantial enough for the license to apply in a court of law, which is a human-arbitrated threshold.
It will most likely be a copyleft author who eventually brings a court case, but the attribution licenses do not allow for LLM usage either unless the LLM provides attribution and preserves notices as spelled out in the licenses.
I think it is pretty obvious that lots of GPL authors wouldn’t want their code to be productized in ways that don’t contribute back to the community, because they had the ability to use a BSD or MIT license and didn’t take it.
Legally, of course, LLMs are not people. They don’t have the same rights as people, and it isn’t obvious whether or not they can legally generate new IP. They operate by a complicated but essentially mechanical process, and it is pretty novel to say that such a process could be used to remove copyrights.
But the defendant won't be Microsoft — they just provided a tool, and that's legal. No, the defendant will be the downstream consumer who incorporated the code spat out by the LLM.
It doesn't matter whether the LLM "learned" — intent is irrelevant and the defendant will have committed copyright violation regardless. The LLM can't copyright-wash the code — nobody can.