This is why Black Box Reverse Engineering Exists.
Same is true for console reverse engineering. No self respecting reverse engineer reads code leaks from official console development. Otherwise they'd be in a legal hot water.
This is serious stuff and there's no blurry line in this.
If it emits a large block of copyrighted material, you'll be again in legal hot water.
Considering even fair-use can be abused (see what GamersNexus is going through) at-will, it looks even more bleaker than at first glance.
I think the only unreasonable part is llm companies are implicitly or sometimes explicitly advertizing their products output as being fit for use in other projects. I think that is a false advertising problem.
https://en.wikipedia.org/wiki/Substantial_similarity
(usual factual elements in determining the possibility of a copyright infringement in U.S. law).
I agree with you that it's possible in principle that copyright infringement would not be found even when there was evidence of access. But I think the courts would usually give the defendant a higher burden in that case. You can see in the Wikipedia article that there has been debate about whether access becomes more relevant when the similarity is greater and less relevant when the similarity is less (apparently the current Ninth Circuit standard on that is "no"?).
As an example, if you take a painting someone else made, and try and make your own version using the original as a reference, that is probably subject to the original author's copyright. On the other hand if you both happen to paint the same sunset its all ok.
I think you're more stating how you would like copyright to work, not how it actually does.
Nope. This is just how it works. I don’t care one way or another.