Section 313.2 of the U.S Copyright compendium states, "The (copyright) office will not register works produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author".
Moreover, the Copyright Act's definition of a computer program is “a set of statements or instructions to be used directly or indirectly in a computer to bring about a certain result”. Neural net weights are clearly neither statements nor instructions, so it's unlikely the weights would gain copyright as a computer program.
It's indeterminate whether the SSO doctrine[1] is operable on neural net weights, as it can be difficult to determine in which order weights are used. If those weights were to gain copyright protection, they may not be enforceable on similar sets of weights.
Regarding whether resembling the original work is the legal standard for being considered an infringing work: the test applied by a court is called Substantial Similarity[2], and it's extremely unlikely that neural network weights would be substantially similar to training data when considering the idea-expression divide.
Copying into the model... are you suggesting that if I have an image that's represented by an array in memory, and I copy that array from one memory location to another in my pc, that act of copying is the infringing act?
Fair use may even be a bit non-sequitur in this case. If I look at copyrighted art, I don't have a fair use case, because I haven't incorporated the art into anything, though the act of viewing has changed my brain. Training a neural net on training data may not even be considered incorporation.
On your last broad point... okay? I suppose.
[1] https://en.wikipedia.org/wiki/Structure,_sequence_and_organi... [2] https://en.wikipedia.org/wiki/Substantial_similarity