The few who get paid to record are replacing far more who would have otherwise been paid to play if that was the only way to listen to any music. Is it okay if one musician replaces another but not if a non-musician does so? The person getting replaced didn't get paid either way.
Going back to the previous example, say I pay a different coworker $50 for the data to train the chimpanzee and then use it to replace the first person. In either case they lost their jobs while receiving nothing for it. In either case, what happened to them is the same, so how would they be stolen from in one case and not in another?