You're asking the question backwards. We have a concept, "byproduct of human action". We needed a word for this concept. We more or less arbitrarily chose "unnatural" or "artificial" as the words for this concept.
You can argue that "unnatural" was a bad choice for this concept. But that's irrelevant to what words mean. There are other words like this - for example, "antisemitic" means "something that is against Jewish people", even though "semitic" means "of Jewish or Arabic or Phoenician etc. descent". So something discriminatory against semitic peoples is not necessarily anti-semitic.
Natural language doesn't follow strict logical rules. And of course, natural language is in fact itself an artificial, unnatural, construct.