Because otherwise it becomes convoluted. It is acceptable to cite and source published material. Having to account for the source of one’s ideas, however, citing friends and influences - it shouldn’t be a moral requirement, just imagine!
But there is, I think, a big gap between "it is not a source to cite" from your original post, and "it shouldn't be a moral requirement" in this one. I think that, while not every utterance should be annotated with references to every person or resource that contributed to it, there is a lot of room particularly in academic discourse for acknowledging informal contributions just as much as formal ones.
These are not the uses with which I am familiar—as Fomite says in a sibling comment, I am used to referring to citing personal communications; but, if you are using "cite" to mean only "produce as a reproducible testament to truth," and "source" only as "something that reproducibly demonstrates truth," which is a distinction whose value I can acknowledge making even if it's not the one I am used to, then your argument makes more sense to me.