1. Assigning credit to where you heard about something.
2. Giving a claim (the best possible, and sufficiently strong) support.
From the credit perspective you should cite wherever you heard something, even if it was alleyway graffiti. From the support perspective, you should take the idea you read on the wall look it up to see if someone credible has said the same thing, and then cite them, and if not maybe not mention it all.Since publishers are commercial entities, their prestige is an asset they want to protect.
So there are other incentives at play when it comes why citing ArXiv is currently not en vogue.
I always find it refreshing to see how pre-print driven reserach communities like physics operate in comparison.
"If you haven't climbed the ivory tower you don't get to speak in the ivory tower... Also, we in the ivory tower don't listen to those not in the ivory tower. Less we ourselves are cast down from our perched position."
3. Show what is novel in your work. Novelty is important in many academic situations.
However, sometimes it happens that you read first about a topic on wikipedia, and worked something out on the information you found there, before you had time to consult wikipedia's sources. In this case you definitely should cite wikipedia.
I was advised to always cite the primary source, even if I learnt of something in a secondary source, eg. lit review or Wikipedia. The reason being that if the reader wants to follow up on it, it's the quickest path. I'd say it's also just good practice to credit the original authors for their work.
Citing Wikipedia also puts the reader in the uncomfortable position of either taking your word for something, or having to go through Wikipedia's sources themselves to verify something.
How does this differ from citing anything else?
https://www5.in.tum.de/~huckle/Sutton_Spinach_Iron_and_Popey...
I must add that this was such a pain for me, as I found several relevant articles on arXiv and I could download and read them. I can't say the same for articles found on Elsevier or ACM, where the relevant articles were mostly in the journals to which my University did not have access...
Citations to a finalized version of a published, peer-reviewed article are "best", both in terms of assigning credit (this is what the authors are supposed to be producing) and as a pointer to more information for the reader (the article has been reviewed[0], it won't change, and there's a stable location for it). Work that isn't peer reviewed shouldn't be outright banned or ignored, but the citation should carry a lot less weight. It hasn't been reviewed, it's subject to change, etc. Since these are essentially someone's musings on a topic, when you cite paper to "prove something" (e.g., you write "The work of XYZ et al. (2017) shows that <some confound> is not a problem"), people will give it correspondingly less weight.
There is a long tradition, predating arXiv by decades, of citing technical reports or "white papers". These are usually written up like a journal article, but might be difficult to publish (all negative results) or contain more details than a typical journal publication would allow. If there is a "journal" version and a "tech report" version, it would probably be better to cite journal version, but I would be shocked if someone actively objected to including a tech report.
(In some disciples, the white papers are also the only thing available. The World Bank and Federal Reserve, for example, often release white papers containing their own data. They rarely bother to publish them in a journal though).
[0] For whatever peer review's worth :-/