There's still very much a distinction between (self-)publishing on the arXiv and publishing in a 'real' journal/conference within the academic community, and I think it comes down to two factors:
1. The arXiv moderation process has a much lower bar; you see some pretty rubbish papers (often from large tech companies) make their way onto the arXiv which would never be published in a 'real' journal; ultimately this isn't a shortcoming (the arXiv has had a monumental impact on academia), but rather that the arXiv isn't trying to be a peer-reviewed journal.
2. Visibility/'impact' is lower on the arXiv; there are ~14k submissions per month, so inevitably the signal to noise ratio is low.
I feel what we really need is for a few universities to put the many millions they spend on annual subscriptions into some kind of endowment to pay for proper editorial boards for a peer-reviewed arXiv instead, open access, perhaps with a token fee for submission (or a slightly higher academic affiliation bar). I think if two or three big universities from each of the US/UK/Europe suddenly made this change we would see the death of academic publishing in months.