The real downside is that with no gatekeepers there and so many acts (orders of magnitudes more) you can make and release albums that nobody hears at all.
Whereas in the past, if you made it to have an album out, you had a small following almost guaranteed.
At least back when you couldn't even make the album without being picked by a record company, you could blame your failure to those A&R guys that failed to appreciate you, or get your lack of contract as a sign that you're not good enough to begin with.
Now that you can easily make an album, everybody both feels more entitled to success, but also gets to see firsthand how the audience just doesn't care.