Do you remember when the only way to make your site discoverable was to 1) get yahoo to add you to their hand-curated list of sites or 2) trade banners with a super scummy link exchange? If we still had that internet, the only sites that would get any traffic at all would be the ones that could justify their existence with traffic, i.e. these[1]. How do you get traffic if you can't get listed? You can't unless you advertise through other media.
Is everything better? No. Privacy's worse, signal to noise is worse (that's a natural consequence of availability, anyway). Are a lot of things better? Yep. Niche sites can get traffic in a way they never could before. That's the natural consequence of taking humans out of the loop- instead of having to interest Jerry or David, now your only requirement for indexing is to interest anyone who is also interesting.
Let's keep working to make things better, but let's also not forget the improvements that have already been made.