I think it's not surprising that conferences have to perform some selection. You need the right amount of participants and talks. If a high rejection rate achieves the right amount, it is hard to argue against it.

But why are publications tied to conference attendance anyways? Sure, there also journals, but submitting to a journal tends to be an especially slow process. If you are in a fast-paced field, submitting to a journal is a dangerous game.

Why can we not just upload our papers to something like arxiv and then give people the option to vote on papers analogous to reddit submissions, so that promising stuff organically rises to the top. That way it would at least be based on the opinions of a sizable number of judges, not just three preselected peers.

Oh no, but what about peer review. What about it? Is it difficult to get past peer review at a top conference? Yes. Is it difficult to get past peer review in general? No. You can publish anything you want already, you will just have to jump through senseless hoops to do it. Why not skip the hoops and just upload it somewhere? We can still have journals and conferences that select high quality material from the uploaded papers and it will be an honour to be featured in one of those. You can still use features at conferences and journals as a bad metric to judge the quality of researchers, but the actual publishing will be decoupled from these institutions.