There's trillions of dollars sitting in indexes that are quite literally 'passively' invested. Virtually everything holds this bundle in one way or another. Passive indexing has both outperformed and overtaken active investing - leading a lot of money into VOO/VTI/QQQ/etc that track the S&P500 or some other index ("the market"). For retirement funds like 401ks, retail contributes money every paycheck that gets routed into these indexes. There may not even be much of a choice - your 'plan' may only let you pick some kind of "Target Date Fund" and then the institution picks what it goes into, usually indexes.

If you fully actively managed your own money and picked mostly individual stocks (not broad indexes) then yeah you could change your allocations. But there's a lot of money already in.