I agree with you. I believe the biggest problem with indexification is that it encourages a generic flow of money into the market, regardless of whether the market can handle it or not. Although the market isn't just gambling, and there is real underlying value, it's not an infinite source of value to give returns to anybody buying into it. Eventually there can be too much money chasing too little corporate earnings, and the market gets overpriced.
I don't think it's unreasonable for a person with spare cash to expect to get a reliable 5% or so long term just from the inevitable progress of technology. That may not continue forever, but if it ends, the problems will be bigger than just what index fund to purchase. The world will be a very different place.
(Or the market could be, if people stop seeking to raise capital there, but that's also a very different future and hard to predict.)