If a stock goes up, they don't need to buy more of it - the value of the shares they hold will increase to exactly the right proportion that they should have.
(There do exist other types of funds which try equally weight the stocks they hold, which means the fund has to rebalance when the stocks change price.)
Typically, cap-weighted funds only have to rebalance when individual members of the fund buy/sell - and they try to make it so people are buying/selling to those within the fund where possible to avoid having to do even that. The result is a very low churn, which is part of why index funds have lower fees.
Usually, the more assets an index fund has under management, the lower an index fund's fees get.