The painful truth about it is that we're both right.
Those luxury condos aren't built by landlords wanting to bring down the rent, obviously. They're built with full intention of taking advantage of clueless yuppie transplants who haven't a clue what things should actually cost, or whose monstrous senses of suburban entitlement leave them thinking they need a 2500 SF (232 m^2) space in the city. In the short term, they actually do drive up rents. The opportunities and bargains come out in the long term.
What then happens is that the developers overbuild, and a few years later a large quantity of housing stock comes on to the market, depressing price. Look at what has happened in Florida, Nevada, and even California. It can happen again, and it's ugly but still far better than the alternative (Manhattan rent levels).
Also, most "luxury" housing is not of a quality that real rich people would accept. "Luxury" means there's a small gym and a doorman, but the quality of this housing/renovation is usually low-- which is fine given that you're just using it as a crash-pad while you work a 60+ hour job that requires living close to work. (Rent if you can; don't buy.) "Luxury apartments" are for people making upper-middle incomes in their professional jobs who need college-like pampering because of their insane work schedules, rather than genuine rich.
That said, you have to be extremely careful in playing this sort of game. Manhattan "luxury condos" are selling at 40-50 times annual rent for a variety of reasons, so you'll actually lose a lot of money if you buy one and try to rent it out. Trying to time the crashes caused by overbuilding is not something most people can afford to do. However, it is abstractly and also obviously good for society when more housing is created.