Brokers are compensated for their market knowledge, not just opening doors. The fast that he was "in" with the management company is hugely valuable, and you were able to take advantage of that. 15% is a lot of money, but brokers meet tens of clients each week that don't rent with them, so they are compensated for that low hit rate as well.

And yes, many apartments are exclusive so you have to pay a full 15% fee (split between the broker who has the listing and your broker).

You could have the greatest collection of listings in NYC and still not solve the "rental issue" because you'll always have landlords that only work with certain brokers, exclusives and apartments that are rented on the spot (with no real-time updates). It is nearly impossible to get a real-time feed of data. I've been in the business a year now and I've seen apartments get rented as I was showing my [interested] client the gym. I've showed up to apartments with clients and, embarrassingly, found out that they've been rented between the time I called to make the appointment and the time I showed up.

The entire system is outdated, and the real fix is one that incorporates brokers to make it easiest for clients to rent.