It's not just that different industries have their hubs - media, design, finance in NYC, entertainment in LA, tech in SF. A whole ecosystem of supporting firms grows around these companies. So the most influential and prestigious law, accounting, and consulting firms tend to cluster in these cities too.
There also are a lot of people who don't necessarily want to spend their entire lives in, say, NYC but think it will be a good career move to get a prestigious name on their resume before going somewhere else.
At this point I think there's kind of a chicken and egg situation where a lot of ambitious, talented people flock to major cities because firms are there, and firms stay planted in these cities because so many talented, ambitious people are flocking there.
That said, other industries are geographically concentrated as well. If you have your heart set on working in the oil business, you probably shouldn't set your sights on living in Boston.