Also I've found that a lot of VCs don't want to talk to you if you aren't local, because they don't seem to believe in video conferencing and like to be able to have last minutes meetings. This seems to be changing though.
The key here is that my company is fully remote and my employees don't live here, so they get to take advantage of this purchasing power, while I get the advantages of being SV.
I don't think the whole company needs to be in SV, but it certainly helps if the founder is.
Mind you, I don't even live there, because the downsides of living there in my current situation outweigh the upsides. However,I have been there many times, and if I had to start a company right now, that's where I would do it.
P.S. Try hiring engineers for a company in Philly and a company in SV. Then try to compare the quality and number of candidates knocking on your door in each scenario.