There are a ton of tech startups, and, not surprisingly, a lot of them are in the finance space. You know what's hot, too? Mobile. And it's going to continue to be, because New York is the walking city.
The other thing about it is that New York startups tend to keep things a bit closer to the chest. You don't tend to see a writeup in TechCrunch about an embryonic startup in NYC. Better or worse, most NYC startups start making money before the press hits and the rounds start coming in. It's a very different philosophy.
New York's not for everyone. You're among the wealthiest and most powerful, so yes, you might get a little squeamish if you're sensitive. You won't be mollycoddled.
And what's the food got to do with startup culture? I don't think the food scene is the right platform to attack New York. SF is only a food town if you want a burrito. I like Argentinian beef, other people like some hippie's broccoli. I'll give up the wine: California has some good stuff, if you're in to alcohol levels. 15%? Damn, Napa.
As others have pointed out, there are welcoming cafes literally river-to-river through the villages, and on the LES you can set up shop practically anywhere.
Now is a terrific time to be involved in NYC startups. It might not be quite as prestigious as being in the the valley, but it comes with a salary, and options aren't coastal.