Second thumbs up for Micro Center. I like that they have a combination of consumer-level electronics, mid-level stuff like Raspberry Pis, and low-level stuff like resistors and whatnot.
It's a bit of a pain for me to get to MicroCenter (about an hour one the train), but it has always been useful for me when I'm in a pinch.
When I visit my parents in Orlando, there's a smaller place called "Skycraft" which has a lot of low-level components, but I haven't been able to find an analog to it in NY just yet. Do you know of any?