In NY, I’d say take a look at Left Bank books (rare, old literature, photography, art books - some are even early editions of classics), Codex (little used shop with a fantastic selection), Mercer St Books and Records (basement hole in the wall), Westsider Rare and Used books (an UWS classic), Unnameable Books (good events and a literary selection).
For new books, McNally Jackson is the preferred one for the reading public. Their staff selections are useful and they have a dedicated poetry and chapbook section.