Same for the non-San Jose parts of Silicon Valley: Palo Alto and Mountain View are ritzier than neighboring Sunnyvale and Santa Clara. I don't know enough about San Jose itself to say, nor about San Francisco. In SF, though, it seems like many of the ritzier areas (Pac Heights, Marina, Marin) are to the north.
NYC is a notable exception: Harlem and the Bronx are in the north, while SoHo, Tribeca, Greenwich Village, etc. are all in the south.