Sure, rent is expensive. That's about it, and even then if you're willing to live in the right part of Jersey you can still be 30 minutes from work and have very cheap rent.
Most big ticket items are the same. Anything ordered from Amazon is the same. Laptops, tablets, sound systems - the same. Tons of clothing deals all over the city. Food is actually cheaper because there are so many little farmer markets and dollar stores. I understand that if you just go by what some people are paying for a hamburger, you could think that NYC is much more expensive, but you don't have to eat at the same place i-bankers and tourists eat! A street gyro cost $4 and there's $1 slices of pizza.
Finally, even if the cost of living is higher, it's not a comparable product. You are paying not only to have a place to sleep, but you are also paying to live in New York City, one of the most exciting cities in the world with endless amounts of diverse people, places, and activities. Paying higher rent so that you can live in the city many people dream about makes sense, and the fact that salaries are largely adjusted to make this a reasonable proposition makes it an easy choice.