So from a purely dollars and cents point of view it makes the most sense to live with your parents forever, or find the smallest apartment in the worst neighborhood with the worst commute and the most roommates. But that doesn't make sense. So what does make sense?
Look at the opportunity cost of your house. What is your expected small return if you buy a large house vs. your larger return if you get a small apartment and invest the rest in the market? Remember to include all the cost of owning, taxes, maintenance, etc.
That difference, the opportunity cost, is what the decision to buy a house will cost you.
My opinion is that you should still go ahead and buy the house, knowing the opportunity cost. What are you working so hard for? A nice sized house, a car, vacations, money for retirement.
Don't buy a home so expensive that you can't save for retirement that you want or the vacation you want. Banks will let you borrow enough money to get yourself into this situation. It's up to you to figure out your whole financial picture, and buy a house that fits your income.
So go ahead and spend your money on those things, as long as they're in balance with each other. A balance that's right for you. Some people really want to retire early. Some people really want a large house. Each person has a different balance that's right for them.