That's the difference. If you have a $2500/month mortgage, by year 10 or so that's probably $1000 equity + $1500 interest.
The "interest" portion gets tax-deducted (so you get a portion of it back), while the $1000 equity is literally yours. When you sell the house, that's the portion you get back.
---------
So really, $2500/month mortgage (after a few years of living there) is really a $1000 cost + $1500 forced savings account. Then some maintenance / taxes on top of that.
Since you're building equity while living in a home, a $2500 mortgage is in fact far far cheaper than a $2500 rental.
-------
If this is confusing, then think about the "down payment" when you first get the mortgage. A $100,000 down payment may have cost you cash, but its not like the down payment disappears. When you sell the house and close the mortgage, that $100,000 comes back to you (plus all the equity you gained).