The mortgage interest deduction distorts the market in several ways. Making credit more available does allow people to spend more on homes, but this is then priced into the market. As you hint at, the subsidy is net neutral for homeowners, as it ultimately benefits the housing and banking industries. People who owned suburban land at the implementation of the deduction also benefited hugely, but now that prices are artificially inflated, any removal of the deduction would take money away from homeowners.

Additionally, it makes paying cash for a home unrealistic, making the housing market (more) dependent on banks and their attendant federal regulations. Over time this has had numerous negative effects on land use patterns and urban health by encouraging car-oriented sprawl and devaluing urban mixed-use neighborhoods. There's also the whole race-based redlining thing. The worst of these policies have been mitigated in the recent past, but they still pose obstacles to certain kinds of development (that are highly desirable lately) and would not have existed in the first place without government intervention in home mortgages.