https://en.wikipedia.org/wiki/Inflation says "The measure of inflation is the inflation rate, the annualized percentage change in a general price index, usually the consumer price index, over time." and links to https://en.wikipedia.org/wiki/Consumer_price_index which says "The index is usually computed monthly, or quarterly in some countries, as a weighted average of sub-indices for different components of consumer expenditure, such as food, housing, shoes, clothing, each of which is in turn a weighted average of sub-sub-indices." and goes on to give an example (apparently ficional) in which housing makes up 41.4% of the index.
Edit: My copy of Samuelson and Nordhaus, Economics, 19th edition, has an example of a consumer price index including housing weighted at 42.4%. At that weighting, unless my math is off, the remaining 57.6% of the stuff in the index would need to get cheaper by 1.2% to get to overall 1% inflation with a 4% increase in housing prices:
42.4 * 1.04 + 57.6 * (1 - 0.012) = 101.00479999999999
Some things do get cheaper over time (like consumer electronics, sometimes), but others not so much.