By that logic you shouldn't buy anything at any time besides food and shelter because that money could be $$$ after appreciating for years or decades.
We only live one life, you can't try to optimize every dollar. There's a tremendous price on "investing" in your own happiness throughout every part of your life.