Yeah, having a 3k$ carbon bike or a 1k$ aluminum model doesn't make a difference that's true. However, having good cycling clothes is way more comfortable, as is a lightweight helmet. A more upright bicycle absolutely robs you of energy due to wind resistance, even at low-ish speeds on flats (30 km/h).
I could exercise on my 40 lbs, 3 speed upright 1940s english roadster in jeans and a sweater for cred, but that doesn't sound fun at all. People who don't cycle (unlike me and you I guess) can't price cycle kit by eye so they'll see me in lycra and think that I'm some dentist when in fact my gear is all very reasonably priced, comfortable and appropriate to my level.
Also, it always bother me that people talk about 10,000$ bikes or whatever which is very far from reality, meanwhile they buy 45,000$ cars when 20,000$ cars exist and no one bats an eye.