I'm basically on board with you but I think we're conflating two things here.
Fashion, by which I include clothes, jewellery, accessories, etc., is not a single uniform thing. Fast-fashion, in particular, is categorically not art, and is particularly egregious in terms of the environment.
But, on the other hand, I just got a sweet deal on an absolutely beautiful motorcycle jacket (that I will wear whilst motorcycling, not just for show!) that, even on sale, was as expensive as my my winter jacket and trousers combined, but will probably last me a decade or more if I stay roughly the same shape.
Is that art?
I don't know, it might be: it definitely looks really awesome, and it's probably still going to look awesome in 10 years' time.
Then you've got handmade stuff, in terms of both clothing and jewellery. I don't tend to go there because it's generally way beyond my budget, but it becomes hard to deny that it's not art.
And whatever you may think about the value of it, there is something to be said for wearing nice stuff: the trick is to reach the point where you can do it for yourself without caring what anyone else thinks, and to make sure you buy things that are going to last, and ideally have a certain amount of timelessness to them.