But why? That would imply productivity in the industry hasn't risen at all. Which isn't true.
Look at televisions, for example. 1% of what they cost in 1960 and 1000x better.
(Don't @ me with "smart TVs have ads now". You know what I really mean)
A lot of it is that the production improvements have mostly been in developing processes for synthetic replacements. Natural fibers are agricultural products: wool comes from sheep, that are raised on land, harvested and processed by skilled laborers, with natural variance in the input & output; linen, cotton, silk different variants of the same constraints. Polyester is not like this and it indeed can scale vastly and be very cheap. Rayon can be produced from basically any cellulose input so same.
So a lot of what would be productive gains have just shifted over to these other modes. Cotton is the main natural fiber the industry focuses on and it is mostly a lot cheaper now than it was in the past.