Sure thing, that'd be a pity and would generate some unemployment. However, overall we'd be better off: people would be freed from the costs of buying toys for kids and would have their resources available for other industries and endeavors...
Don't get me wrong: if this happened overnight, it'd be catastrophic for many people, but still, on the long run, we'd be better off.
A real problem, though, is that we're still wearing a lot of low quality clothing. We, as a society, would really benefit from a shift to buying higher-quality clothing that would stay good and last for longer.