It's hugely about: cost, and the desire of the USB forum to ship a single "does everything" protocol that keeps a large chunk of the industry from choosing to stay on some old version of the spec forever.
Tons and tons of manufacturers of cheap low end devices out there aren't willing to replace a $0.10 USB controller with something that costs 30x as much when they have no need for the high-end speeds of Thunderbolt 3, fast charging, DisplayPort alternate modes, etc. Consequently the spec can either contain a lot of optional features they can choose to ignore, or it can mandate support for all of the 40 Gbps active cable fast charging etc options, and watch as devices partition themselves between the few the need all of that stuff, and the majority that ship simple micro-USB 2.0 hardware.
If they did the latter, then USB 4 is essentially pointless: that's the exact situation we were already in 5+ years ago, except the cheap side was called "USB 2" and the expensive side was called "Thunderbolt". All we've done is rename the expensive side to "USB 4".
If they do the former, they ship a "spec" so full of options that it's already nearly impossible to tell exactly what devices or features any given USB C port will support without recourse to a specifications sheet.
Fundamentally, "one port and protocol to rule them all" is probably just a bad idea. Trying to serve the needs of both cheap slow devices and expensive fast ones puts the forum in a contradictory position, and the only "benefit" from combining the standards that serve each niche is opaque ports and cables that consumers can't easily reason about the compatibility of.