The complexity caused the variety, not the other way around. Networked systems are inherently complex. Most of the technologies you mention are attempts to solve that complexity in some way, and the ones that stuck ended up being ideal for specific use-cases but not others.

The industry trends towards the most useful solution, not the simplest one. React isn’t internally simple, but it killed the frontend JS framework experiments which used to come out daily because it really established a useful paradigm that covers a lot of the web GUI usecases.

The process is messy but it’s not illogical