I suspect a lot of it is historical. If you go back a few years there were lots of browser incompatibilities and many limitations in Javascript, CSS and HTML. To work around those difficulties and limitations many frameworks were created. Today many of the limitations of the basic technologies have been fixed but the frameworks are still here, though probably less necessary than ever. But once people have gotten used to doing things a certain way (for example using jquery instead of querySelector) they tend not to change.

This isn't unique to web development. The same thing happened with C++ and Boost, for example, where gradually the language and standard library evolved to include functionality that previously required 3rd party libraries.

But I think far more work and money has been invested in web development than other areas, hence creating a bigger mess.

My impression is that the actual user experience of most web sites, including the big ones like Amazon, has improved very little over the last decade, so it feels like all the effort put into web technologies has, so far at least, not produced a lot of value.