Having gone from junior Web designer/developer to CTO/CPO and then into startups over the last 25 years, I'm absolutely convinced that the reasons for complexity in what we do day to day now is for no good reason other than CV building for FAANG type job hunting, job niche building and job security self-indulgence, and a fundamental disregard for maturing the industry.
Whilst I somewhat agree with the op I think it’s as much a hiring / resourcing issue. Hiring managers often want experts in the tech stack and domain and overlook mastery of similarly complex topics as a good proxy for ability to pick up whatever their specifics are. And the expectation of that person also having enough generalist knowledge to do gui, qa & infra as well as the rest of the engineering process.
In that regard as cto are you part of the problem or aiming to be part of the solution ?
Managers of humans build artificial empires to climb the ladder.
Managers of machines build artificial complexity to climb the ladder.
Bad for the parent org, but necessary to pad the CV.
Yet doing everything from first-principles is not viable, nor is focusing only on non-functional requirements. The solution is engineering leadership that values code not written, dependencies not added. They wince at a big commit. Every component must pull its weight. They use every part of every component and dwell in the community of it. If they don't have time for a new community, they hire a new person who does, and they may represent a new specialization on your team.
A solid team needs 5 people. CSS and Figma ('designer'). SPA ('front-end eng'). The appservers, database and outward api calls ('backend eng'). Infrastructure and CI/CD ('devops'). Finally, you need a person who owns goals, measuring past and articulating future, and take point on user and business comms ('product'). Project management can be a part-time role of anyone on the team, but fits well with product. I do not think a single human mind can be a designer/front-end/back-end/devops role and do a good job. They just don't have time to learn and stay up-to-date with all of that, and it requires an untenable amount of context switching.
I was going to add that the engineers themselves do (or should, in a healthy organisation) have the mandate to knowingly choose not to bring in artifical complexity where it's not useful or required.
But then I thought that in a lot of situtations perhaps the engineers aren't really 'Knowingly choosing not to' but 'Unknowingly choosing to' bring in that complexity, because the volume of content and mindshare that goes into proposing new and alternate ways of working and tools that aren't actually different seems to dwarf the amount of information around fundamentals and things that don't change as often.
So perhaps any accountability really should go as much to CTOs, open source developers trying to make their mark, influencers.
It seems like what we need is a sort of an integrated virtual machine for each application. The OS provides a virtual CPU+memory machine, but that needs to extend to virtual filesystem, peripherals, displays (which the OS would draw into a window), network with user-limitable urls, etc. This would prevent apps from misusing the filesystem, enable users to limit surveillance network requests, and simplify graphics.
I think there is both an inherent complexity and artificial (or 'introduced') complexity in software.
Inherent complexity comes from increasing expectations over what kinds of software we want to write (the evolution of the web and browser 'features' since its initial inception, the kinds of interaction on mobile, tablet and desktop devices) and _how_ we want to write that.
Some amount of higher-level tooling is almost a necessity because the absolute amount of lower-level things needed to make the more powerful abstractions work is too high to deal with practical. Want to write a cross-platform fully-featured-with-animation UI toolkit with just the raw hardware in your desktop? Fantastic exercise for learning, but you're almost guaranteed to start reaching for higher-level tools at some point in the journey.
The growth in artificial complexity, however, comes from so many places (including all the ones you listed).
Sometimes it's from processes and ways of working that are introduced (I sure hope you do fully automated releasable builds including management of the computers that do those).
Sometimes it's from socio-technical problems (working in a single shared code base and having to coordinate changes and releases? Surely it would be easier for teams to never have to talk to each other and just release small things independently...).
Sometimes it's from an approach to building software that is perceived as better, easier and faster (shipping a desktop application? Wouldn't it be so much easier to just bundle a full UI layout engine, scripting language interpreter, hierarchical appearance control syntax, portable bytecode interpreter and some application code that was itself compiled from a _different_ language so it could be interpreted by the interpreter you actually shipped...)
Sometimes it's from system design approaches that are pushed as 'best practice' and introduced without necessarily understanding whether you ever had the problem they were intended to solve (please, tell me more about how your relatively small ecommerce application needed to be event based in order to 'scale up').
Sometime it's because the handful of companies shipping acceleration hardware have practically zero commercial incentive to standardise on an API for writing programs that can run on said hardware, giving us a world with four (at least) slightly different syntaxes that all sort-of-mostly accomplish the same thing while being awkwardly different.
I could go on...
Not that some of the artifical introduced complexity doesn't sometimes solve actual problems or be an overall-useful thing to have. No criticism here on introducing software-based processes to formalise things, improve software quality, and so on.
But it's useful to keep in mind how much of the complexity is introduced to improve overall processes versus 'solving' for some local maxima by adding more tech without the overall software quality improving, with the end result being we've made our own lives worse in some way, without measurably improving the resulting application.