Unity definitely seems like it has lost sight of what made the engine rise to prominence in 2010-2015.
> I've honestly wondered a lot why Unity is successful at all
Unity was the least painful way to develop your hobby game and have it run on multiple platforms for people (artists, new game devs) who reasonably didn't want to leave a GUI.
> Why is it so hard to make a good game engine?
Really great question. I wonder if it's because the list of useful features grows so fast.
Imagine you make an engine from scratch. Great news! You have a 3D cube rendering successfully.
Now you'd like to walk around the cube. You now need a system to handle input.
Maybe you'd like to import geometry into the engine? You now need to write an import script.
Maybe you'd like a pbr shader on the cube. You've written a pbr shader, but artists don't want to edit the source of the shader. So then there's an ever expanding slider/node system as shaders get more and more complex.
Each of these small features quickly become giant wish lists filled with complexity.
One way to handle this complexity is to have a great open source community like Godot, which tirelessly develops feature and feature. In the last 10 years, the expected features of an engine have grow so large that it's hard for newcomers to compete.
In Unity it's not exceptionally easy to find the "correct" way of doing things because these engines are balls of mud, with system after system pilled on top.
Which is why we've ended up with hard to use large engines and easy to use small engines.