Software development is riddled with unscientific practices that are derived from some person’s blog somewhere.
100 years from now when they’ve found out the optimal way to do development we won’t have all these methodologies.
It's getting better, but it's still present.
For example, nearly each source tree I've seen has its own idiosyncrasies in term of organization and layout.
Are the tests in ./test/ or ./tests/? how do I run the tests? where do you put the headers? ./inc/? ./include? what are the options available at compile time? how are they exposed? what are the install targets? does it support the usual variables (looking at you DESTDIR)?
Granted more modern languages tend to be more normalized than the previous examples which are valid for C/C++ mainly. But these questions still applies. For example, the Go projects I've seen tend to vary wildly in term of compile tool chain, some have nothing, just run go build, other have a ./build.sh and the better ones have a basic Makefile which is generally not so great (no variable present to override the path to the go binary or set some compilation variables, no proper install target, very weird handling of the ./vendor/ directory in some cases).
It's not that immature. I'd measure the maturity of engineering, or any field really, as it's ability to create reproducible results. That result could be designing bridges that last a predicable amount of time, cars that run for a certain number kilometers before wearing out, or surgery that produces the outcomes predicted. This doesn't mean the surgery always works, or there isn't the odd car that fails in it's first 1000km: just that we know what the rates outcomes are and we can keep below a given level for a known cost. That level is never 0 because the cost for 0 rises to infinity.
For software it's a case of delivering something that does the job asked for and that operates at below a certain failure rate. This doesn't mean a zero failure rate or no bugs: just that we set a upper limit and can reliably meet it. It seems pretty clear there are many large teams of software engineers out there that can do precisely that. Google has even allowed their engineers to write & publish books about how they achieve that: https://landing.google.com/sre/workbook/toc/ It's not rocket science - just a method of doing things team of sufficient size can replicate.
> For example, nearly each source tree I've seen has its own idiosyncrasies in term of organization and layout.
I've got some news: that happens in every engineering discipline, even the mature ones. There are some things that don't effect the outcome overly. What you name you directories may be one of them. It may be it so unimportant they leave it to an engineers discretion, or it may be an organisation can settle on any name, provided they stick to it.
> But these questions still applies.
No, it doesn't. If you look at that site reliability engineering handbook, it's not about naming directories. It's mostly about measuring failure rates, and then applying well known techniques if it is getting too high. The well known techniques are mostly about change control, which happens to be a very boring subject most young software engineers almost universally hate. Trying to impose change control on a young software engineer is like trying to put a lead on a cat: it will do anything to avoid it including pulling in the reverse direction to where the lead is going, playing dead and refusing to acknowledge it's possible to move at all with the lead on, and attacking the person trying to put it on.