People are already unreliable and non-deterministic. Looking at that aspect, we're not losing anything.
I have a suspicion that there's a "best design pattern" and "best architecture" for getting the most out of existing LLMs (and some equivalents for non-software usage of LLMs and also non-LLM AI), but I'm not sure it's worth the trouble to find out what that is rather than just wait for AI models to get better.
software system is like legos. they form a system of dependencies. each component in the chain has interfaces which other components depend on. 99% reliability doesnt cut it for software components.
The word "need" is an extreme overstatement here. The vast majority of software out there is unreliable. If anything, I believe it is AI that can finally bring formally verified software into the industry, because us regular human devs definitely aren't doing that.
but why should the solution involve AI (thats just the latest bandwagon)? formal verification of software has a long history which has nothing to do with AI.
Because AI is able to produce lots of results, covering a wide range of domains, and it can do so cheaply.
Sure, there are so quality issues. But that is the case for most software.
One case, the other dev refused to allow a commit (fine) because some function had known flaws and was should no longer be used for new code (good reason), this fact wasn't documented anywhere (raising flags) so I tried to add a deprecation tag as well as changing the thing, they refused to allow any deprecation tags "because committed code should not generate warnings" (putting the cart before the horse) — and even refused accept that such a warning might be a useful thing for anyone. So, they became a human compiler in the mode of all-warnings-are-errors… but only they knew what the warnings were because they refused to allow them to be entered into code. No sense of irony. And of course, they didn't like it when someone else approved a commit before they could get in and say "no, because ${thing nobody else knew}".
A different case, years after Apple had switched ObjC to use ARC, the other dev was refusing to update despite the semi-automated tool Apple provided to help with the ARC transition. The C++ parts of their codebase were even worse, as they didn't know anything about smart pointers and were using raw pointers, new, delete everywhere — I still don't count myself as a C++ despite having occasionally used it in a few workplaces, and yet I knew about it even then.
And, I'm sure like everyone here has experience of, I've seen a few too many places that rely on manual testing.
It really seems like the tech-bro space hates humans so much that their motivation in working on these products is replacing them to never have to work with a human again.
Sure, but then humanity was denigrated the first time a calculator was used to compute a sum instead of asking John Q Human to do it.
I'd argue that the more we find ways to replace humans with AI, we're more clearly defining what humanity is. Not about denigration or elevation, just truth.