I wouldn’t bet on this
The reality is lots of software problems can’t be solved with the level of “intelligence” LLMs have. And if they could, it wouldn’t be just software in danger - it’d be every human profession. Even the physical ones, since AI would quickly figure out how to build machinery to automate those.
Because it's just cope. Look at the current reality. Are companies rushing to fix bad or even buggy code written by human devs? No, not in most cases. In most cases, if a piece of code "works", it is left the hell alone. And that's the thing about AI code: it does work. The quality is irrelevant in the overwhelming majority of cases (especially if it's other AIs that are adding to it, which is the case more and more often).
Software quality is especially important in safety critical applications.
We should not expect an LLM trained solely on formally-verified code to produce formally-verified code. I don't think that also training on specs and hateful training material will fix that.
So then we're back to the original software engineering objectives of writing better SAST, DAST, Formal Method, side channel, and fuzzing tools for software quality assurance.
Like ~100k people in Meta - nothing critical there, right? Many thousands could be replaced with AI-coders there.
Compare traditional woodworking with modern carpentry on quality, longevity, and marginal efficiency.
From "Why Don't People Use Formal Methods?" https://news.ycombinator.com/item?id=18965964 :
> Which universities teach formal methods?
> Is formal verification a required course or curriculum competency for any Computer Science or Software Engineering / Computer Engineering degree programs?
> Is there a certification for formal methods? Something like for Engineer status in other industries?
(I did make sure the challenges didn’t ban AI).
The most recent thing I did prior to typing this comment was have it look at a cosmosdb integration and recommend changes to my CosmosClientOptions to reduce CPU, based on a few exceptions I'm seeing.
It recommended 5 changes to 5 settings, and every single one of them was outside of the allowed range for those 5 settings. I told it as much, and asked nicely (always nicely, I don't want to be on the Singularity's naughty list) to try again, those aren't valid values. It came back with 4/5 values within the allowed scope, one still outside of the scope, and 2 of the 4 that were accurate were the same values that I already have configured.
Not there yet. Better every day, but definitely not there.