As someone who leans pro in this debate, I don't think I would make that statement. I would say the results are exactly as we expect.
Also, a highly verifiable task like this is well suited to LLMs, and I expect within the next ~2 years AI tools will produce a better compiler than gcc.
That's what always puts me off: when AI replaces artists, SO and FOSS projects, it can only feed into itself and deteriorate..
We can't do it for all domains, but I believe we can for efficient code.
today's models could be probably already good enough to compose tasks, and evaluate the results.
Building a "better compiler than gcc" is a matter of cutting-age scientific research, not of being able to write good code
Right.
and the "anti" crowd will point to some exotic architecture where it is worse
Yes it will be far easier than if they did it without AI, but should we really call it “produced by AI” at that point?
I think compilers though are a rare case where large scale automated verification is possible. My guess is that starting from gcc, and all existing documentation on compilers, etc. and putting ridiculous amounts of compute into this problem will yield a compiler that significantly improves benchmarks.