AI Improves at Improving Itself Using an Evolutionary Trick
spectrum.ieee.org
spectrum.ieee.org
"Some evolutionary algorithms keep only the best performers in the population, on the assumption that progress moves endlessly forward. DGMs, however, keep them all, in case an innovation that initially fails actually holds the key to a later breakthrough when further tweaked. It’s a form of “open-ended exploration,” not closing any paths to progress. (DGMs do prioritize higher scorers when selecting progenitors.)"
[0]: https://en.wikipedia.org/wiki/Kenneth_Stanley
[1]: https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_t...
While prioritizing higher scorers for selecting progenitors will initially mitigate some of the problems, you will eventually end up with hundreds of thousands of agents that only learned to repeat the letter "a" a million times in a row, which is a huge waste of processing.
I think the limits of machine learning are related to the fact that all ML "knowledge" is secondhand, except talking to humans and already to a much smaller extent programming. Getting AIs to interact with, say, cars, during training is the way forward.
>In April, Microsoft’s CEO said that artificial intelligence now wrote close to a third of the company’s code. Last October, Google’s CEO put their number at around a quarter. Other tech companies can’t be far off.
Take a moment to reflect -- a third of the company's code? Generative AI capable enough to write reasonable code has arguably not been around longer than 5 years. In the 50 years of Microsoft, have the last 5 years contributed to a third of the total code base? This itself would require that not a single engineer write a single line of code in these 5 years.
Okay, maybe Microsoft meant to say new/incremental code?
No, because Satya is reported to have said, "I’d say maybe 20%, 30% of the code that is inside of our repos today [...] written by software".
Is there any evidence of this (anywhere, not just MS or Google)?
Source: https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-a...
I think there’s some quibble to be made about “written by software” vs “generated by AI,” but it doesn’t seem like he’s talking about new code, right? He goes out of his way to phrase it “code that is inside of our repos.”
It doesn’t really make any sense, but it does seem to be what he said. Maybe there was some context, maybe it is about some specific repos that was not included in the quote.
But, it isn’t “very clear” in any case.
Zuck asks: “in terms of the coding, and how it improves that, do you have a sense of how much of the code, like what percent of the code that’s being written in Microsoft at this point is written by AI as opposed to by engineers?”
Nadella: “yeah so there’s two sort of thing we’re tracking, one is the accept rate itself, that’s around 30-40, it’s going up monotonically, [talks for a bit about the fact that it works well with Python, less so for C++, from context he’s talking about code completions here, now back to…] the place where the agentic code still, it’s ver-it’s sort of nascent—for new greenfield it’s very very high—but as I said nothing is greenfield in many cases and so therefore I’d say maybe at this point the PR—oh by, the way the code reviews are very high, and so, the agents we have for reviewing code [makes a happy expression], so that usage has increased—and so I’ll say, [this is around 45:00] maybe 20-30 percent of the code that is inside of our repos and some of our projects are probably all, uh, written by software. […]
I dunno. In conclusion having listened to it a couple times and done my best at transcribing it fairly close to what he actually said, I’m still confused as to what he meant. I was going to try and make some point with this, but I lost track of it while writing out the quote. At least folks have the full thing to argue about here, more or less!
Source: https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-a...
They never did so well
The issue with Copilot is that it is running GPT-4o and GPT-4o-mini, and both models are not good at programming.
> Other tech companies can’t be far off.
First, MS and Google are working on coding assistants so I’d expect them to be quite ahead of the curve in terms of what their CEOs report. Both in terms of what they are actually doing (since they have a bunch of people working there who are interested in AI coding assistants, surely they are using them). And in terms of that the head advertisers for these products, the CEOs are willing to say (although I should be clear, I’m not even necessarily saying he’s lying or being misleading. He’s in charge of a company that is advertising some AI tool, maybe all his reports are also emphasizing how good the dogfood is).
Second and relatedly, quoting a AI tool salesman on how much of his company’s code is written by AI… eh, it is a big company, the CEO of MS is a known figure. But maybe they should be explicitly skeptical toward him. As you note, I wouldn’t be surprised if MS was itself far off from what he said in the quote, let alone other companies…
Although, if he says:
> "I’d say maybe 20%, 30% of the code that is inside of our repos today [...] written by software".
Depending on how you look at it, that doesn’t necessarily preclude, like, classic macros and other classic code generation tools, so actually I have no idea what it even means. If an AI touches a JavaScript minifier, does it get credit for all the JavaScript that gets generated by it? Haha.
First line from the article: In April, Microsoft’s CEO said that artificial intelligence now wrote close to a third of the company’s code.
Software != AI
Source: https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-a...
CNBC misquotes Satya in the same article with his actual quote.
Do the authors really believe "safety" is necessary, that is, there is a risk that somethign goes wrong ? What kind of risk ?