If you can write software that improved itself better over time, our species as a whole would advance at incredible speeds. Think about all the possibilities that could evolve from self-maintained programs. Obviously the thought is a bit scary, but it's also incredibly exciting!
For the curious, here is the preprint: https://arxiv.org/pdf/1610.06918v1.pdf
The technique is an off-the-shelf GAN that attempts to learn a transformation (not arbitrary code!) from input to encrypted output. The learned model is not Turing complete, and most importantly, is _not_ self-modifying. The optimization procedure is what modifies the network, not the network itself.
GANs have been used before to create realistic pictures. They're not new here -- the application is. It's a cool application, sure, but doesn't involve self-modifying code.
Neural networks fix this problem by being sort of 'continuous programming language' where any small change to the net results in a small change to the output. And because of that we have algorithms that can find good changes to make much easier than with code. They are theoretically turing complete under some circumstances. But even when they aren't completely recurrent as in this case, they can still learn very sophisticated functions. Theoretically a neural net can learn anything a fixed number of logic gates can learn, which is still pretty powerful.
I don't think parent comment meant by computers that can "re-write their own code" was that the programs are self modifying. But the computer itself is self modifying, as one program can change another on that computer, so the computer has modified itself.
It's like calling pencil and paper self-modifying because somebody could use a pencil to write down plans to build a better pencil.
How's that for a bombastic editorial title?
(To be clear, I'm 100% agreeing with gcr here. Parameter optimization is a tremendously powerful technique, and quite possibly the "true currency" of our universe, but there's a few abstraction layers missing between that and a claim of "self-modifying AI")
This technique generalizes poorly, it really only works for algorithms with specific, somewhat narrow characteristics. If you understand the process as a weak form of algorithmic induction then the practical limitations become more apparent. Most code cannot be usefully constructed this way.
What are some that you are considering? Most maintenance I do is in the form of bug repair and change requests. Most of those wouldn't be fixable (or even identifiable) by machines. It's not errors like "value x should be y" (almost all of those get shaken out in alpha.) It's requests like "When I get to step H in our workflow, I shouldn't have to fill out form 2 to get information about the customer I'm helping."
I mean, for technical things like encryption, I could see a lot of growth potential. Other algorithms where there is a good metric for "better" would also be candidates (compression also springs to mind.) But on the whole, most applications have a lot of human interfacing and critical thinking that needs to happen. AI is going to take a long time to get to the point where it can fix those kinds of problems.