Teaching an AI to write Python code with Python code | Hacker News Reader

Teaching an AI to write Python code with Python code

benjamintd.com

91 pointsdsr1222 comments

Look, I was told that I wouldn't be automated out of my job for at least the next five years. Can't they start with Java?

Java is so verbose, any character-based network will probably get sidetracked before finishing a function declaration...

...thereby becoming indistinguishable from the average Java developer

You can never fire the AI because without it the whole place would burn down and nobody else understands how the code works.

Python's context-sensitive grammar makes it a particularly unsuitable target for text-based generation of code in this manner. If you continue with this project I expect you will find you get far better results from generating ASTs or choosing a language with a simple grammar in which most possible compositions of tokens are valid.

So... some form of lisp would probably be best, right? Or maybe just binary or assembly?

Read the article, these type of algorithms have an easy time with syntax. The real issue is that the code produced is meaningless.

> The real issue is that the code produced is meaningless.

Sounds like something I'd code up.

Exactly, the difficulty is not really in producing valid code but meaningful code. Same with text generation - while a recurrent neural network might produce grammatically correct sentences, I don't think we're close to it producing an story with an actual plot, discernible characters etc.

What about a statically analyzable language? That could be factored into the loss function.

I was thinking the same. Not only that, if you would add multiple networks that, for example, weigh the possibility that a full line is "good" code, and another that evaluates a whole block (like google deep mind did with AlphaGo) so it looks like dense and complex code this could be a nice program trolling with pull requests on github :-D

So it's just a char-RNN? I was hoping this was using one of the seq2seq frameworks for actually writing code, like https://arxiv.org/abs/1605.06640

Yeah, there's no effort to direct or encode intent. It's just producing what it probabilistically wants to produce. If it happens to run...cool?

Here's a interesting example from Google DeepMind learning to generate code for Magic The Gathering and Hearthstone: http://arxiv.org/pdf/1603.06744v2.pdf

This is a cool project. I did something similar with LSTM in Tensorflow to generate Reddit comments. The biggest problem I found in my experiment is that the comments generated had not real meaning, nor even tried to deliver a message. Real Reddit users identified this right away, resulting in my Bot being banned. I think the author faced the same issues here.

Does anyone has any suggestions on papers that try to solve that problem?

The problem is that the latent space of a model trained on all comments is still massive. For any sort of coherence, you need to condition the probabilities on smaller spaces, or enforce structural bounds.

Here's an example of a paper where they simultaneously optimize high-level semantics with token-level sequence probabilities: http://arxiv.org/pdf/1606.00776v2.pdf.

I don't have any papers, but I guess you could try and scrape other short comments on the same article and basing it of those?

What should we do, If computers are going to be coding?

Unionize?

It'd be interesting to make sure the minimum data (code) sent to the neural network is all meaningful or purely functional. Perhaps as somebody pointed out taking ast and then figuring out stateless chunks might be a good idea?? I'll take a shot..

Much like I can learn to speak Chinese by listening lots of sound tracks of people speaking Chinese and over time building a probabilistic model of what sound will come next.

Here's a similar example that writes LaTeX code: http://karpathy.github.io/2015/05/21/rnn-effectiveness/

This is cute, but the title got me excited for something well beyond this. Heck, even if an AI produced spontaneously code that defined a function and later used it, I'd be ecstatic. This is just a fun piece of art.

Reply on news.ycombinator.com