Show HN: Deep Learning for Program Synthesis | Hacker News Reader

Show HN: Deep Learning for Program Synthesis

microsoft.com

93 pointssuryabhupa23 comments

One of the authors here -- would love to answer any questions about the work! :)

The first thing I thought of when reading this article was genetic programming.

Is this a significant improvement over evolutionary computation methods? Has that been attempted in the past?

I'm not too familiar with evolutionary computation methods, but I imagine the approaches may be similar in nature.

Are there any difficulties in generating a program in standard languages in Python? Did you choose a DSL because neural network is sensitive to the output programming language?

It turns out the full grammar of Python (and almost all real programming languages) is quite large; this is very early and new work in neural program synthesis, and so we chose a pretty limited DSL to make sure that we could at least solve this one before moving on to more general ones that contain state, conditionals, for-loops, etc. In theory however, we can apply the exact architecture to Python programs and see what happens. We haven't tried yet. :)

Are you going to publish the source code for reproduction?

Eventually, yes.

Any chance this could be implemented for automated proof synthesis? Would love to see this used in F* or Lean.

Theorem solving is very closely related to program induction (we just change the grammar). Just as with Python, the underlying search space would be incredibly large, and while in theory, we could simply change the DSL and it should work, it'll probably involve a few more iterations of the model or other insights to see this to fruition (but it's definitely not impossible).

Why does the final example in figure 14 fail completely? The outputs are correct as far as they go, but they're all incomplete.

Is it because the scoring metric has a point where enough of a good start outscores an alternative in the beam search that could lead to a more complete solution? In non-trivial real-world examples, would the be a major problem?

What is the link with Excel FlashFill feature?

It looks like this is a more comprehensive version of FlashFill (can do more tasks), and it is based on deep learning instead of previous rule-based techniques in FlashFill.

As much as I'd love a program to infer my intent to a specific language, I wouldn't think it would help me as much. In my everyday work, most of the time I need to handle edge cases, which need specific instructions for specific scenarios. Once the logic is figured out, the writing part is just minimal.

Machine learning/AI can learn from common cases to infer logic from intent. For such special cases, it may not be as common and need instructions. By then, I may as well write code by hand.

Edit: grammar

@suryabhupa How similar is this work to the Grammatical Inference field? There has been a lot of work over the years in specification inference which feels similar. Many of the studies in specification inference learn automata representations of object interactions. I know there have been other application grammatical inference in Software Engineering as well.

Program synthesis is grammatical inference grown up, and statistical approaches are being experimented w/ for modern synthesis just as they were for the genetic programming & grammatical inference era. (I believe even now at the SAT solver level today.)

At a quick skim, this seems fun more as (1) an experience report of jumping on the DNN train instead of other ML algs and (2) more intriguing to me, the training formulation (irrespective of neural nets). Dawn Song's recent explorations here also sounded pretty interesting in terms of bridging logical synthesis of general programs with statistical..

Grammatical inference learns a grammar from a set of examples, where here it seems the paper is learning a program (a derivation in the grammar) from examples.

Which Dawn Song paper are you talking about here? I think among all the recent approaches proposed recently on neural program induction, this is the first one that is end-to-end trained and learns only from input-output examples without any hacks!

I would like a web API which when given training examples it generates the program in a given language such as Java

That's one manifestation of this kind of research being used in real life by programmers around the world. :)

@suryabhupa is this going to be available to try out in Excel?

There are talks of incorporating this into Excel at some point in the future, but it may take a _while_ before it can be fully productionized.

looking forward to playing with it!

Do you ever mess with the system and feed it crazy to see what it does?

aardvark

gorilla

orangutan

elephant

select list and extend

That would be pretty cool to see what it learns, but I don't think we've tried that :P

Reply on news.ycombinator.com