@Darmani Can you think of any ways traditional program synthesis techniques could be combined with machine learning to perform #2? Assume your system has access to a large amount of practice problems/solutions to train with.
Oh certainly. DARPA's "MUSES" program (which I'm partially funded by) is $40 million into incorporating big data techniques into program analysis and synthesis. There are systems like FlashFill and Prophet which develop a statistical model of what human-written programs tend to look like, and use that to help prioritize the search space. There are also components in the problem other than the actual synthesis part, namely the natural language part. Fan Long and Tao Lei have a paper where they automatically read the "Input" section of programming contest problems and write an input generator. It's classic NLP, except for the part where they try running it on a test case (simple, but makes a big difference).
The reverse also is happening, with people incorporating synthesis into machine-learning. The paper by Kevin Ellis and Armando Solar-Lezama (my advisor) is a recent example.
I do get touchy when people label this kind of work "machine learning" and seem oblivious to the fact that an entire separate field exists and has most of the answers to these kinds of problems. Those examples are really both logic-based synthesizers that use a bit of machine learning inside, as opposed to "machine learning" systems.
Also, NLP is at the very least closely aligned with "AI" research bit traditionally and looking at current trends.
I do get touchy when people label this kind of work "machine learning"
Don't ;) (Seriously - it's just a label. Embrace the attention)
In short, you can say "it's just a label," but that's not a reason not to fight the battle over words.