Hacker News Reader

Comment by kevinlu1248 | Hacker News Reader

Yup, saw a few papers about this over the past two years, using graph neural networks for code generation. There's also another thread below on this topic.

Edit: Here's some of the papers: https://arxiv.org/abs/1911.09983 and https://aclanthology.org/2021.findings-acl.384.pdf

View on HN

Very interesting! Thank you.

I’ll try to explain something I’m thinking, it comes down to a type of agglutination.

https://en.wikipedia.org/wiki/Agglutination

The ASTs need to become sequences of tokens for LLMs to work well. Also the embedding space is related, this should all be specifically optimized for code, not based on general human language .

Of course the description of the code would be english. But my point is that a subspace of the encoding should be specifically designed for an agglutination based sequential expression of ASTs.

Not sure any of that makes sense to someone with more expertise in this space than me.

This is really interesting, will take a look. So is this basically English to code translation?

Yeah what I’m thinking about is that and how an LLM could use ASTs expressed in some type of compact encoding and with that code tied to english descriptions, perhaps it will learn really well the semantic cross space.

Like stochastically generate into an intermediate representation that can be procedurally compiled to code right?

Yes

Reply on news.ycombinator.com