Hacker News Reader

Comment by skydhash | Hacker News Reader

A naive thought: What you would get if you hardcode the language grammar and not let the training discern it, so instead of it, kinda like an expert system constraining its output?

View on HN

I try to do something like this when I ask the AI to generate tests -- I'll cook up a grammar and feed it to the LLM in a prompt, and then ask it to generate strings from the grammar. It's pretty good at it, but it'll produce mistakes, which is why I'll write a parser for the grammar and have the LLM feed the strings it makes through the parser and correct them if they're wrong. Works well.

What kind of hard coding do you have in mind? How the technique would look like?

word2vec meets category theory

https://en.wikipedia.org/wiki/DisCoCat

>In this post, we are going to build a generalization of Transformer models that can operate on (almost) arbitrary structures such as functions, graphs, probability distributions, not just matrices and vectors.

https://cybercat.institute/2025/02/12/transformers-applicati...

We already know the keywords of the language and symbols from the standard library and othe major ones. As well as the rules of the grammar. So the weight can be biased against that. Not sure how that would work, though.

I don’t think that would help with natural language to programming language, but that can probably help with patterns, kinda like a powerful suggestion engine.

Reply on news.ycombinator.com