Hacker News Reader

Yeah, that seems just unclear language and because it's trained on human language, 'is' does not equal 'equals'. Using 'equals' will help.

View on HN

That's the whole problem of LLM: they work only on human language.

Even before computers we created formal languages (mathematics, logic equations) precisely because human language is too often ambiguous.

Don't you lump math in there, math is 99% human language. The symbol pushing you learned in HS is just advanced arithmetic. Math is more like legalese with some very loose additional notation than a formal language.

Can you expand on this? Notation like "=" can be written using language but we define an exact meaning for the operator regardless, unlike language.

I suppose that begs the question, what if we trained an LLM only on examples of formal languages?

In the particular cases being discussed, there's no ambiguity: "is a" means "member of" and "is the" means equals.

> In the particular cases being discussed, there's no ambiguity: "is a" means "member of" and "is the" means equals.

Yes, and fitting just those cases would result in a model that handled other cases incorrectly, because idioms inconsistent with that rule exist. (“Jodie is the bomb” has a meaning distinct from the individual words taken separately which is not stating a reflexive equivalency, for instance.)

Reply on news.ycombinator.com