[flagged]
Yes. If you give models that have a cutoff of 2024 the documentation for a programming language written in 2025 it is able to write code in that language.
Grammars are an attempt at describing a language. A broken attempt if you ask me. Humans also don't like them.
For formal languages, which programming languages (and related ones like query languages, markup languages, etc) are an instance of, the grammar defines the language. It come first, examples second.
Historically, computers were very good at formal languages. With LLMs we are entering a new age where machines are becoming terrible at something they once excelled at.
Have you lately tried asking Google whether it's 2025? The very first data keeping machines (clocks) were also pretty unreliable at that. Full circle I guess.
YES! Sometimes. You’ll often hear the term “zero-shot generation”, meaning creating something new given zero examples, this is something many modern models are capable of.
Neither does your average human. What's your point?
But being better than an average human is usually one of the higher bars.
Autocomplete/intellisense in an IDE is probably the most salient example. An autocomplete that performs _as well_ as the average programmer is, well, totally useless.
It’s not very precise though. Autocomplete will give me a list of valid APIs for current given version of any library sorted by most recently used with a locality bonus. Copilot on the other hand does not have that level of precision. Two different tools that I use for slightly different things.
Of course it can. It will experiment and learn just like humans do.
Hacker news people still think LLMs are just some statistical model guessing things.
That's exactly what they are. It's the definition of what they are. If you are talking about something that is doing something else, then it's not an LLM.
The power of LLMs are the emergent properties that have emerged which weren't specifically taught to the model. That is not a property that exists in statistical, largely more deterministic models.
If you think it's just a giant token prediction machine you've just ignored the last 5 years.
this is so reductive it's almost not even worth talking about. you can prove yourself wrong within 30 minutes but you choose not to.