Ask HN: Has anyone had even minor success with LLM powered code completion? | Hacker News Reader

Ask HN: Has anyone had even minor success with LLM powered code completion?

1 pointLiamPowell4 comments

I keep hearing about how all these LLM code completion tools are increasing productivity on social media posts, yet every time I've tried one they've generated complete garbage for even the most simple tasks. Below are some examples:

- Here I'm implementing a series of procedures in exactly the same way from a list of signatures. The tool just makes up a parameter with a type that does not exist anywhere in the codebase: https://files.catbox.moe/4yqz09.png

- Here's a 128 line Python program where I'm just writing the same line multiple times with a very common sequence of words as the parameter and the suggested next parameter is just nonsensical: https://files.catbox.moe/p4c5io.png

- Here I'm writing the same line over and over for each member of a record. Once again it just makes up a name that doesn't appear anywhere in the codebase. There's also hundreds of lines of the exact same thing above this, not just the few shown here: https://files.catbox.moe/8o8lgg.png

This type of thing is the usual case every time I've tried these tools and a correct suggestion is the exception. Is this the experience that everyone else has had or do these tools actually work for some people?

The completion seems good to me. In fact suggesting "speed" as a parameter seems unexpectedly good rather than nonsensical.

In any case, even if you dont like speed, the actual code is spot on.

Foes anyone really expect better than what your images show which in my opinion is pretty damn good. Do you really expect to write a program just mindlessly hitting tab all day long?

> The completion seems good to me. In fact suggesting "speed" as a parameter seems unexpectedly good rather than nonsensical.

I can't see any way the "speed" could ever be the next item in the series "velocity, acceleration, jerk, snap", could you explain how it could be? It's an extremely common sequence so it should be something a LLM can get right easily. Even the auto complete in a Google search gives the correct answer.

> Foes anyone really expect better than what your images show which in my opinion is pretty damn good. Do you really expect to write a program just mindlessly hitting tab all day long?

The other examples (and most of my other tests) show variables and types being invented that don't exist anywhere in the code base, how is that good? In both of these cases the LLM is strictly worse than my regular auto-complete that gives the correct result. I fail to see how a downgrade from that can be considered good.

You cant see any connection between speed and velocity or acceleration? Speed and velocity are literally synonyms at least in the common use and certainly as close to velocity as acceleration.

But hey noone is forcing you to use this tool right so just turn it off I guess.

Even without noticing the pattern (the names are nth derviatives of a vector displacement with respect to time), "speed" makes no sense to include in a sequence that already includes "velocity".

Reply on news.ycombinator.com