Hacker News Reader

Comment by AshleysBrain | Hacker News Reader

As cool as copilot may look, it does seem like a fundamental problem could be: if people widely use bad practices, and only a small amount of code uses good practices, an AI model will probably suggest bad practices. The AI doesn't know what's good practice, just what people tend to do. Similarly it will probably tend to suggest legacy/outdated code, since less code will be using the newer modern alternatives. I'd guess that's what happened here, and it's a bit embarrassing that their headline marketing demo does exactly that. It may be difficult to mitigate as well, as this will be rife throughout the training data.

View on HN

The is a big problem with Stack Overflow as well, which causes exactly the same issue.

Questions answered 10 years ago have an accepted answer that was right at the time, but it's no longer the best answer. If a better answer was made 5 years ago, it might have a chance to at least be voted higher by now, but often the current best answer is simply too new and has only a small percentage of votes compared to the others.

In a lot of ways, it's likely to be a self-reinforcing problem, same as SO: someone chooses the recommended code -- which "works" but is not the most efficient, uses deprecated API, or worse has a security vulnerability -- and this trains the algorithm to think it's a good answer and recommend it more.

For what it's worth, this problem predates Stack Overflow, and to some degree Stack Overflow tried to fix it.

Before SO, the typical way people would find answers would be to go to their favorite search engine and type their query, and search engine's heuristics were really bad for this sort of thing. If you were very lucky, you'd get a (current) reference manual, but usually you'd end up with somone new web developer who had just learned something writing a tutorial for others and it was just the blind leading the blind.

I suspect Copilot will be somewhere in-between that and the current SO copy-pasta, with the main downside being that writing bad code is now that much more easier that reviewing it.

Well, yes this is kinda true, but comments help and the ability for others to edit your answer if they have enough karma also helps. Plus a ton of people update their answers to say "Update: My answer was once the right one, but in Python 3.3+ there is a better answer by Michelle below."

What would be cool is if StackOverflow let you choose to move your question down in the ranking to be below someone else. That way the history of the answers is still there, but the more update answer would get the first impression.

This is why you always look at (at least) the top 2 answers on SO!

Could you combine copilot with an updated linter or similar? In Visual Studio the intellisense does a pretty good job of converting my manually-typed old way of doing things code into a newer, simpler version supported by newer releases of C#.

Example:

    using (var reader = new StringReader(manyLines))
    {
        string? item;
        do {
            item = reader.ReadLine();
            Console.WriteLine(item);
        } while(item != null);
    }

becomes:

    using var reader = new StringReader(manyLines);
    string? item;
    do {
        item = reader.ReadLine();
        Console.WriteLine(item);
    } while(item != null);

so it just flattened the code by removing parentheses and braces around `using`? Looks like a really simple transformation.

I agree in principle, but think it's possibly a good opportunity, to utilize this to create a compendium of industry practices, some of which could then be labeled as anti-patterns.

Reply on news.ycombinator.com