Hacker News Reader

Comment by cbsks | Hacker News Reader

I don't think we need to start looking for new career paths yet. This example has a few bugs and it took me longer to track them down than it would have to write it myself:

  #!/bin/bash
  # List all python source files which are more than 1KB and contain the word "copilot".
  find . \
    -name "*.py" \
    -size +1000 \
    -exec grep -n copilot {}\;

"-exec grep -n copilot {}\;" needs to have a space before the semicolon otherwise find fails with "find: missing argument to '-exec'".

The "1000" in "-size +1000" has a unit of 512 byte blocks so it is looking for files that are greater than 512000 bytes, not 1KB. This would be very easy to miss in a code review and is one of those terrible bugs that causes the code to mostly work, but not quite.

https://linux.die.net/man/1/find

View on HN

It also doesn't list the files. It prints all matching lines (and their line numbers), without the corresponding filenames.

You are right! I missed that. The number of bugs to lines of code ratio is approaching one.

The following line is quite widespread in use, but not as portable as it could be:

    #!/bin/bash

For increased portability, respect users' PATH environment variable using:

    #!/usr/bin/env bash

Using #!/bin/bash could lead to subtle bugs on systems with more than one version of bash installed, or outright breakage on systems where bash isn't installed at /bin/bash. (OpenBSD is known to use /usr/local/bin/bash, for example.)

That isn't a Copilot bug though, it was typed by a person.

I wonder if copilot would have produced better code if the user had used /usr/bin/env instead. If you already have buggy code, will copilot also suggest buggy code?

the main argument against Copilot for me. it takes longer to grok existing code than just write it from ground up.

This is actually a pretty important thing to understand. It can be better to rewrite than fix an existing mess. It’s similar to construction work in that sense: if you rebuild, you know what’s inside the walls.

If your house doesn’t have fire proofing (Sheetrock / lathe & plaster) then you know what’s inside the walls.

Kind of comical to open a rebuttal with “if”, showing you don’t in fact know. That aside - no, you don’t know necessarily.

That's why this copilot won't fly. The junior programmer will not be able to spot subtle errors but will kind-of feel "productive" by some random pastes from a giant brain, which cannot be interrogated.

If anything, I see copilot generating more work for existing, senior programmers - so there you have it.

Plus if you use this for anything non-trivial, it's functionally similar to using a third-party dependency, except the licensing terms are vague and there's no one to ask for help.

I could see it being useful maybe for testing. Even if it straight up copies code from the Linux kernel, you don't usually ship tests to customers so the GPL (probably) isn't a problem.

But the problem with testing is that you need to be 100% sure about how it behaves, and an AI generated test might not be reliable enough to be useful.

i'd argue the danger is even greater with tests bc the entire point of tests is to provide certainty. wrong tests are a false sense of safety that's unquantifiable without understanding every single line.

Is that your main argument against Stack Overflow? Because for some percentage of people the use cases will be similar (that is, learning some quick ways to do something that they can then explore and learn about).

Sometimes resources like these are used as references and the solution is used as is. Sometimes they are used as a survey, and it's more like asking a librarian "I want to learn more about how to X" and having them give a short exposition and point out some sources to study. It's important not to let you view of it's usefulness as one type of resource bleed into your view of how useful it might be for the other.

As a learning resource, this is very interesting.

When one gets snippets from Stack Overflow, he/she doesn't assume it is tailored specifically for the task, so extra check, modification are applied. With copilot people (eventually, especially beginners probably) will assume that the code produced by "magic" AI does exactly what they asked for.

Funnily this problem seems trivial when you have good test coverage for your code. Especially trivial compared to the insane amount of help you could get in writing boilerplate and simple logic from a magical tool like this, if it delivers on its promises.

I read this as a criticism of bash

I don't see why. It is on Copilot to produce syntactically correct code that at least doesn't fail to run, even if we ignore the correctness.

This doesn’t really have anything to do with bash, just the ‘find’ posix utility. But I agree that the arguments not intuitive.

Give them a year. It sounds like the things people said about chess engines that played stupidly

Just kidding... writing imperative code is fundamentally different than most AI recognition tasks, ie you can have GPT-4 produce consistently nice HTML but not C++

Also: "-exec grep {} \+" will be a lot faster if you have many files, and you will get the actual filename too instead of just the line number which is what the comment says it should do.

I convinced I'll do same mistake.

I am not surprised. Bash is notoriously unreadable.

There is no bash code in that script, it's POSIX find.

POSIX find is also terrible. I'm sorry but I go out of my way to avoid using it as it's always a pain.

Reply on news.ycombinator.com