Hacker News Reader

Comment by kylenessen | Hacker News Reader

The trap was hidden text instructing the AI to use the word “Madagascar” in a nonsensical way. The assignment was about the Industrial Revolution. “Madagascar wore a toaster to the basketball game” is an example of what a student turned in. The instructor says it was the midterm, but looks like it was more of an online discussion. Anyway, I knew cheating was bad, but this is just flagrant.

View on HN

It always amazes me that people don’t even check the output of the AI.

I am at the moment going through a codebase where the dev told me that he had AI write the tests and that the project is finished. Turns out that some tests are totally nonsensical and a lot of the real world edge conditions aren’t tested at all.

I am all for AI but you can’t just blindly let it do its thing and think you are fine.

It's human nature to use a probabilistic strategy of cheating until one cheat works. We will never eradicate that, and the careful person that checks the output of AI will always be a minority.

This happens all the time. I looked at a PR, many tests were broken, told the guy he needed to at least fix the previously working tests, got tons of slop like changing assert 200 to 500.

Test cases being nonsensical and not testing any real world cases was extremely common even before AI, in my experience.

When teams rely on metrics like code and branch coverage, developers will write tests that only serve to satisfy those metrics. I've seen tests that end with "assert true == true"; they don't test anything and only exist to "cover" a branch.

The situation has improved vastly because of AI. AI agents frequently write good tests that have a real chance of actually catching bugs and improving the reliability of the application.

I couldn't imagine cheating like that and then not even checking/editing the output.

Don't look at how people submit code in industry then.

I can very easily see why there is cognitive burnout with LLM code. Now you essentially have to quality check 10x the code and not all of it is going to be producing stuff that is easy to read or ever remotely checked before being submitted.

I’ve just had the displeasure of interviewing a batch of candidates over zoom. ALL of them were cheating with AI in realtime.

If you have so little scruples, I can imagine you wouldn’t think twice about doing that with an assignment.

I absolutely would not, and I’m concerned that my conviction on that is going to make me unemployable in the near future.

I’m simply not built for the “puzzle based memory trial in front of a live firing squad”. It will not measure my ability.

Sorry, I wasn't suggesting you had little scruples. That's just a silly part of common parlance that can be confusing sometimes.

Reply on news.ycombinator.com