My first clue was that PR description was absurdly detailed and well structured... yet the actual changes were really scattershot. A human with the experience and attention to detail to produce that detailed description would likely also have broken it down into seperate PRs.
And the code seemed alright until I noticed a small one-line change: a UI component had been replaced with a comment that stated "Insantiating component now requires X"
Except the new insantiation wasn't anywhere. Their coding agent had commented out insantiating the component instead of figuring out dependency injection.
That component was the container for all of the app's settings.
-
It's interesting because the PR wasn't entirely useless: individual parts of it were good enough that even if I took over the PR I'd be fine keeping them.
But whatever coded it couldn't understand architecture well enough. I suspect whoever was piloting it probably tested the core functionality and assumed their small UI changes wouldn't break anything.
I hope we normalize just admitting when most of a piece of code is AI generated. I'm not a luddite about these tools, but it also changes how I'll approach a piece of code.
Things that are easy for humans get very hard for AI, and vice versa.
The thing is, if you’re talking about making laws (as GP is), your “surely people understand this difference” strategy doesn’t matter squat and the impact will be larger than you think.
>Even if AI contributed just a tiny bit.
Which would imply autocorrect should be reported as AI use.
I assume the OP was talking about things like LLMs and diffusion models which one could definitely single out for regulatory purposes. At the end of the day I don't think it would ever be realistically possible to have a law like this anyway, at least not one that wouldn't come with a bunch of ambiguity that would need to be resolved in court.
Laws are built on definitions and this hand-wavy BS is how we got nonsense like the current version of the AI act.
I also don't understand why you think this would be so impossible to define. There are regulations for all kinds of areas where specific things are targeted like chemicals or drugs and just because some of these have incentivized people to slightly change a regulated thing into an unregulated thing does not mean we don't regulate these areas at all. So how are AI systems so different that you think it'd be impossible to find an adequate definition?
People using AI tools in their work is becoming normal. In the end, it doesn't matter how the code is created if the code works and is otherwise high quality. The person contributing is responsible for checking the quality of their contributions. Generally, a pull request that changes half the system for no good reason without good motivation is clearly not acceptable in most OSS systems. Likewise, a pull request that ignores existing design and conventions is also not acceptable. If you do such a pull request manually, it will probably also get rejected and get told off by repository maintainers.
The beauty of the pull request system is that it puts the responsibility on the PR creator to make sure their pull request is good enough. Creating huge pull requests is generally not appreciated and creates a lot of review overhead. It's also good practice to work via the issue tracker and discuss your plans before you start the work. Especially with bigger changes. The problem here is not necessary AI but but people using AI to create low quality pull requests and people not communicating properly.
I've not yet received any obvious AI generated pull requests on any of my projects. But I've used codex on my own projects for a few pull requests. I'd probably disclose that fact if I was going to contribute something to somebody else's code base and would also take the time to properly clean up the pull request and make sure it delivers as promised.