Hacker News Reader

Comment by PostOnce | Hacker News Reader

Pre AI and Post AI code review hours are both 0.75 in this made up example. I find that implausible.

Even with the same amount of code, AI code is less trustworthy* and requires more attention... but we know it won't be the same amount, it will be more. This means it will take longer to review, or there will be unforeseen consequences of not spending that extra time.

*meaning no human eyes have looked at it and said "this doesn't make sense", or "this is cheating", or "this doesn't meet requirements", and won't be caught until code review if at all.

View on HN

To me the biggest gotcha with AI code is that the bugs are not “normal”.

When reviewing human code I focus on specific parts because I know that there are parts where a person will just not make a bug (unless very junior).

AI on the other hand, will not do an off-by-one mistake, but it will happily just delete perfectly working code for no obvious reason. Or monkey patch a dependency because it missed a config flag. Or generally fail in a very novel and creative way.

The effort it takes to review AI code is much greater. And this is in a code base I am deeply familiar with.

Imo the future lies in a solid core programs with powerful plugin frameworks that expect all plugins to be code that was never read.

Yup. I can expect an llm to write proper code to update an hashmap or update a db, but ffs the amount of race conditions, use after free and general concurrency issues I found on colleagues PRs developed fully with these things is disgusting.

> Imo the future lies in a solid core programs with powerful plugin frameworks that expect all plugins to be code that was never read.

What makes you think that?

I think this is a good practice anyway. Putting as much code as possible into silos with guarded permissions.

Historically plugins have been kind of crappy because they were constantly breaking with updates. However if they only live as a spec, and are regenerated when needed they can easily survive API changes.

Bonus feature is that if all “installed” plugins are generated together, the llm can also find ways to avoid them being buggy due to weird interactions.

All this while keeping the main program from crashing.

Security wise the spec can also be inspected using a trusted LLM. It is trivial to hide exfiltration or malicious code in plugin/extension code (e.g.: honey). But it is much harder to hide it in a spec.

The hard part is that LLM code looks like there is some sort of flow. It is like a nice statistical smooth flow. It looks very convincing at a glance. No one would write code like that and not know what they are doing comments self assured and all.

LLMs are incredibly good at replicating common, coarse statistical features - which is what backs "looks very convincing at a glance".

If it's a general signal that's easy for you to recognize at a glance, it's a signal that's natural and easy for an LLM to replicate.

They're much worse at making the underlying structure work. Not incapable at all, especially not the modern LLMs. Frontier models kick ass. But it's true that an LLM denies you a lot of the classic "tell at a glance" by its very nature.

My experience as well, it is too fond of abstractions and will constantly spin up functions like: isUserAdmin(){return user.isAdmin}

which look nice on a surface level but obfuscates real understanding of the code and the actual data structures being used. Your end result is pretty and reads nice, but is bloated and difficult to reason with code.

> My experience as well, it is too fond of abstractions and will constantly spin up functions like: isUserAdmin(){return user.isAdmin}

Sounds like Uncle Bob disciples

What a nightmare, AI only knows how to write crappy Clean Code*

> Even with the same amount of code, AI code is less trustworthy* and requires more attention... but we know it won't be the same amount, it will be more.

If you think that, you aren't reviewing human code closely enough IMO. Human code and AI code should be scrutinized equally closely. Or rather, if you're relying on where you think the code came from rather than actually, y'know, reviewing the code itself, then you aren't doing a good job.

at big tech the numbers seem about right. at smaller firms - you've less admin, less meetings - so the coding part is higher.

mind you most of the stuff posted here is in regards to big tech - even though it's 'hacker' news.

Reply on news.ycombinator.com