Hacker News Reader

Comment by userbinator | Hacker News Reader

What remains to be seen is whether Google also introduced more Chrome bugs in June than over the past two years, thanks to AI.

The big problem is that AI output can be very convincing and look "right", even appear to work, until you examine it in detail and realise all the edge-cases it didn't handle.

View on HN

Let's be honest with ourselves, AI review is better than human right now, especially top tier models. Not saying that for important project like Chrome, you can skip reviewing changes. Far from it. But it is probably better and more thorough than most human devs already.

To be fair it’s also worth noting that it’s much easier to find a buggy edge case with existing code than it is to write bug free code that doesn’t have any edge cases at all. It’s so much easier to read some concrete logic and find holes in it, than it is to start from nothing and end up with perfection. It’s true both for humans and agents, but agents are better are validating correctness.

Yes this is why models are superior here: they can equally (opportunity wise) attend everything in their context window.

It’s not better than human review. It’s good at catching low-hanging issues but it is often requested overly-defensive code to handle edge cases that don’t exist or are handled elsewhere in the stack. It generates a ton of noise with low-signal comments. It basically cannot at all identify problems that pose the highest risk, like integrations, correctness from a business perspective, etc.

Ehhhh, it's better at some things and not others.

My own reviews have shifted now. I tend not to examine detailed semantics anymore. The AI is as good or better than me at assessing whether a chunk of code does what the author said it was supposed to do.

Instead my job is to spot design and architecture smells, broader semantic errors, violations of unspoken business requirements, etc.

For example, I was recently reviewing code that built out a transactional flow. Part of that flow involved recording the transaction somewhere user visible and I knew that should only happen after the transaction was confirmed. AI implemented it where the transaction was posted.

That starts as an issue of underspecified requirements but that always happens in the real world. Thus that's where I can provide the most valuable insight: assessing with that context, whether business, operational, historical, or forward looking.

I wonder, how much dogfooding did they do in this run?

to be fair, with AI reviews i'm finding the opposite to be true. Anytime I push a one line fix it continuously tells me about all of the edge cases I didn't handle until it's pretty much bullet proof

Google improved chrome by reading code: Google built an agent harness using Gemini to scan Chrome's codebase, trained on a knowledge base of prior CVEs and the entire Git history, with a "critic" agent consuming developer-supplied SECURITY.md files.

In my experience, I find it to be exceptionally good at exploring and fixing the edge cases. Of course the output is not human maintainable for these fixes and needs to be heavily tests controlled, refactored or just accepted as being agent-maintained going forward.

I think I've seen it the other way. I told an AI I had data coming in at some fixed, slow rate that I wanted displayed. It did the job but only for that case. It literally put a polling loop to catch one sampling at that duration. I've noticed it gives me very specific code to the exact data, or wildly complicated code for all possible data to ever exist. Finding the middle ground has been my challenge

If that is the case, isn't that the failure of your testing strategy/setup? If your style doesn't fit full vibe coding - you can do the path of AI generate code - human reviews and tests all edge cases flow - I think in any serious software - that should be the flow until next few iterations. But either way, don't see a going back to coding every line by hand now, those days are gone. Was fun while it lasted!

No. I’ve heard this argument multiple times before - “let the human review and write tests” - and sure it might work, but this is almost never practiced. AI makes coding a lot faster, so no one is really ready to spend time on manual review and writing tests when LLMs can do that as well and take you most of the way.

This is evidence of culture problems in whatever teams you are a part of, or extrapolating what you see on social media to all of software engineering.

We still have a very strong review culture, and people work hard to review their own code before making PRs to avoid wasting other people's time.

The AI will not only write you subtly wrong code, it'll also write you as many tests as you want that are also subtly wrong and pass when run.

I've heard screen-reader users reporting that Chrome and Edge are now virtually unusable in their Windows screen-reader, while Firefox remains fine.

Every single new working student we get, we have to teach every single thing again.

Its a massive effort for our team it has to be a massive effort around the globe if you take quality series.

It will just be easier better cheaper to teach ONE LLM how to do good code review

Yep! Then you’ll never need to train any students ever again.

So…are you planning to hire only seniors who already know everything? Where do these seniors come from?

I'm happy that AI hasn't taken over yet but I do not know how the future for studens will look like and I don't know how we will continue training humans.

Its also has no big picture - it just can add ever more - cancer can grow, but it will kill the host organism and end years down the line as a dead end.

The "engineers" using that AI. Could they repass the whiteboard tests of old? Have not some of them quietly left?

Let's replay this post but imagine Google made this announcement before LLMs were wide-spread (say, 2019). Below, I replace "AI" with "human programmers".

    > What remains to be seen is whether Google also introduced more Chrome bugs in June than over the past two years, thanks to human programmers.

    > The big problem is that human programmers output can be very convincing and look "right", even appear to work, until you examine it in detail and realise all the edge-cases it didn't handle.

Your argument falls apart quickly because the exact reverse is equally as likely. Also, whenever I see someone say "just wait longer" for some effect to become apparent, it feels like a lazy argument.

Reply on news.ycombinator.com