trjordan

1. I absolutely agree there's a bubble. Everybody is shipping a code review agent.

2. What on earth is this defense of their product? I could see so many arguments for why their code reviewer is the best, and this contains none of them.

More broadly, though, if you've gotten to the point where you're relying on AI code review to catch bugs, you've lost the plot.

The point of a PR is to share knowledge and to catch structural gaps. Bug-finding is a bonus. Catching bugs, automated self-review, structuring your code to be sensible: that's _your_ job. Write the code to be as sensible as possible, either by yourself or with an AI. Get the review because you work on a team, not in a vacuum.

This is going to sound sarcastic, but I mean this fully: why haven't they merged that PR.

The implied future here is _unreal cool_. Swarms of coding agents that can build anything, with little oversight. Long-running projects that converge on high-quality, complex projects.

But the examples feel thin. Web browsers, Excel, and Windows 7 exist, and they specifically exist in the LLM's training sets. The closest to real code is what they've done with Cursor's codebase .... but it's not merged yet.

I don't want to say, call me when it's merged. But I'm not worried about agents ability to produce millions of lines of code. I'm worried about their ability to intersect with the humans in the real world, both as users of that code and developers who want to build on top of it.

Does Google also not handle this well?

Copy-pasted from the chat: https://www.google.com/search?q=translate+%D5%AB%D5%B6%D5%B9...

I'm interested in why Claude loses it's mind here,

but also, getting shut down for safety reasons seems entirely foreseeable when the initial request is "how do I make a bomb?"

There's something about this that's unsatisfying to me. Like it's just a trivia trick.

My first read of this was "this seems impossible." You're asked to move bits around without any working space, because you're not allowed to allocate memory. I guess you could interpret this pedantically in C/C++ land and decide that they mean no additional usage of the heap, so there's other places (registers, stack, etc.) to store bits. The title is "in constant memory" so I guess I'm allowed some constant memory, which is vaguely at odds with "can you do this without allocating additional memory?" in the text.

But even with that constraint ... std::rotate allocates memory! It'll throw std::bad_alloc when it can't. It's not using it for the core algorithm (... which only puts values on the heap ... which I guess is not memory ...), but that function can 100% allocate new memory in the right conditions.

It's cool you can do this simply with a couple rotates, but it feels like a party trick.

They mention this at the end of the article:

> And here is the delicious part: you can give the whole setup to the students and let them prepare for the exam by practicing it multiple times. Unlike traditional exams, where leaked questions are a disaster, here the questions are generated fresh each time. The more you practice, the better you get. That is... actually how learning is supposed to work.

The verification asymmetry framing is good, but I think it undersells the organizational piece.

Daniel works because someone built the regime he operates in. Platform teams standardized the patterns and defined what "correct" looks like and built test infrastructure that makes spot-checking meaningful and and and .... that's not free.

Product teams are about to pour a lot more slop into your codebase. That's good! Shipping fast and messy is how products get built. But someone has to build the container that makes slop safe, and have levers to tighten things when context changes.

The hard part is you don't know ahead of time which slop will hurt you. Nobody cares if product teams use deprecated React patterns. Until you're doing a migration and those patterns are blocking 200 files. Then you care a lot.

You (or rather, platform teams) need a way to say "this matters now" and make it real. There's a lot of verification that's broadly true everywhere, but there's also a lot of company-scoped or even team-scoped definitions of "correct."

(Disclosure: we're working on this at tern.sh, with migrations as the forcing function. There's a lot of surprises in migrations, so we're starting there, but eventually, this notion of "organizational validation" is a big piece of what we're driving at.)

This is a really good article. Don't get caught up in the tone of "anti-politics" or "slow is good." It's describing a brand of politics and impact that is just as mercurial as product development if you do it wrong. Infra and DevEx behaves fundamentally differently, and it can be a really great path if it suites your personality.

For context: my last job was PM for the infra team at Slack. I did it for 5 years. I didn't learn about Slack's product launch process until year 4. Everything until that point was internal work, on our k8s/service mesh and DB infrastructure.

The important insight here is about customer success and shadow management. Every successful engineer (and my own success) derived from figuring out what product engineers needed and delivering it. The "Shadow Hierarchy" feedback was make-or-break for those promotions. It's _hard_ to optimize for that, because you need to seek that feedback, understand if addressing it will actually fix the problem, and deliver it quickly enough to matter in the product org.

If you're willing to optimize for that internal success, you'll be rewarded, but in your career and in stability in the organization. I disagree this is only at Big Tech -- companies as small as 100 engineers have real and strong cultures in the right team, under the right manager.

But don't think this is some magical cheat code to ignoring what's important to the business. It's just a different, perhaps more palatable, route to managing the alignment and politics that are a necessary part of growth at any company.

> the gap to GPT-4o, Gemini 2 ... is shrinking fast

Are you ... aware that OpenAI and Google have launched more recent models?

Sure. I lost 20lbs in the last 12 months. I agree it’s worth working on, but not that it’s worth stressing about.

Appreciate it. The Apple-Watch-measured version came up to 44 since I’ve started running. I’ve been pleased.

None of my markers are high enough to trigger a doctor to care.

Between this and http://myticker.com (posted recently), I want to share a theory of mine:

1) the internet is mostly made up of spaces where the median opinion is vanishingly rare among actual humans.

2) the median internet opinion is that of a person who is deep into the topic they're writing about.

The net result is that for most topics, you will feel moderate to severe anxiety about being "behind" about what you shuld be doing.

I'm 40, and I'm active. I ran a half marathon last weekend. I spent 5 hours climbing with my kids this weekend. My reaction to these articles, emotionally, was "I'm probably going to die of heart disease," because my cholesterol is a bit high and my BMI is 30. When I was biking 90 miles a week, my VO2 max was "sub-standard."

Let's assume this information is true. That's OK. It's all dialed up to 11, and you don't have to do anything about it right now.

If you're thinking about, e.g. upgrading to Django 5, there's a bunch of changes that are sort of code-mod-shaped. It's possible that there's not a codemod for it it that works for you.

Tern can write that tool for you, then use it. It gives you more control in certain cases than simply asking the AI to do something that might appear hundreds of times in your code.

For sure. One of our first posts was called "You Have To Decide" -- https://tern.sh/blog/you-have-to-decide/

And my process now (and what we're baking into the product) is:

- Make a prompt

- Run it in a loop over N files. Full agentic toolkit, but don't be wasteful (no "full typecheck, run the test suite" on every file).

- Have an agent check the output. Look for repeated exploration, look for failures. Those imply confusion.

- Iterate the prompt to remove the confusion.

First pass on the current project (a Vue 3 migration) went from 45 min of agentic time on 5 files to 10 min on 50 files, and the latter passed tests/typecheck/my own scrolling through it.

I don't think we ever get away from the code being the source of truth. There has to be one source of truth.

If you want to go all in on specs, you must fully commit to allowing the AI to regenerate the codebase from scratch at any point. I'm an AI optimist, but this is a laughable stance with current tools.

That said, the idea of operating on the codebase as a mutable, complex entity, at arms length, makes a TON of sense to me. I love touching and feeling the code, but as soon as there's 1) schedule pressure and 2) a company's worth of code, operating at a systems level of understanding just makes way more sense. Defining what you want done, using a mix of user-centric intent and architecture constraints, seems like a super high-leverage way to work.

The feedback mechanisms are still pretty tough, because you need to understand what the AI is implicitly doing as it works through your spec. There are decisions you didn't realize you needed to make, until you get there.

We're thinking a lot about this at https://tern.sh, and I'm currently excited about the idea of throwing an agentic loop around the implementation itself. Adversarially have an AI read through that huge implementation log and surface where it's struggling. It's a model that gives real leverage, especially over the "watch Claude flail" mode that's common in bigger projects/codebases.

My conversation

"Design a schema like Calendly" --> Did it

"OK let's scale this to 100m users" --> Tells me how it would. No schema change.

"Did you update the schema?" --> Updates the schema, tells me what it did.

We've been running into this EXACT failure mode with current models, and it's so irritating. Our agent plans migrations, so it's code-adjacent, but the output is a structured plan (basically: tasks, which are prompt + regex. What to do; where to do it.)

The agent really wants to talk to you about it. Claude wants to write code about it. None of the models want to communicate with the user primarily through tool use, even when (as I'm sure ChartDB is) HEAVILY prompted to do so.

I think there's still a lot of value there, but it's a bummer that we as users are going to have to remind all LLMs for a little bit to do keep using their tools beyond the 1st prompt.

It really seems like all the next big leaps in AI are going to be fine-tuning fit-for-purpose models.

Everything past GPT5 has been ... fine. It's better at chat (sort of, depending on your tone preferenc) and way better at coding/tool use. In our product (plan out a migration with AI), they've gotten worse, because they want to chat or code. I'd have expected the coding knowledge to generalize, but no! Especially Claude really wants to change our code or explain the existing plan to me.

We're getting around it with examples and dynamic prompts, but it's pretty clear that fine-tuning is in our future. I suspect most of the broad-based AI success is going to look like that in the next couple years.

100% agree. If you care about API design, data flow, and data storage schemas, you're already halfway there.

I think there's more juice to squeeze there. A lot of what we're going to learn is how to pick the right altitude of engagement with AI, I think.

There's plenty of changes that don't require deep review, though. If you're written a script that's, say, a couple fancy find/replaces, you probably don't need to review every usage. Check 10 of 500, make sure it passes lint/tests/typecheck, and it's likely fine.

The problem is that LLM-driven changes require this adversarial review on every line, because you don't know the intent. Human changes have a coherence to them that speeds up review.

(And you your company culture is line-by-line review of every PR, regardless of complexity ... congratulations, I think? But that's wildly out of the norm.)

LLMs absolutely produce reams of hard-to-debug code. It's a real problem.

But "Teams that care about quality will take the time to review and understand LLM-generated code" is already failing. Sounds nice to say, but you can't review code being generated faster than you can read it. You either become a bottleneck (defeats the point) or you rubber-stamp it (creates the debt). Pick your poison.

Everyone's trying to bolt review processes onto this. That's the wrong layer. That's how you'd coach a junior dev, who learns. AI doesn't learn. You'll be arguing about the same 7 issues forever.

These things are context-hungry but most people give them nothing. "Write a function that fixes my problem" doesn't work, surprise surprise.

We need different primitives. Not "read everything the LLM wrote very carefully" ways to feed it the why, the motivation, the discussion and prior art. Otherwise yeah, we're building a mountain of code nobody understands.

Right -- the stereotypes of "selling" or "telling" or "persuading" are unhelpful in a lot of contexts.

Even in direct selling, many people don't want to feel they're being sold to! At a minimum, they don't want to feel out of control on decision they care about. But they're frequently open to learning, even if the constraints of how much time / credit they'll give you are extremely different.

This is a very HN sort of sentiment. How can I be persuasive without being gross?

I had a bit of a moment when I first became a PM. (I've done a bunch of things, engineering / sales / founding, but PM only sort of recently.) I realized that my job was to wake up in the morning and pick fights. Or more diplomatically: to tell people they were doing the wrong thing, and they should be doing a different thing, in a way that made them want to listen to me more in the future, not less.

That's the job. In fact, in almost every job, that's the job.

Impact happens when you reach people and they behave differently because of you. That's nothing to be ashamed of. If you do it authentically and with good intent, it's one of the best things you can do with your time.

Everybody is different, but the biggest reason I struggle with this right now is the pace of modern life.

Doing hard things is hard, and that means I won't be thinking about the other stuff I have to do. I'm more apt to miss a text from my family when I'm running or writing a document than when I'm vibe coding, because the effort is all-encompassing. Subconsciously, that's stressful, so I steer away from it.

Habits help here, because with enough repetition, I learn that it's OK to disappear for an hour to do the thing. But the real issue is getting the meta-organization of my life right enough that I'm not scared to shut down my ambient executive function for that hour. This shows up as both "I'm too busy to do the hard thing" and "I'm too tired to do the hard thing."

Slowing down isn't the answer, but it's been pretty transformative to notice that that's what I'm worried about.

I feel like the main missing point of every "coding will go away" articles (which this mostly, but not fully, is) misses that applications of any reasonable complexity are not fully defined anywhere but their code. That drives everything about modern software.

We've always had the ability to tie the code extremely directly to user experiences. The first problem is that it creates massive code duplication. The second (related) problem is that it creates an enormous, unsquashable number of bugs. There's no internal consistency.

Modern apps are COMPLEX. The abstractions and internal, non-user structure is the only reason they're maintainable at all. I'm working an a 6-month-old, 4-person codebase, and LLM-driven refactors, with me in the drivers seat, miss stuff all the time. If I forget that feature exists in this tiny codebase, how do organizations of any size function? Abstraction, interfaces, etc. -- software design.

The future is probably closer to "LLMs help clean up codebases" than "LLMs own the codebase," because the 2nd statement is effectively equal to "I will rewrite my code in English." English is a _bad_ language for describing complex systems. We could do better than Python, sure, but it's already _far_ better than English for the kind of cross-cutting behavior-and-dependency description software requires. It's valuable to have tools that translate intent into compact, high-quality code, but you still end up working with code as the fundamental artifact.

I was talking with somebody about their migration recently [0], and we got to speculating about AI and how it might have helped. There were basically 2 paths:

- Use the AI and ask for answers. It'll generate something! It'll also be pleasant, because it'll replace the thinking you were planning on doing.

- Use the AI to automate away the dumb stuff, like writing a bespoke test suite or new infra to run those tests. It'll almost certainly succeed, and be faster than you. And you'll move onto the next hard problem quickly.

It's funny, because these two things represent wildly different vibes. The first one, work is so much easier. AI is doing the job. In the second one, work is harder. You've compressed all your thinking work, back-to-back, and you're just doing hard thing after hard thing, because all the easy work happens in the background via LLM.

If you're in a position where there's any amount of competition (like at work, typically), it's hard to imagine where the people operating in the 2nd mode don't wildly outpace the people operating in the first, both in quality and volume of output.

But also, it's exhausting. Thinking always is, I guess.

[0] Rijnard, about https://sourcegraph.com/blog/how-not-to-break-a-search-engin...

AI sloppiness of this blog post aside, it's a reasonable observation.

If you're thinking about how to integrate AI into your system, it's worth asking the question of why your system isn't just ChatGPT.

- Do you have unique data you can pass as context?

- Do you have APIs or actions that are awkward to teach to other systems via MCP?

- Do you have a unique viewpoint that you are writing into your system prompt?

- Do you have a way to structure stored information that's more valuable than freeform text memories?

- etc.

For instance, we [0] are writing an agent that helps you plan migrations. You can do this with ChatGPT, but it hugely benefits from (in descending order of uniqueness) access to

1) a structured memory that's a cross between Asana and the output of `grep` in a spreadsheet,

2) a bunch of best-practice instructions on how to prep your codebase for a migration, and

3) the AI code assistant-style tools like ls, find, bash, etc.

So yeah, we're writing at agent, not building a model. And I'm not worried about ChatGPT doing this, despite the fact that GPT5 is pretty good at it.

[0] https://tern.sh

This article is right insofar as "development velocity" has been redefined to be "typing speed."

With LLMs, you can type so much faster! So we should be going faster! It feels faster!

(We are not going faster.)

But your definition, the right one, is spot on. The pace of learning and decisions is exactly what drives development velocity. My one quibble is that if you want to learn whether something is worth doing, implementing it isn't always the answer. Prototyping vs. production-quality implementation is different, even within that. But yeah, broadly, you need to test and validate as many _ideas_ as possible, in order take make as many correct _decisions_ as possible.

That's one place I'm pretty bullish on AI: using it to explore/test ideas, which otherwise would have been too expensive. You can learn a ton by sending the AI off to research stuff (code, web search, your production logs, whatever), which lets you try more stuff. That genuinely tightens the feedback loop, and you go faster.

I wrote a bit more about that here: https://tern.sh/blog/you-have-to-decide/

OK, so I love this, because we all recognize it.

It's not fully just a tic of language, though. Responses that start off with "You're right!" are alignment mechanisms. The LLM, with its single-token prediction approach, follows up with a suggestion that much more closely follows the user's desires, instead of latching onto it's own previous approach.

The other tic I love is "Actually, that's not right." That happens because once agents finish their tool-calling, they'll do a self-reflection step. That generates the "here's what I did response" or, if it sees an error, the "Actually, ..." change in approach. And again, that message contains a stub of how the approach should change, which allows the subsequent tool calls to actually pull that thread instead of stubbornly sticking to its guns.

The people behind the agents are fighting with the LLM just as much as we are, I'm pretty sure!

This is almost certainly what happened. Google's AI answers aren't magic -- they're just summarizing across searches. In this case, "Israel" + "Jordan" pulled back a video with opposite views than the author.

It's somewhat less obvious to debug, because it'll pull more context than Google wants to show in the UI. You can see this happening in AI mode, where it'll fire half a dozen searches and aggregate snippets of 100+ sites before writing its summary.

Not exactly. Black holes are not lopsided, for instance. There isn't anything on "one side" of them, once the matter has crossed the event horizon.

Rotating black holes are pretty well modeled as a spinning ring: https://en.wikipedia.org/wiki/Ring_singularity