Hacker News Reader

Comment by matsemann | Hacker News Reader

Before, if I would start working / thinking on a problem and discover it was harder than anticipated, that often was a signal that the feature may not be worth it. Because implementing it doesn't fit the current model of how things work or similar. Then it was back to the drawing table and find a better way of solving the underlying issue.

But an LLM just happily chugs along and does it, no one feels the friction, which means you never stop up to think if you're solving the right problem, or in the right way. I'm already seeing this bite us in the ass, as you get hacks upon hacks.

View on HN

That's why you first ask the LLM: "tell me how you would do this, and ask me any clarifications and questions you may have, and share any other thoughts you may have. Don't implement yet."

The LLM will look through the codebase, think, and tell you what it would do and if there's any design decisions you would have to make, as well as other things you probably need to be aware of. Then you go through the LLM's output and address all those decisions point by point, asking the LLM more questions if something isn't clear or requires more investigation or you're unsure what to do. And end with "address what I wrote, and share any other thoughts or questions or things to clarify you might still have, don't implement yet"

And you do this back and forth until all the design decisions have been addressed and you feel confident of what the code and architecture will look like, and only then say "ok, implement"

That way you get a lot of the benefits of writing it by hand (being forced to think through what the best design would be and how it would integrate with existing code, and increasing understanding of how existing code works) but it's still much faster. The tool I personally use is Cursor in auto mode.

P.S. actually before even that you first ask the LLM "what is the current state of X in the codebase" and then you ask follow-up questions until you have a good understanding of all the details that are relevant to you. And then you can start having the design/implementation conversation in the same chat context, since having the above information in context is useful.

P.P.S. and you can also ask a bridge question like "can XYZ be cleanly added with how things are currently structured?" or "what would it take to add XYZ to the project?"

I like using grill me in conjunction with obra superpowers spec driven development

https://www.aihero.dev/skills-grill-me

https://github.com/obra/superpowers

When I'm doing something quite complex I'll also throw in gstack office hours but I find that its usually overkill for many things

https://github.com/garrytan/gstack

I do often use the /qa skill regardless though. I always hated QA because it was so monotonous and it required a level of organization and creativity i never had (i mean im the developer so if I thought of edge cases id have solved for them)

I like that /qa covers and documents every edge case that it tested and I can nudge it about other ones

Sure, but I'm only able to do that because I'm intimately aware of everything in the code base. The more I do this where the LLM ends up writing the code (even based on my decisions), the less good critique I'll be able to give in the future.

Is that the experience you had? What if you ask it questions about the codebase?

I find AI agents really great for codebase exploration and understanding how it works. In some ways even moreso than from manual implementation since it's easier to get a global picture.

You can also ask it questions like "are there recurring patterns of how X is done in the codebase?" to which it might answer sometime like "there are actually three competing patterns" and tell you what they are and the exact files, and then you can choose to refactor them if you want.

Not everything is encoded in code, though.

Like what?

LLMs absolutely do not write good code, nor documentation. They write even worse code when they have to modify what they wrote earlier and we are exclusively using the top models. The code is coming faster than it can be evaluated and there is a push to just implement things with less review but on so many occasions I have found things that would have been completely destructive.

I think this is one of the most popular patterns seen nowadays. We approach problems that we didn't before because they were absurdly stupid and touched so many places that it seemed risky, hacky, costed too much time. It would be nice to go back to simpler problem solving and saying no to things that extend team skills

So exactly true! With ideas people unbridled, the painting-yourself-in-a-corner tendency that was always there has gone exponential, add to it the not-completely-aligned incentives, and the oh-so-clever impressive writer of long prose LLMs just end up churning. The real value add may end up being the lesson in honesty and humility to us all...

I first brought this up late last year – that there used to exist a kind of selection pressure against both developers and against features which would prevent certain functionality from ever being shipped, specifically:

- Functionality which exceed the technical ability or knowledge of the developer which built it; and

- Functionality which would require an excessive number of changes that time/cost became a constraint (likely because it wasn't an incremental change but a significant rework, or just a bad fit with the existing product).

Sometimes I had the displeasure of joining a company where you could see developers hit these blockers and tried to fight through them (likely under pressure from management) rather than avoid them. And in the process they created a mess of buggy, half-functional spaghetti code which someone else later had to resolve.

Today however, people can use agents to smash through these blockers and ship an incomprehensible amount of crap. And what's worse is they're celebrated by management who don't know any better and see this all as a productivity win with no downside.

I suppose there are two ways to look at this – some would argue that engineers don't need to understand what they're building in detail anymore so non of this matters. Instead they can always use agents to explain what's going on and prompt them to fix any issues that come up.

Then there's another camp which might argue that agents will fundamentally have all the same limitations as humans, and that at some point a codebase will grow too complex that it exceeds even the limits of an agent's knowledge retention or intelligence. Or a codebase may just grow so large that it costs 1 million tokens to make even a simple change.

Unless agents start saying, "there's no way I'm shipping that" like humans used to I don't really see how we avoid the latter scenario... Complexity simply must have limits even if agents allow the bounds of those limits to grow.

At some point models will have to take control or we risk hitting these limits in irrecoverable ways since complexity far exceed that of what a human can reason about well.

If this wasn't true then "build me all the software" would be a reasonable prompt. Because why wouldn't we just get an agent to build everything we could ever possibly need? It's obvious that in the limit there will be limits in knowledge, intelligence and cost.

Whether it's humans or agents, someone needs to manage complexity. That is the most important thing a good SWE used to do. It's why technology selection mattered, it's why good architecture mattered, it's why clean code mattered.

There is a limit, after a few 10,000's of LOC ai projects start disintegrating and becoming unmanageable.

10,000 LOC, hell my LLM can do that in a day!

Current project has over 60,000 LOC, I find using an antagonistic agent code and architecture reviews are really handy for keeping things grounded.

How do you know things are grounded?

My tests work, multiple agents confirm the code base. Code reviews pass.

I do very little actual reading of code anymore. Now I am more of a results merchant.

Right, but what are you doing? Placing UI elements on a page?

I believe tinygrad still caps the total LoC in their repository to 10k

Depends on the language, obviously. Some languages are more verbose than others, and relatedly, some languages force simple code structures that are less likely to become unmanageable.

Really?

I just did the LoC measure that isn't "official" but actually closer to how AI would read it: I.e. simply actual lines. Never mind "context window".

Just in the actual source folder there are 1.9 million lines. That's not tokens. Lines. Not even LoC as per some official canonicalized definition. Various languages. ~15 year old code base.

We work with Claude just fine.

For variable measures of "fine" depending on how much devs just trust whatever Claude spits out ;)

Yes, a 15 year old code base written by hand obviously, not by a coding agent from the start.

At the same size I only heard of AI generated projects reimplementing compilers and browser engines which come with amazing tests written by hand, but even these projects generally don't become reliable enough to use.

Well we also have an AI first code base, new product 160k "LoC" by the same measure. Not even a year old.

That 1.9m code base, within the same time frame as the 160k was created? <10k by the same measure.

10k I can comfortably throw at a local agent on my system (see recent posts in my history). Whether I can keep it that small is another question I suppose lol!

I agree 10K LOC is too little, I was saying "a few 10,000's of LOC"

In my personal project I've found having a hard lines of code limit has been a good limit on complexity. Makes me really ask if I need things.

One theory I have floating around in my head is that if a whole code base was microservices and micro front ends that were all less than 5,000 loc then you could fit the whole thing into a 100k token AI context window when working on it. And being few lines of code would force them to be somewhat simple

Yeah, I think microservices are going to see a new wave of enthusiasm for this reason in the coming years.

I've had a lot success at the company I work for shipping small contained microservices for less technical teams to maintain with agents.

Personally I think this most of what SWEs should be doing these days... Code is easy to produce now. We should be handing over the creation and shipping of code to the relevant teams.

Maybe there will be more complexity overall, but it will be constrained via well designed microservices and distributed across the organisation.

> I suppose there are two ways to look at this – some would argue that engineers don't need to understand what they're building in detail any more so none of this matters.

Is there ANY field where good engineers don't understand the basics? Can an electrical engineer do any work without understanding Volts and Amperes?

I do agree with you, but I found that with good llm discipline you can prevent the llm from chugging in the wrong direction with a bit of planning and good ol engineering before even talking to the llm.

Reply on news.ycombinator.com