Hacker News Reader

> I'm not sure that having the patience to work with something with a very inconsistent performance and that frequently lies is an extension of existing development skills.

If you’ve be been tasked with leadership of an engineering effort involving multiple engineers and stakeholders you know that this is in fact a crucial part of the role the more senior you get. It is much the same with people: know their limitations, show them a path to success, help them overcome their limitations by laying down the right abstractions and giving them the right coaching, make it easier to do the right thing. Most of the same approaches apply. When we do these things with people it’s called leadership or management. With agents, it’s context engineering.

Because I reached that position 15 years ago, I can tell you that this is untrue (in the sense that the experience is completely different from an LLM).

Training is one thing, but training doesn't increase the productivity of the trainer; it's meant to improve the capability of the trainee.

At any level of capability, though - whether we're talking about an intern after one year of university or a senior developer with 20 years of experience - effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know. We may not be talking 100% of trust, but not too far from that, either. You can't continue working with someone that doesn't tell you what you need to know even 10% of the time, regardless of their level. LLMs are not at that acceptable level yet, so the experience is not similar to technical leadership.

If you've ever been tasked with leading one or more significant projects you'd know that if you feel you have to review every line of code anyone on the team writes, at every step of the process, that's not the path to success (if you did that, not only would progress be slow, but your team wouldn't like you very much). Code review is a very important part of the process, but it's not an efficient mechanism for day-to-day communication.

> effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know

Nope, effective management is on YOU, not them. If everyone you’re managing is completely transparent and immediately tells you stuff, you’re playing in easy mode

So the role of a coding agent is to challenge me to play in hard mode?

And suppose getting developers to not lie or hide important information is on me, what should I do to get an LLM to not do that?

no, the point is LLMs will behave the same way humans you have to manage do (there's obviously differences - eg LLMs tend to forget context more often than most humans, but also they tend to know a lot more than the average human). So some of the same skills that'll help you manage humans will also help you get more consistency out of LLMs.

I don't know of anyone who would like to work with someone who lies to them over and over, and will never stop. LLMs do certain things better than people, but my point is that there's nothing you can trust them to do. That's fine for research (we don't trust, and don't need to trust, any human or tool to do a fully exhaustive research, anyway), but not for most other work tasks. That's not to say that LLMs can't be utilised usefully, but something that can never be trusted behaves like neither person nor tool.

Anthropomorphizing LLMs is not going to help anyone. They're not "lying" to you. There's no intent to deceive.

I really think that the people who have the hardest time adapting to AI tools are the ones that take everything personally.

It's just a text generator, not a colleague.

> It's just a text generator, not a colleague.

The person you are responding to is quite literally making the same point. This entire thread of conversation is in response to the post's author stating that using a coding agent is strongly akin to collaborating with a colleague.

Yes, I want to play in easy mode. Why would I want to play in hard mode?

You're trying to sell AI here, right? And the argument is that AI is like hard mode... which developers are already in, but might not be.

It's just not a very good sales pitch.

> Yes, I want to play in easy mode. Why would I want to play in hard mode?

Working alone can be much easier than managing others in a team. But also, working in a team can be far more effective if you can figure out how to pull it off.

It's much the same as working with agents. Working alone, without the agents, it's easier to make exactly what you want happen. But working with agents, you can get a lot more done a lot faster-- if you can figure out how to make it happen. This is why you might want hard mode.

The point you missed entirely, young padawan

Are you going to say the point or are you just going to dance around it?

> If everyone you’re managing is completely transparent and immediately tells you stuff, you’re playing in easy mode

So much this. There are many managers who are effective at managing people who do not need management.

The vast majority of managers, much like most engineers, only has to deal with “maintenance mode” throughout most of their career. Particularly common in people whose experience has been in large corporations - you simply don’t realize how much was built for you and “works” (even if badly)

> effective management requires that you're able to trust that the person tells you when they've hit a snag or anything else you may need to know

This is what we shoot for, yes, but many of the most interesting war stories involve times when people should have been telling you about snags but weren't-- either because they didn't realize they were spinning their wheels, or because they were hoping they'd somehow magically pull off the win before the due date, or innumerable other variations on the theme. People are most definitely not reliable about telling you things they should have told you.

> if you feel you have to review every line of code anyone on the team writes...

Somebody has to review the code, and step back and think about it. Not necessarily the manager, but someone does.

> the most interesting war stories involve times when people should have been telling you about snags but weren't-

This comes up a lot. A person sometimes does an undesirable thing that an AI also does. So you might as well use the AI.

But we don't apply this thinking to people. If a person does something undesirable sometimes then we accept that because they are human. If they do it very frequently then at some point, given a choice, you will stop working with that person.

1000% this. Today LLMs are like enthusiastic, energetic, over-confident, well-read junior engineers.

Does it take effort to work with them and get them to be effective in your code base? Yes. But is there a way to lead them in such a way that your "team" (you in this case) gets more done? Yes.

But it does take effort. That's why I love "vibe engineering" as a term because the engineering (or "senior" or "lead" engineering) is STILL what we are doing.

Inconsistent performance and frequent lies are a crucial part of the role, really? I've only met a couple of people like that on my career. Interviews go both ways: if I can't establish that the team I'll be working with is composed and managed by honest and competent people, I don't accept their offer. Sometimes it has meant missing out on the highest compensation, but at least I don't deal with lies and inconsistent performance.

> incredibly impressive and completely unreliable.

There have been methods of protecting against this since before AI, and they still apply. LLMs work great with test driven development, for example.

I would say that high-level knowledge and good engineering practices more important than ever, but they were always important.

Test-driven development helps protect against wrong code, but it's not code I'm interested in, and it's not wrong code that I'm afraid of (I mean, that's table stakes). What I need is something that would help me generate understanding and do so reliably (even if the performance is poor). I can't exercise high-level knowledge efficiently if my only reliable input is code. Once you have to work at the code level at every step, there's no raising of the level of thought. The problem for me isn't that the agent might generate code that doesn't pass the test suite, but that it cannot reliably tell me what I need to know about the code. There's nothing I can reliably offload to the machine other than typing. That could still be useful, but it's not necessarily a game-changer.

Writing code in Java or Python as opposed to Assembly also raises the level of abstract thought. Not as much as we hope AI will be able to do someday, but at least it does the job reliably enough. Imagine how useful Java or Python would be if 10% of the time they would emit the wrong machine instructions. If there's no trust on anything, then the offloading of effort is drastically diminished.

In my experience with Claude Code and Sonnet, it is absolutely possible to have architectural and design-oriented conversations about the work, at an entirely different and higher level than using a (formerly) high-level programming language. I have been able to learn new systems and frameworks far faster with Claude than with any previous system I have used. It definitely does require close attention to detect mistakes it does not realize it is making, but that is where the skill comes in. I find it being right 80% of the time and wrong 20% of the time to be a hugely acceptable tradeoff, when it allows me to go radically faster because it can do that 80% much quicker than I could. Especially when it comes to learning new code bases and exploring new repos I have cloned -- it can read code superhumanly quickly and explain it to me in depth.

It is certainly a hugely different style of interaction, but it helps to think of it as a conversation, or more precisely, a series of individual small targeted specific conversations, each aimed at researching a specific issue or solving a specific problem.

Indeed, I successfully use LLMs for research, and they're an improvement because old-school search isn't very reliable either.

But as to the 80-20 tradeoff on other tasks, the problem isn't that the tool is wrong 20% of the time, but that it's not trustworthy 100% of the time. I have to check the work. Maybe that's still valuable, but just how valuable that is depends on many factors, some of which are very domain-dependent and others are completely subjective. We're talking about replacing one style with another that is much better in some respects and much worse in others. If, on the whole, it was better in almost all cases, that would be one thing (and make the investment safer), but reports suggest it isn't.

I've yet to try an LLM to learn a new codebase, and I have no doubt it will help a lot, but while that is undoubtedly a very expensive task, it's also not a very frequent one. It could maybe save me a week per year, amortised. That's not nothing (and I will certainly give it a try next time I need to learn a new codebase), but it's also not a game-changer.

80-20 is also a gracious ratio, my experience it’s more like 65-35

Without meaning to sound flippant or dismissive, I think you're overthinking it. By the sounds of it, agents aren't offering what you say you need. What are _are_ offering is the boilerplate, the research, the planning etc. All the stuff that's ancillary. You could quite fairly say that it's in the pursuit of this stuff where details and ideas emerge and I would agree, but sometimes you don't need ideas. You need solutions which are run-of-the-mill and boring.

I'm well aware that LLMs are more than capable enough to successfully perform straightforward, boring tasks 90% of the time. The problem is that there's a small but significant enough portion of time where I think a problem is simple and straightforward, but it turns out not to be once you get into the weeds, and if I can't trust the tool to tell me if we're in the 90% problem or the 10% problem, then I have to carefully review everything.

I'm used to working with tools, such as SMT solvers, that may fail to perform a task, but they don't lie about their success or failure. Automation that doesn't either succeed or report a failure reliably is not really automation.

Again, I'm not saying that the work done by the LLM is useless, but the tradeoffs it requires make it dramatically different from how both tools and humans usually operate.

If you're writing your own tests, sure, AI is fast at writing code that passes the tests.

But if you write a comprehensive test suite for a problem, you've effectively done the hard development work to solve the problem in the first place. How did the AI help?

Oh have the AI write unit tests you say? Claude cheats constantly at the tests ime. It frequently tests the mock instead of the UUT and reports a pass. That's worse than useless! I'm sure a huge swath of slop unit tests that all pass is acceptable quality for a lot of businesses out there.

> But if you write a comprehensive test suite for a problem, you've effectively done the hard development work to solve the problem in the first place. How did the AI help?

By making you not write the implementation?

Also, the AI writing anything bad isn’t an excuse. You’re the one piloting that ship, and if not, you’re probably the one reviewing the code. It’s your job to review your own and others’ code with a critical eye, and that goes double in the LLM age.

> doesn't work like people developers work with

I don't know.

This is true for people working in an environment that provides psychological safety, has room for mistakes and rewards hard work.

This might sound cynical, but in all other places I see the "lying to cover your ass" behavior present in one form or another.

> It doesn't work like tools developers use and it doesn't work like people developers work with. Furthermore, techniques of working with agents today may be completely outdated a year from now.

Sounds like big money to be made in improving UX

> I'm not sure that having the patience to work with something with a very inconsistent performance and that frequently lies is an extension of existing development skills.

that's a basic skill you gotta have if you're leading anything or anyone. There'll always be levels of that. So if you're planning to lead anyone in your career, it's a good skillset to develop

That's not the same skill at all https://news.ycombinator.com/item?id=45518204