Hacker News Reader

Comment by Krssst | Hacker News Reader

Yes, I dislike this kind of take so much. It keeps being repeated as a truthism and a way of putting down people that don't do what the speaker wants. It's fine to disagree, but there's no need to get such a threatening tone.

A lot of tech jobs seem to be only about sheer output volume, with quality (maintenability, availability, security, generally understanding what the thing is doing) not mattering much. In that case sure, LLM all the way and whatever happens happens. But not all jobs are like that.

View on HN

My experience with regard to quality is quite the opposite.

With LLM at my disposal, I had the time- and effort-budget to expand test suites considerably, I was even able to attack a somewhat thorny question of reproducible builds on MSVC, which is not exactly friendly towards determinism.

These tasks would take me personally so much time that I would have to set them aside, at the cost of output quality.

Sure, but this is not at all uniform in how people use these tools any more than there was uniformity in how people balanced quality and speed before LLMs entered the picture. There was already a lot of variety in how some developers moved fast and broke things, others moved slow and fixed things, some would prototype new crazy ideas and others would spent time on the long tail of getting something from working adequately to being robust and polished.

This isn't to say that LLMs aren't impactful, but that there's an argument for viewing them less as being a fundamental shift in how our profession works and more as another tool we can use to pursue essentially the same goals more efficiently than before. Like any other tool that's worth having, they can do things our existing tools couldn't do as well, or else we wouldn't have added it to our toolbox, but you still need to be able to recognize when to use it and when not to (and potentially how to use it when you do).

I think that part of why these tools are so polarizing is that there was already some assymetry in how much longer it takes to clean up things than to create things that need to be cleaned up, so a new tool that makes everyone more productive has a lot of potential to exacerbate the existing imbalance. To make up some numbers for illustrative purposes, if someone introduced four new flaky tests in the time it took to fully diagnose and clean up one, and then LLMs came and made everybody twice as productive, now in the same amount of time someone might introduce eight flaky tests while you fixed two, so you're falling behind twice as fast. Unless the productivity gain disproportionately speeds up the people working on making things more robust and polished (which I find dubious; if anything I think the opposite seems more likely) or LLMs suddenly make everyone who didn't care about quality when rushing things out take it more seriously (which seems even more dubious), then LLMs don't improve the situation for people who already felt that the balance was slanted too heavily towards speed over quality.

Yep, none of my current team-mates test their work especially thoroughly... so now we have even more bugs being found in testing in production. Not a net win.

Likewise - it's not just about generating production code faster; it's now feasible to create all kinds of experiments, test fixtures, visualizers, analyzers, and support software you never would have had time for before, which you can use to improve your process and create a better product.

I recently had to evaluate some data one of my coworkers had produced, to decide whether some process was going to be feasible. It would have taken probably a couple of weeks for me to dig through it all by hand, so he whipped up a quick little interactive web app that let me explore the data with all the connections and context visible, and I reviewed it much more quickly. A single-use application for a single user - what a luxury! - and it took less time to create than the time I saved by using it.

I will check in Monday morning on the result of an A/B test I set running over the weekend, comparing a reimplementation of a certain tool with its original. The test spins up AWS boxes, pulls code repos, compiles them, runs analysis, recovers from errors and maintains a retry queue, etc., etc., and ultimately collates the results and generates a report. I didn't write a line of this! I wouldn't have written anything nearly so sophisticated, and the results I'd have gotten would have been far less useful. But here we are: I tell the computer "make it so", and I get a really valuable test which runs itself while I enjoy my life, and we'll be able to switch to the new tool with confidence.

"it's now feasible to create all kinds of experiments, test fixtures, visualizers, analyzers, and support software you never would have had time for before, which you can use to improve your process and create a better product."

Yes, yes, yes, so much yes. Log analyzers, consistency checkers, performance evaluators, all sorts of hooks and helpers, precisely of the "single-use application for a single user - what a luxury!" sort, which made the entire development process so much more secure and comfortable. Most of the tedious moments (looking for a single bad value in a heap of logs etc.) went away.

The scaffolding I have now is a castle. The scaffolding I had a year ago was a log cabin. Nothing against log cabins in general, but castles are a lot harder to break into and have fewer holes for problems to sneak in.

Reply on news.ycombinator.com