Hacker News Reader

> I can do things while the machine works

I find this workflow makes my life extremely tedious, because reviewing and fixing whatever the machine produces is a slog for me

I suppose it would be exhilarating if I just trusted the output but somehow I just can't bring myself to do that

How do you reconcile this with your own work? Maybe you just skim the output? Or do you run it and test that way?

Please don't tell me you just trust the AI to write automated tests or something...

In my experience, the machine works for 10-30s. I think the only productive use of time during those gaps is meditation. Still however, flow state cannot be achieved. Vibe coding, for me is pretty horrible for anything that might approximate production code. It is useful for quick prototyping or used as a learning tool. I assume it will get better but a far more satisfying experience would be for it to work offline, creating PRs for incremental improvements that are easy to review. I suppose this is something like the "Devin" model but I haven't used this due to the price and have heard bad things about it.

Just like magic, OpenAI codex is now available in my ChatGPT+ subscription. I greatly prefer this async style workflow. For me, getting a useful PR is so much better than banging my head against the wall with synchronous tools like Windsurf.

> I think the only productive use of time during those gaps is meditation

Check out several copies of the repo and work on different branches concurrently.

I suspect the context switches would kill productivity. This is not where the puck is going in my opinion.

Not GP but

> I find this workflow makes my life extremely tedious, because reviewing and fixing whatever the machine produces is a slog for me

This is a matter of personal preference, but reviewing code (from humans) was already a huge chunk of my job and one that I enjoy. Now I can have a similar workflow for side projects, especially for things that I find less enjoyment in coding myself.

> Please don't tell me you just trust the AI to write automated tests or something...

Like most tools, the more I use LLMs for coding the more I get a feel for what its good at, what its okay at, and what it will just make a mess of. I find for changes like extending existing test coverage before refactoring, I can often just skim the output. For writing tests from whole cloth it requires a lot more attention (depending on the area). For non-test code, it very much depends on how novel the work is and how much context/many examples it has to go on.

Create stubs to ensure code structure roughly aligns with expectations, then tell the AI to generate code and paired tests and look at code test coverage and manually inspect the tests rather than inspecting the code directly.

> I find this workflow makes my life extremely tedious, because reviewing and fixing whatever the machine produces is a slog for me

I don't actually mind the code review process, it's what I'm used to with open source contributions. Not infrequently did I look at a PR that people left me and decide to rewrite it entirely anyways because I did not quite appreciate the method they chose. That however did not mean the PR was useless, it might have provided the necessary input that made me go down that path ultimately.

I also think that what can make code review enjoyable is not necessarily because the person on the other side learns something from it. I actually tend to think that this something that does not necessarily happen in PR review, but in 1:1 conversations in person or though more synchronous collaboration.

So really my answer is that I do not mind code review. That said there is an important part here:

> How do you reconcile this with your own work? Maybe you just skim the output? Or do you run it and test that way?

What makes agentic workflows work is good tooling and fast iteration cyclces. That is also exactly what a human needs for a PR review to be actually thorough. I cannot accept a change when I do not know that it exhibits the intended behavior, and to validate that behavior I usually need a test case. Do I need to run the test case myself? Not necessarily because the agent / CI already does for me. However I do sometimes run it with verbose output to also see the intermediate outputs that might otherwise be hidden from me to give me more confidence.

It really is hard to describe, but when you put the necessary effort in to make an agentic workflow work, it's really enjoyable. But it does require some upfront investment for it to work.

> Please don't tell me you just trust the AI to write automated tests or something...

I do in fact trust the AI a lot to write tests. They have become incredibly good at taking certain requirements definitions and derive something that can be mostly trusted. It does not always work, but you get a pretty good feeling for if it will or not. At the end of the day many of the tasks we do are basic CRUD or data transformation tasks.

> It really is hard to describe, but when you put the necessary effort in to make an agentic workflow work, it's really enjoyable.

I wish it wasn't hard to describe because every attempt I've made to reach that point with agents/tools-in-a-loop has ended with more frustration, more errors, and more broken code than when I started, even with common and small, tedious tasks. I'd very much like to understand what proponents are doing and seeing that works.

It would be interesting to watch a video with an example of effective work. To see why are we so unlucky with results:)

All too often with AI proponents I get the you’re holding it wrong vibe. I can’t help but develop the suspicion these people are putting out bad work and just never notice. Or to be a little less scathing, fine, there’s a very narrow range of tasks where these models can usefully alleviate drudgery—quite often that comes at the cost of code bloat, since a generative model inherently makes it easy to puke out code with total indifference to its burden over time.

there was literally a post a few days ago where someone from cloudflare or fly (can't remember) released a new feature built with AI (albeit heavily supervised) and answered q's along the way about it.

it's funny because what i'd also like to see is people who are skeptics make a video as well since sometimes i also have the opposite suspicion. i get a lot of the criticism, but i don't get the "it produces pure garbage" type ones.

My comment was admittedly a little inflammatory, so I’ll try to elaborate on what I find so exasperating about the whole experience of using LLMs for coding (in any form, chat, agent, etc.): at root, it’s their total lack of a model of computation.

This leads to mistakes that are difficult to catch, because while I know the kinds of mistakes another human might make, having been through the process of learning not to make them myself, LLMs produce whole classes of bizarre mistakes that I have no interest in learning to catch. There is no discernible flawed mental model behind these errors—which with a human could be discussed and corrected—just an opaque stochastic process which I can tediously try to set on a better course with ‘incantations’, attempting to dial in to a better part of the training data that avoids the relevant class of error.

It’s honestly amusing how much of ‘prompt engineering’ (if it can be dignified with that term) amounts to a modern-day kind of mysticism. What better can we really do though when these models’ structure and operation is utterly opaque, on the one hand through deliberate, commercially-oriented obfuscation, and on the other because we still just cannot explain how a multi-billion parameter model works.

It’s rewarding to work with human juniors because people actually learn and improve. My learning how to coax these models into producing better than trash just is not, especially on anything either remotely novel, or in a legacy codebase that requires genuine understanding of how an existing system functions at runtime.

Once those two ends of the spectrum are ruled out, I find there’s little left that an LLM can accomplish, without necessitating a loop of prompt refinement that leaves me feeling like a worse developer for not just having thought through the problem myself, and resentful of the time wasted.

Edit: this entire thread about coercing Gemini into behaving is exactly the kind of crap I have zero interest in: https://news.ycombinator.com/item?id=44194061

it's really interesting to me because i feel like legacy code is where it actually excels very well. i have a large mental map in my head, so i usually just feed it some vague notion of what i want it do and use existing files as a reference ("i want to build this feature, look at X, Y, Z for reference. approach it this way" if i have some notion of how i want it go).

i usually don't let it go run off on its own unless it's a very defined task that i can review quickly later, i just review and approve every change and it takes big cognitive load off for me. at some point maybe this doesn't feel like "programming", but then i'll just tweak something else or modify it and then go onto the next review. i find i can't have it produce the entire thing and then review it since i have no idea how it got to where it did or takes just as much time to understand. but doing it this way it's faster + i gain understanding.

the prompts aren't overly complex or take a lot of time, certainly way less than speccing something out for a junior. all i have a is a base file for style and structure and then i describe the general problem and reference files ad-hoc. where i find it actually fails a lot is in novel code because it has nothing to ground it and starts exploring random stuff. i only use it for novel exploration to see what approaches it comes up with.

still trying to understand why there's this huge chasm between the two viewpoints. like a lot of the things you just said i can't resonate at all with. like maybe the 20% i feel like i'm "fighting" the LLM i just stop and go in myself. does that suck? sort of, but it's certainly way less tedious than directing some other person to do it or the time saved had i not used it at all.

edit: but to your point, yeah it really is just like magic with no way to like actually direct it in a way you would where a human would learn. maybe over a year ago i tried and wrote anything AI off beyond basic co-pilot completions (same issues, "fighting" the AI, having to specify a tons of exceptions in some god awful file). the new agents changed everything for me, esp claude code. i think it will only get better, so it's best to pick it up.

my only fears are

1) no juniors being trained, thus no future seniors. part of the power is that you have experienced people using it to enhance their context or understanding. for those with no experience and no drive to "improve" (honestly, think of the avg dev at big co) or straight up "vibe coding" i shudder at the output.

my hypothesis we are now going to enter a period where a LOT of shitty code is going to be created. it's already happening in education with people just cheating w/o learning. i already had issues trying to hire people who were using AI to get past initial exercises but failing on complex issues because they were just probably copy and pasting everything. best time to be a nimble startup.

2) top-down mandates to use this stuff. you should only use it when you want and if it helps you. i think there's this element of companies buying into the hype 110% and that puts a bad taste in everyone's mouth. "all devs replaced in 1 year!" type stuff.

Hmm, I should say that in the legacy case, it’s only been with legacy I’m already (painfully) familiar with, and knowing how difficult that code has been before to disentangle or to carefully stack another card on the already teetering house, I know upfront it’ll be a waste of time to let the LLM try.

In legacy that I don’t yet understand, maybe I am missing something in using a model with all that code in its context as an aid to build my understanding. I just cannot imagine handing off responsibility of modifying that code to the mystery machine. Tedious as it might be, I’m firmly of the view that I’d do myself and my team a disservice if I introduce changes I don’t fully comprehend, in fear that it’d only bite us later.

That fear comes from a couple cases of being bitten by changes made by other supposed seniors where I had my suspicions they let the LLM do the work for them, and went against my judgement in accepting their assurances that they’d tested everything worked.

Although, OK, for the legacy case, perhaps I should loosen my embargo in legacy frontend code where there’s a tight enough ‘blast radius’—meaning, the damage of a bad change is constrained to the bit of UI in question not working. Especially if that’s back-office frontend code where I have responsive users who I know will let me know if I broke something (because they surely know the warts of the system inside out, unlike me). In mission-critical backend code? Not a chance.

On the question of ‘fighting’ the LLM, maybe I do need to loosen up in how I want something done. In fairness I’m much more tolerant of that in a human, because a) there’s just often more than one way to do things; and b) with some devs I know there would or could be a fight that just isn’t worth it.

Which does come to an interesting point about ego and ownership, that if I regard the LLM code less as ‘my own’ perhaps I’d be more forgiving of its contributions. Would honestly make a difference if it’s not got my name on the commit.

Also comes back to one of the cases where I got burned. If the other dev hadn’t put their name on code which I’m sure was not theirs, then we could have had an honest discussion about it, and I could have better helped figure out whether what seemed to work was really trustworthy. Instead I had to weigh challenging them on it, and the awkward implication that I didn’t think they could have come up with the code themself.

To your two fears, totally agree, and undoubtedly some variant on these two cases has put a very bad taste in my mouth. Witnessing a junior essentially cheat their way through their final project for school, making a mockery of the piece of paper they got at the end. Being a victim not quite of a top-down mandate—somehow worse, with an exec head-over-heels bought into the hype, thinking they could lose a chunk of expensive headcount to no ill effect; not firing people, just making the situation miserable enough that people quit.

I'm curious to understand the nature of what you're working on better.

Whether or not AI performs well is influenced both by the work you're doing and how experienced you are in it. AI performs better when the work is closer to mainstream work than novel work. It also performs better with lower level instructions, eg. being more specific. As for experience, there are two things: people with less experience get a bigger lift than those with lots of experience, or people wiht lots of experience get a lift by having AI doing the work they don't want to do, which is often unittests and comments or writing the bobdyllionth api endpoint.

I read your post the other day and appreciated the optimism, but I wasn't able to work out what kind of work you've been doing since leaving Sentry.

The issue about executive mandate is likely coming from the context of large corporations. It creates fatigue, even though the underlying technology can be used very effectively to do "real work". It becomes hard for people to really see where the tech is valuable (reduce cost to test ideas, accelerate getting into a new area, etc.) vs where it is just a BS generator.

Those are typical dysfunctions in larger companies w/ weak leadership. They magnified by a few factors: AI is indistinguishable from magic for non tech leadership, demos that can be slapped quickly but that don't actually work w/o actual investments (which was what leadership wanted to avoid in the first place), and ofc the promise to reduce costs.

This happens in parallel to people using it to do their own work in a more bottom up manner. My anecdotal observation is that it is overused for low-value/high visibility work. E.g. it replaces "writing as proof of work" by reducing the cost to write bland, low information, documents used by middle management, which increases bureaucratic load.

> Deep research and similar tools alone have helped me navigate complex legal matters recently for my incorporation,

On the flipside, it will make it even easier for corporations to use the legal system in their favour and corporations will more easily and effectively use GenAI against individuals even if both have it due to the nature of corporations and their ability to invest in better tools than the individual.

So it's just an arms race/prisoner's dilemma and while it provides incremental advantages, it makes everything worse because the system becomes more complex and more oppressive.

I don’t understand these people that say they are getting no value from LLM. I’m getting value from them probably every other hour.

Simple, I bother to subtract all the time LLMs have wasted by giving me fake leads and lying to me from the time it saves when it successfully generates a few lines of code

This right here. Every time I talk to someone claiming 2-4x multipliers, after a few rounds of conversation they eventually admit "yeah you have to learn when to stop when it's no longer productive, you get a feel for when it's veering into wasting your time".

It can't be both. It can't be 2-4x multiplier _and_ be wasting your time to the extent that you have to "get a feel for when it has been wasting your time".

Having intermittently given in to the hype and tried whatever is supposedly latest and greatest, it without fail goes like this. I’ve even on occasion deceived myself into thinking I’ve finally hit on the right way to use these tools, only to realize in retrospect that under a complete accounting, the time spent to reach that productive mode of working in no way generalizes, so must be paid effectively every time.

Sometimes I try to do stuff with AI. Results are mixed. Sometimes it's somewhat helpful. Sometimes it's a complete waste of time. Maybe that means I'm holding it wrong. In the end, it's probably a wash for me. Maybe slightly positive.

I don't understand these people that are saying they are getting huge value from LLMs. I haven't put a ton of effort into figuring out how to "hold it right" because stuff is changing so fast still. I've had bad experiences with rabbit holes like this where the true believers will tell me the pot of gold is always right around the next corner. If and when I can get some positive value out it using whatever's easily available on the default, then I might investigate deeper.

My experience mostly consists of GPT and Copilot, as provided by my employer. I'm part of a pilot program to evaluate Copilot. My feedback is that it's slightly positive in agggregate, but individual results are very mixed. It's not worth much to me.

Basically his article says: check the documented issue with the climate, the education, the destruction of trust in society, and your answer is "yes but it makes my nice cuddly programming job better I like it"... don't you understand the issue?

I responded to quite specific quotes which I took issues with, and I intentionally did not talk about the rest of it. I did not make a statement about the rest because it's too complex of a topic to which I do not have enough data to contribute meaningfully to the conversation.

> I did not make a statement about the rest because it's too complex of a topic to which I do not have enough data to contribute meaningfully to the conversation.

> There is probably a lot of truth in the tail risks of all of that, and I share these. It just does not at all take away from my enjoyment and optimism at all.

On the one hand, I get not expanding on things you don't feel you can contribute to meaningfully and respect that.

On the other, isn't this a problem?

While I agree with you and cannot fully understand the author's conclusion that it's all bad (I've also found value), I wrestle with the downsides on an ongoing basis and have found myself shying away from using these tools as a result, even though I know they are quite capable of some things. I think that if I only responded "Well, I personally find the tools valuable", that this is only a part of the picture and is a net negative to the overall conversation because the "real" conversation that needs to be had IMO is about those harms, not haggling over whether or not the thing can actually do stuff.

There's quite a lot of pro-AI content and opinion floating around right now that amounts to: "Yeah, this thing is dangerous, threatens climate goals, is accelerating an education crisis, only exists because of mass theft, creates a massive category of new problems we don't yet know how to solve...but I'm getting value out of it". And the thing that keeps coming to mind is "Don't Look Up".

> There is probably a lot of truth in the tail risks of all of that, and I share these. It just does not at all take away from my enjoyment and optimism at all.

...is just coded "white rich guy" speak to say "I won't suffer the consequences, so I might as well not bother"

shameful

But DON'T YOU UNDERSTAND THIS IS THE ISSUE RIGHT HERE??

The fact that you're so sheltered as a western white rich programmer that you can AFFORD to be misinformed about all the way AI is bad for the rest of us (spoiler: you won't have to suffer it because you're sheltered)

That's exactly the issue: rich white guy pushing for "more, more, more" of their toys, without looking to the bad consequences of them because "oh no, I'm not interested and I'm not an expert"

Then you might as well keep taking the airplane every weekend, etc, and just "not get interested" or "not get enough data" about why it's bad...