Hacker News Reader

> My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype)

That only works for certain type of simpler products (mostly one-man projects, things like web apps) - you're not going to be building a throw-away prototype, either by hand or using AI, of something more complex like your company's core operating systems, or an industrial control system.

I can’t speak to OS development but industrial coding there’s a lot of experimenting and throw away. You generally don’t write a lot of code for the platform you’re building on (PLCs, automation components). It’s well tested and if it doesn’t hit industry standards (eg. timing, braking) you iterate or start over. At least that was my experience.

When it comes to general software development for customers in the everyday world (phones, computers, web). I often write once for proof, iterate as product requirements becomes clearer/refined, rewrite if necessary (code smell, initial pattern was inefficient for the final outcome).

On a large project, often I’ll touch something I wrote a year ago and realize I’ve evolved the pattern or learned something new in the language/system and I’ll do a little refactor while I’m in there. Even if it’s just code organization for readability.

> to rapidly build a crappy version of something so that I could better understand it, then rework it

I do this, too. And it makes me awful at generating "preliminary LOEs", because I can't tell how long something will take until I get in there and experiment a little.

In my experience, the only reliable LOE estimate is from someone who just touched that feature or someone who scaffolded it out or did the scrappy prototype in the process of generating the LOE.

A formalized form of this is the red-green-refactor pattern common in TDD.

Self created or formalized methods work, but they have to have habits or practices in place that prevent disengagement and complacency.

With LLMs there is the problem with humans and automation bias, which effects almost all human endeavors.

Unfortunately that will become more problematic as tools improve, so make sure to stay engaged and skeptical, which is the only successful strategy I have found with support from fields like human factors research.

NASA and the FAA are good sources for information if you want to develop your own.

This is what I came to comment. I'm seeing this more and more on HN of all places. Commenters are essentially describing TDD as how they use AI but don't seem to know that is what they are doing. (Without the tests though.)

Maybe I am more of a Leet coder than I think?

In my opinion TDD is antithetical to this process.

The primary reason is because what you are rapidly refactoring in these early prototypes/revisions are the meta structure and the contacts.

Before AI the cost of putting tests on from the beginning or TTD slowed your iteration speed dramatically.

In the early prototypes what you are figuring out is the actual shape of the problem and what the best division of responsibilities and how to fit them together to fit the vision for how the code will be required to evolve.

Now with AI, you can let the AI build test harnesses at little velocity cost, but TDD is still not the general approach.

And you have to pay special attention to the tests written by LLMs. I catch them mocking when they shouldn't, passing tests that don't actually pass, etc.

There are multiple schools to TDD, sounds like you were exposed to the kind that aims for coverage vs domain behavior.

Like any framework they all have costs,benefits, and places they work and others that they don’t.

Unless taking time to figure out what your inputs and expected outputs, the schools of thought that targeted writing all tests and even implement detail tests I would agree with you.

If you focus on writing inputs vs outputs, especially during a spike, I need to take prompt engineering classes from you

> With LLMs there is the problem with humans and automation bias, which effects almost all human endeavors.

Yep, and I believe that one will be harder to overcome.

Nudging an LLM into the right direction of debugging is a very different skill from debugging a problem yourself, and the better the LLMs get, the harder it will be to consciously switch between these two modes.

I agree, and even moreso, it's easy to see the (low!) cost of throwing away an implementation. I've had the AI coder produce something that works and technically meets the spec, but I don't like it for some reason and it's not really workable for me to massage it into a better style.

So I look up at the token usage, see that it cost 47 cents, and just `git reset --hard`, and try again with an improved prompt. If I had hand-written that code, it would have been much harder to do.

> My pre-llm workflow was to rapidly build a crappy version of something so that I could better understand it, then rework it (even throw away to the prototype) to build something I now know how I want to handle.

In my experience this is a bad workflow. "Build it crappy and fast" is how you wind up with crappy code in production because your manager sees you have something working fast and thinks it is good enough

Sounds more like a problem with the manager than with the workflow per se?

Maybe, but even so workflows like this don't exist in a vacuum. We have to work within the constraints of the organizational systems that exist. There are many practices that I personally adopt in my side projects that would have benefited many of my main jobs over the years, but to actually implement them at scale in my workplaces would require me to spend more time managing/politicking than building software. I did eventually go into management for this reason (among others), but that still didn't solve the workflow problem at my old jobs.

My experience as well; once it is working, other things take priority.

The question is, will the ability of LLMs to whip out boilerplate code cause managers to be more willing to rebuild currently "working" code into something better, now that the problem is better understood than when the first pass was made? I could believe it, but it's not obvious to me that this is so.

The trick is not to show anybody the prototype, especially your manager.

Oh, man, I've been there. It's worse if sales sees it.

I've been there. It gets even worse if the customer sees it.

I wouldn't be surprised if customers are easier to convince, than sales people, that the prototype is not suitable for deployment.

The old "oh I just came up with that exact set of necessary and sufficient specs" in agile meetings.

Thats why you throw away the prototype.

That's also when you tell your manager: "this is just the happy flow, it isn't production ready". Manager will then ask how long that will take and the answer is that the estimate hasn't changed.

I'm not sure I understand the problem. Just don't publish the prototype.

I have a bad feeling that a prototype I wrote ~15 years ago is still being used by a multinational company... It was pretty crappy because it was supposed to be replaced by something embedded in the shiny new ERP system. Funnily enough the ERP project crashed and burned...

Nothing lasts longer than a temporary fix.