Hacker News Reader

Comment by theamk | Hacker News Reader

> Set up a nightly cron job that executes the prompt: fetch upstream changes to the <software> and rebase all local changes on top of upstream. Check that the software works as intended and replace the current version.

This sounds like hell. You have unreliable actor redoing the software every night, and every day there is a chance you wake up and find your workflow broken.

And no, "Check that the software works as intended" is not going to cut it, as AI are very, very good at obeying the letter but not the spirit of the ask. Yes, your diffs appear but they lack filenames. You've added this requirements to the prompt? Ok, filenames are back but they are font size 4, unreadably small. You want them well-seen? next week they become size 54, taking entire screen.. This is not a problem with regular AI development - you review the changes and test them a bit. But doing it day-to-day with no overview is just asking for trouble.

View on HN

I suspect the people advocating for this don't actually do this, its one of those things that you say because it sounds modern and up to date, and makes for a good sounding blogpost. The actual reality of any of what they're talking about is absolutely terrible

Even just the cost of this would be a disaster, especially once everything goes to token based pricing

Some people even claim that they don't commit the code into source control, only the prompts. The best part about the internet is that you can publish any random shower thought.

I don't even commit the prompts. I just commit the one prompt which says "do what the customer wants. ask when more information is required."

Around 2020 I started getting downvoted for predicting that while the most popular programming languages at the time were Python, Java, and JavaScript, or whatever they were, the most popular programming languages in 2035 would be English and Mandarin.

Looks more like 2030, after all. So the downvoters had a point.

I think having some real examples of this 2035 is a lot more feasible now than it would have looked to most people in 2020, but the majority of projects in 2030 is absurd. Can you name even one piece of software today that is actually built this way?

This means:

Every release, the only thing done is they edit a prompt and then ship the build that comes out of LLM -> CI/CD ?

Then they delete all the source that was generated, along with the context, and only the prompt is carried forward to the next release?

What I'm talking about has only become possible in the last few months. Within a year you'll start to see new products built this way. Within five years you will see very few new products built any other way. Today's source code is tomorrow's object code.

Been doing it for the past several weeks. It is no longer correct to say that LLMs can't/don't/won't exercise reasonable aesthetic judgement on the finished product. Immense progress in this area has been made recently.

If you want the model to maintain a certain look and feel, or to preserve the UX in other ways, tell it. Even if it doesn't get it 100% right today, the direction of progress is pretty clear, so you might as well get used to prompting the models accordingly.

This doesn't appear to relate to the conversation at all, because we aren't talking about generating entire products. Is this just generic AI advertising?

I'm responding specifically to the GP's point: 'And no, "Check that the software works as intended" is not going to cut it, as AI are very, very good at obeying the letter but not the spirit of the ask,' and specifically to your rejoinder ('I suspect the people advocating for this don't actually do this.')

This is simply not the case when working with Fable-class models. Those who persist in believing it are hopefully very close to retirement.

I'm not 'advocating' anything, just stating a fact.

Author here. FWIW, I do this with my Shelley modifications. It works.

> "Check that the software works as intended" is not going to cut it, as AI are very, very good at obeying the letter but not the spirit of the ask.

Surely you don't think that he literally meant to use those seven words as a bullet proof prompt, right?

I will never understand this way of responding literally and pedantically to verbal psudocode. He's leaving it as an exercise to the reader to specify "check that the software works as intended" because he could never possibly properly articulate a perfectly comprehensive QA and validation flow for every piece of software that would satisfy every individual's need.

It's still the same problem. Now instead of checking if the code is correct, you have to check every day if your prompt is still validating the correct features that you expect to work.

And my point is, no matter how advanced you are going to make that prompt, AI will find a way to mess it up. If you say, "A, B and C must be true", then eventually an agent will find property D which you did not thing about, making the program unusable.

Again, this is normal if you are interactively working on software and can fix any problems you see. It is only problematic if you do what OP says, and make it on cron, daily and unattended.

If your coding agent workflow isn't sandboxed, backed up, and rollback-able, it's fundamentally broken. And yes I realize most of us aren't working this way today. But I think we'll have the tooling pretty soon. There's currently a cambrian explosion of solutions in this space.

I don't know; I've been rebasing huge branches in really mean divergent states with Claude Code, and it handled it flawlessly pretty much every single time. So I don't think the actor is as unreliable as you think.

I don’t auto-rebase but I definitely vendor in and choose to update on my own cadence.

yeah, vendor in is the only way to stay sane over time, if you understand the cost of dependencies (and have a reasonable security model). And forking some Python or Go dep on Github and using that instead of the canonical one is pretty ergonomic. I don't know how Rust folks manage with so many dependencies via Cargo - my friend who loves Rust says they are smaller and more "one thing done well" but the complexity of hundreds or thousands of deps just scares me.

My number one problem with Rust (which I otherwise like) is that Cargo allows build-time code execution so I need to run in a sandbox (unergonomic for me) or provide agent instructions to inspect `build.rs` style code in libraries etc. prior to compile. In practice it's easy to build and then ship a dev binary into a sandbox, but it's not so pleasant to build in a sandbox, so I don't want to get build-time pwned. Makes me very unhappy.

Right now, I just have a build server that I ship things to and does lots of build-caching etc. so it's fine, but I would have preferred not to have a system that actively allows build-time thievery of my ~/.ssh

Workflow being broken is the least of your worries. This is a security nightmare.

> This sounds like hell. You have unreliable actor...

But, at least you know the cost up-front.

Oh, wait...

Reply on news.ycombinator.com