Hacker News Reader

Comment by dwb | Hacker News Reader

I’m certainly on the lookout for something like this and I’m happy to see your account has published software from before the LLM boom as well. I guess I’d like some kind of LLM-use-statement attached to projects: did you use an LLM to generate this, and if so, how much and what stages (design, build, test)? How carefully did you review the output? Do you feel the quality is at least what you could have produced by yourself? That sort of thing.

Not casting aspersions on you personally, I’d really like this from every project, and would do the same myself.

View on HN

I’m sorry this sounds a bit too entitled - no one is putting a gun to your head to use this project and you know you can always read the code and review it yourself and make an educated decision on whether you want to use it or not

I don’t think I’m entitled to what I suggested, I don’t see how you’re reading that.

it's free code. if you're so concerned about the quality, you can read it yourself.

And if anyone is so put out by my suggestion (that I said I'd hold myself to as well), they can just ignore me and move on. I posted a comment on a public forum about what I'd like, I'm not the boss of anyone.

There a huge difference between suggestions and demands. I posted a comment about the tone of your comment you can also ignore it.

Statements that start “I guess I’d like…” are not demands.

Weird. The project is obviously holding itself out to be some amazing awesome new thing. Extraordinary claims at least invite questions like this.

Or are you all suggesting we should be comfortable with, and never question, flowery unchallenged advertising copy?

This is a fair question, but not one I feel we can let people self answer.

I doubt many people will honestly admit they did no design, testing and that they believe the code is sub par.

It does give me an idea that maybe we need a third party system which can try and answer some of the questions you are asking… of course it too would be LLM driven and quite subjective.

> I doubt many people will honestly admit they did no design, testing and that they believe the code is sub par.

Well no, but if people want to see a statement like this, and given that most people will want to be at least halfway honest and not admit to slop, maybe it will help nudge things in the right direction.

> I doubt many people will honestly admit they did no design, testing and that they believe the code is sub par

I'd doubt any engineer that doesn't call most of their own code subpar after a week or two after looking back. "Hacking" also famously involves little design or (automated) testing too, so sharing something like that doesn't mean much, unless you're trying to launch a business, but I see no evidence of that for this project.

There are many ways to use an LLM to generate a piece of software. I base most of my projects these days around sets of Markdown files where I use AI first to research, then plan and finally track the progress of implementation (which I do step-wise with the plan, always reviewing as I go along). If I was asked to provide documentation for my workflow those files would be it. My code is 99% generated, but I take care to ensure the LLM generates it in a way that I am happy with. I'd argue the result is often better than what I'd have managed on my own.

Yep pretty much same, although if I’m lax at any point of the reviewing (in-progress or final), I’d say the quality quickly drops to below my average manual effort, and then I don’t even have the benefit of thinking it all through as directly. I think getting really quality results out of LLM code generation for non-trivial projects still needs quite a bit of discipline and work.

What's the point? You can make good or bad software, with or without LLMs. Do you ask a carpenter if they use a hammer or nail gun? Did they only use the nail gun for the roof and the deck?

If you care that much and don't have a foundation of trust, you need to either verify the construction is good, or build it yourself. Anything else is just wishful thinking.

We do ask whether it's handmade or factory.

We even ask when cakes are made in house or frozen even though they look and taste great (at first).

It’s not all-or-nothing: a statement like what I want would be part of the assurance, not the whole thing.

It's meaningless though. It's performative virtue signalling with no assurance. Some people will produce great/poor quality regardless of the tools used. For ever vibe coder barfing up spaghetti, there's an engineer using the same tools to enhance their craft.

If you want assurance, look for other signals IMHO.

Code quality is indeed a virtue that I would like signalled and partly assured, through the means of the author feeling the social pressure of performing an explanation of their use of LLMs, and wanting to not appear careless. In my direct experience, the way you use these tools has a big effect on the outcome, and that’s the information I’d like conveyed here. Clearly it doesn’t cover people who would have produced rubbish anyway, but it isn’t intended to be complete.

Fair enough, we all have our preferences and opinions. What signals would you be looking for to make your judgement call?

> it doesn’t cover people who would have produced rubbish anyway

How do you determine this though?

If it becomes common for people to include an AI usage profile, what does that really do? I can say I followed best practices, steered design, supervised the LLM, reviewed the code, etc... but those are just a stranger's words with little value, and my code might still be garbage.

If you keep saying things that don't hold up, that tends to harm your reputation. Do I really need to explain basic social dynamics? Sure, maybe I'm wrong and this totally wouldn't work. Maybe (probably) not enough people think it's a good idea and it never happens. Maybe everyone just decides to lie through their teeth. Maybe there's no significant relationship between how carefully you use an LLM and the quality of the product. But I thought it was worth suggesting.

I get the social dynamics, and I agree that we need more signals to build trust, even if I disagree on this particular signal.

I think enhancing SDLC visibility could be valuable, which is somewhat related to your original suggestion, though I'm not sure how to validate something like that since workflows and tests tend to be unique. I suppose we have badges like 'build:passing' that aren't standard, but are somewhat adopted and show that some extra effort went into setting that up. It would be nice to have something a bit more standardized and verifiable though.

Anyway, I appreciate your engaged responses, and even if we don't see eye to eye on the LLM signal, I think we both want the same thing, which is increased trust in third-party software. I hope we get there someday, even if it seems we're moving in the opposite direction right now.

Reply on news.ycombinator.com