Hacker News Reader

Comment by mortalapeman | Hacker News Reader

With generated code, the directory structure, interface design and general state management is usually a haphazard mess. Even with the best frontier models. But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt. Subtle things like which error states are "oh shit we need to bail" vs "this isn't a deal breaker." Sometimes it will ask, but more often than not it will just make a decision and it's often the wrong one. If I don't have a fully kitted out test suit and a good type checker to verify the final product against, the the whole looping thing is just useless to me and I'm back to reviewing every line of code it puts out and having to draw on my years of architecture experience to make sure we don't build a giant pile of trash.

View on HN

Because they are designed to be used by managers who don't know how to answer these questions and don't want to be asked them. Just have the magic answers box pick something.

They're RLHF'ed to an inch of their lives to be able to one-shot complete tasks, since requiring human input defeats the purpose of being able to replace the labor force.

But once the insanity ends LLMs will be packaged as tools for developers to use to boost their productivity, and we'll consider them as we do IDE's and debuggers and stuff. But we have to get through this hype cycle first.

wake up, slopinthebag. wake up..

You think LLM’s will be more than a tool?

It's about agentic systems who view humans as their "tool". Much like the companies who created them. Just read the system cards of the new frontier models how those behave without strong guardrails. And remember that we already have only a generation behing capable models in the wild without guardrails

Do you think that you can continuously train LLMs for more and more complex and more and more agentic tasks without them becoming "more than a tool"?

The generated code is fine at the functional level, the directory structure is usually the standard pattern for the given type of project.

The error types and codes, it will produce to spec.

If you type 'make me that thingy' - yes, it's probably not going to do what you want, but if you give it spec and guidance, it usually will.

The 'interface design' ... not very good though.

At the point you fully define your spec and behavior, what you have is code.

Telling the app how to handle errors doesn't specify the code.

The AI will handle a lot of the common, in-between issues.

The more framework you can give it, the more it can fill in blanks.

> If I don't have a fully kitted out test suit and a good type checker to verify the final product against

I have a template I use with generated python projects that sets up all the test info first.

https://github.com/brian-learns/testafize

I start the coding session in a directory where I've already set this up and maybe drafted a README.md, and usually it just automatically starts running the `make check` without prompting.

I've also used it for migrating old code. I put an old python 2 script in src and asked it get make check to work, and it even added test on its own initiative.

I'm mostly using Qwen3.6 (now Qwen3.7) running on my DGX Spark in llama.cpp using different harnesses, or the default model on opencode.

> But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt.

This is a problem with your instructions, your specification. An LLM isn't a mind reader. It will attempt to succeed regardless of missing requirements and ambiguity.

But a big part of software engineering is literally defining the requirements.

> With generated code, the directory structure, interface design and general state management is usually a haphazard mess. Even with the best frontier models. But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt.

As always it's a usage problem, specifically you aren't pre-feeding it a large enough sample of reference code that it can use for context. I have a custom functional DSL that I prefeed the BNF + about few thousand lines of code and it has no problem writing it, in almost the same style as the reference. Literally point your model to a reference repo as say "read this entire repo and replicate the codestyle + directory structure exactly".

> But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt.

When the LLM writes a description of a codebase, you are doing transformation from more semantic meaning to less semantic meaning, and you by definition lose some details.

When you go from requirements to code, transformation is from from less semantic meaning to more semantic meaning. The the missing semantic meaning has to be invented. When inventing the missing semantic meaning, the success depends on what the LLM has trained on.

Surely this is a solvable problem.

If the average, mediocre software developer can address the issue of directory structure, interface design, general state management, edge cases and subtle assumptions it should be possible to train AI systems to address these issues .

Software development is not some mystical magical activity.

I remember people making similar arguments about autonomous driving...

> directory structure, interface design, general state management,

These are currently problematic due to context length. You should have working memory of the whole codebase layout to decide on a sane architecture, identify opportunities for deduplication/consolidation, etc.

> more often than not it will just make a decision and it's often the wrong one.

Let's not forget these chatbots rely on a random number generator to pick output options.

It absolutely hates code that would crash or error in any circumstance. So it adds a ton of dubious fallbacks.

Crash means zero reward, so yeah.

Reply on news.ycombinator.com