Hacker News Reader

Comment by dntrshnthngjxct | Hacker News Reader

If you do not plan out the architecture soundly, no amount of prompting will fix it if it is bad. I know this because my "handmade" project made with backward compatibility and horrible architecture keeps being badly fixed by LLM while the ones that rely on preemptive planning of the features and architecture, end up working right.

View on HN

LLM's keep messing up even on a plain Laravel codebase..

I think that's true, but something even more subtle is going on. The quality of the LLM output depends on how it was prompted in a way more profound than I think most people realize. If you prompt the LLM using jargon and lingo that indicate you are already well experienced with the domain space, the LLM will rollplay an experienced developer. If you prompt it like you're a clueless PHB who's never coded, the LLM will output shitty code to match the style of your prompt. This extends to architecture, if your prompts are written with a mature understanding of the architecture that should be used, the LLM will follow suit, but if not then the LLM will just slap together something that looks like it might work, but isn't well thought out.

This is magical thinking.

LLMs are physically incapable of generating something “well thought out”, because they are physically incapable of thinking.

Tell Donald Knuth that: https://www-cs-faculty.stanford.edu/~knuth/papers/claude-cyc...

It is magical thinking to claim that LLMs are definitely physically incapable of thinking. You don't know that. No one knows that, since such large neural networks are opaque blackboxes that resist interpretation and we don't really know how they function internally.

You are just repeating that because you read that before somewhere else. Like a stochastic parrot. Quite ironic. ;)

They really aren't that mysterious. We can confidently say that they function at the lexical level, using Monte Carlo principles to carve out a likely path in lexical space. The output depends on the distribution of n-grams in the training set, and the composition of the text in it's context window.

This process cannot produce reasoning.

1) an LLM cannot represent the truth value of statements, only their likelihood of being found in its training data.

2) because it uses lexical data, an LLM will answer differently based on the names / terms used in a prompt.

Both of these facts contradict the idea that the LLM is reasoning, or "thinking".

This isn't really a very hit take either, I don't think I've talked to a single researcher who thinks that LLMs are thinking.

I don't care if the machine has a soul, I only care what the machine can produce. With good prompting, the machine produces more ""thoughtful"" results. As an engineer, that's all I care about.

You're just strawmanning now. I've prompted extremely well-specced, contained features, and the LLM has failed nonetheless.

In fact, the more details I give it about a specific problem, the more it seems to hallucinate. Presumably because it is more outside the training set.

because you need to consider the context window, thus separate the prompts by task. Separating by tasks and planning things out is still your own work, no AI can replace that. assuming you do that properly, AI-generating the code may save you up to 15% of your full work time. Please reread my comment: "If you do not plan out the architecture soundly", planning includes breaking the task down and make multiple prompts.

Our job is to break problems down into simpler ones until they are easily solvable, and if a machine simplifies the last steps, it is fine.

Reply on news.ycombinator.com