Hacker News Reader

Comment by thornewolf | Hacker News Reader

Write a prompt, evaluate the prompt, understand that is succeeds 95% of the time.

Write a new prompt, evaluate, it now succeeds 99% of the time. Measure what changes between prompt #1 and prompt #2, understand what contributed to the performance jump.

Write a third prompt, this one succeeds 100% of the time. Increase the size of your evaluation set, find a 1/5000 error-class and a 1/10000 error-class, add some explicit code to correct for this cases.

Roll out to production, collecting usage metrics. You make some tweaks to your harness, your prompts. Eventually you have confidence that your system has fewer mistakes than 1 in 100k.

Now, multiply this iteration across all your different prompts and different ways that they might interact with one another.

View on HN

There's a reason engineers are prissy about people coming along and saying "I write code, I'm an engineer" that people periodically try to sand-paper away.

Engineers don't just tie a sheet to a rock and throw it off a cliff and call themselves aerospace engineers.

They do full diligence on the theory, math, physics, material science, fluid dynamics, etc, and plan a controlled series of tests specifically designed to verify/challenge/disprove their concept and the theories behind it.

Sure, there's a team member ultimately responsible throwing half a dozen rocks off a cliff in the first test.

A technician.

The guy who throws the rock off the cliff is a technician.

The other glossed over part is that the above sounds like science.

Engineering often continues until the concepts and theories are developed into safe, practical methods. "If you stay within these parameters, you can confidently expect these results." The reliability can be codified and reproduced without going from first principles on every application of it.

It's not clear to me that the current AI fad is really developing such reproducible, safe methods. "If you stay within these parameters, you might get these results. Or a teapot. Or some subtly misleading fabrication."

You have to do full due diligence to validate every result. There is safe usage where the hard work was done up front so that day to day practice can skip to boring and reliable application.

To a software engineer, a (current) LLM is a stateless algorithm that performs an idempotent transformation on a large numeric input.

People who think it's a system that thinks and reasons have confused the agentic harness, perhaps forgotten(?) layer0[0] is a seed, the inference engine sets to a concrete value when the caller leaves it as 0.

They probably work on (current) AI software by repeatedly writing prompts like "DON'T READ THE FILES IN /tmp. SOME OF THE FILES IN /tmp ARE VERY LARGE. DUE TO THEIR SIZE, YOU ARE NOT TO READ THE FILES IN /tmp." and wondering why the model becomes obsessed with files in /tmp 100k tokens into every conversation.

It's not engineering if you're just guessing as to what is degrading the performance and what might improve it.

If you can identify gradient (what direction your change will impact the ultimate goal), then just repeating the process (or reverse-process) can find local maximum.

Still it can be a software engineering if the gradient candidate / measuring gradient / repeat process can be done at scale.

Referring you back to this evergreen comment:

https://news.ycombinator.com/item?id=44978319

"Most classical engineering fields deal with probabilistic system components all of the time. In fact I'd go as far as to say that inability to deal with probabilistic components is disqualifying from many engineering endeavors."

This is cope and fundamentally misrepresents engineering. Engineers deal with a problem space that is probabilistic (although they try to model it as best as they can), but design solutions in a deterministic space. With LLMs the solution space itself is non-deterministic.

The GP didn't wrote "guess" and "eyeball", but "measure", "evaluate", "understand".

Reading comprehension 101 is a prerequisite for doing engineering, too.

> It's not engineering if you're just guessing as to what is degrading the performance and what might improve it.

Engineering is literally the art of making educated guesses and then testing/proving/disproving/improving upon them. Nothing is exact. Everything is approximate. Iterate until the result is good enough.

This is false. A bridge is not built with approximations, it is built with a deep understanding of structural physics. Yes there are some unknowns, no it's not "educated guesses".

You call it a guess, I call it a hypothesis.

Engineers do not make hypotheses, we build solutions to problems.

Hypothesis Driven Development would beg to differ.

> understand that is succeeds 95% of the time.

more like 10%.

> Write a new prompt, evaluate, it now succeeds 99% of the time.

in theory, maybe. In practice still 10%.

> Write a third prompt, this one succeeds 100% of the time

Only in Sam Altman's wet dreams.

95% is shit tier engineering. Would you be satisfied if your keyboard randomly failed 5% of the time.

There's actually more than one line in the comment you're replying to.

Things like Voice to Text and biometric unlocks (fingerprint scanners, face ID) have worse success rates and they're used every day by billions of people.

Voice to text and biometrics are noisy sources, so a big part of the problem is dealing with that noise.

Typing is not a noisy source. It should be reliable and deterministic.

Protecting an agent from fairly obvious attacks should also be deterministic.

Typing is absolutely a noisy source.

The fundamental issue is, that "we" somehow decided it would be a good idea to throw all the fundamental ideas of computing (determinism, context, separation between data and execution,...) away and try to solve the issues by running a probabilistic/stochastic word generator on top of deterministic circuits instead at 10 magnitude worse efficiency.

It's not a fundamental issue. Determinism and "separation between data and execution" are artificial constructs, make-believe universe in which we design classical code, and a whole lot of hardware engineering goes into allowing us to briefly forget it's all fake.

Real world is probabilistic in practical / metrological, if not fundamental sense, and separation between data and execution does not exist. Our reality does not support such separation.

> a probabilistic/stochastic word generator on top of deterministic circuits instead at 10 magnitude worse efficiency

It's 10 magnitude better efficiency end-to-end, if you factor in design time you'd have to spend to get your "deterministic circuits" (which really aren't, we just paper over it) into shape so they deterministically solve a specific problem, for each problem you want to solve - where with the "stochastic word generator", you just need to change the text prompt.

A component can be dramatically less deterministic per execution while making the whole system dramatically more capable.

Reply on news.ycombinator.com