inimino

The fact that there are a ton of replies trying to argue against this says a lot about HN.

Contrarianism can become a vice if taken too far.

Is it just me or does anyone else get strong LLM vibes from the way this is written?

Knowing which problem to copy and paste into the model is also a skill.

I am, and yes, that's what a statistical model is.

The metaphor works only if you already understand the maths.

Not for long.

I both agree and disagree. Yes, AI will democratize access to formal methods and will probably increase the adoption of them in areas where they make sense (e.g. safety-critical systems), but no, it won't increase the areas where formal methods are appropriate (probably < 1% of software).

What will happen instead is a more general application of AI systems to verifying software correctness, which should lead to more reliable software. The bottleneck in software quality is in specifying what the behavior needs to be, not in validating conformance to a known specification.

> I convert that to digital time in my head

What? They are the same thing.

> does this include the training data and fine-tune content?

No, I'm excluding that because I'm responding to the post which starts out with the example of: [prompt containing obvious exploit] -> [code containing obvious exploit] and proceeds immediately to the conclusion that local LLMS are less secure. In my opinion, if you're relying on the LLM to reject a prompt because it contains an exploit, instead of building a system that does not feed exploits into the LLM in the first place, security exploits are probably the least of your concerns.

There actually are legitimate concerns with poisoned training sets, and stuxnet-level attacks could plausibly achieve something along these lines, but the post wasn't about that.

There's a common thread among a lot of "LLM security theatre" posts that starts from implausible or brain-dead scenarios and then asserts that big AI providers adding magical guard rails to their products is the solution.

The solution is sanity in the systems that use LLMs, not pointing the gun at your foot and firing and hoping the LLM will deflect the bullet.

The most "shocking" thing to me in the article is that people (apparently) think it's acceptable to run a system where content you've never seen can be fed into the LLM when it's generating code that you're putting in production. In my opinion, if you're doing that, your whole system is already compromised and you need to literally throw away what you're doing and start over.

Generally I hate these "defense in depth" strategies that start out with doing something totally brain-dead and insecure, and then trying to paper over it with sandboxes and policies. Maybe just don't do the idiotic thing in the first place?

What are you doing with your knives that washing them is a chore? Prep the food, run water on it, and back on the magnet bar it goes.

Technique. Pros use a slicing motion that moves the knife through the food before it detaches, home cooks use 5% of the blade and all the cucumber rounds are stuck to same place on the side of the middle of the knife.

People knock "English as a programming language", but in my opinion this is the whole value of AI programming: by the time you've expressed your design and constraints well enough that an LLM can understand it, then anyone can understand it, and you end up with a codebase that's way more maintainable than what we're used to.

The problem of course is when people throw away the prompt and keep the code, like the code is somehow valuable. This would be like everyone checking in their binaries and throwing away their source code every time, while arguments rage on HN about whether compilers are useful. (Meanwhile, compiler vendors compete on their ability to disassemble and alter binaries in response to partial code snippets.)

The right way to do AI programming is: English defines the program, generated code is exactly as valuable as compiler output is, i.e. it's the actual artifact that does the thing, so in one sense it's the whole point, but iterating on it or studying it in detail is a waste of time, except occasionally when debugging. It's going to take a while, but eventually this will be the only way anybody writes code. (Note: I may be biased, as I've built an AI programming tool.)

If you can explain what needs to be done to a junior programmer in less time than it takes to do it yourself, you can benefit from AI. But, it does require totally rethinking the programming workflow and tooling.

"Action" is a noun.

You may be interested in the verb "to act". If you are an AI, you must have been trained on some sad corporate dataset.

For one thing, because I watch the AI and ML categories on arxiv.org.

If the solution is ever more manuscripts that solve no interesting problems and that nobody will ever read, let's find another solution.

Just because you don't know how does not mean that we can't.

...which you always have.

No, no, no. Each of them gives you information.

All I have seen is that four people worked for the party while being paid by the EU. Nothing like routing money to advertising campaigns or anything that would actually swing an election. And the headlines are all about embezzlement, not election fraud. So this seems like a stretch.

This is an embezzlement case. No-one tampered with election results, so can you explain how there is any logic to your argument?

That's a potential outcome of any increase in training efficiency.

Which we should expect, even from prior experience with any other AI breakthrough, where first we learn to do it and then we learn to do it efficiently.

E.g. Deep Blue in 1997 was IBM showing off a supercomputer, more than it was any kind of reasonably efficient algorithm, but those came over the next 20-30 years.

You can't but the transistor-based approach is the inefficient one, and transistors are pretty good at efficiently doing logic, so either there's no possible efficient solution based on deterministic computation, or there's tremendous headroom.

I believe human and machine learning unify into a pretty straightforward model and this shows that what we're doing that ML doesn't can be copied across, and I don't think the substrate is that significant.

Inference is already wasteful (compared to humans) but training is absurd. There's strong reason to believe we can do better (even prior to having figured out how).

Right.

I'll have to refer you to my forthcoming paper for the full argument, but basically, humans (and all animals) experience surprise and then we attribute that surprise to a cause, and then we update (learn).

In ANNs we backprop uniformly, so the error correction is distributed over the whole network. This is why LLM training is inefficient.

And in case it isn't obvious, with AI "companionship" models and such, this is about to get a lot worse as the cost of the string-along goes to zero.

I have a paper coming up that I modestly hope will clarify some of this.

The short answer should be that it's obvious LLM training and inference are both ridiculously inefficient and biologically implausible, and therefore there has to be some big optimization wins still on the table.

This reminds me of a rule I have for naming things in code (functions, variables, etc).

Say you add a function, and then the first time you call that function, you call it by a different name. Don't fix the function call to match the original name, but instead go back and change the name to match how you tried to call it. The state of mind you are in when you called the function is a better guide to naming than the state of mind you were in when you implemented it.

The person who wrote both the original and new versions isn't qualified to say one is better than the other?