Hacker News Reader

Comment by asdff | Hacker News Reader

You’d think ai models would be good enough to write code in pure binary. Python is for humans. Take the human out there’s no point in it.

View on HN

It's possible that one day they'll be. But to me it's very obvious why LLMs are better at writing human-readable code than binary code.

The hype around LLMs come from the fact that you can command them in natural language. To make it possible, LLMs have to be trained mostly on natural language corpus. And human-readable code is closer to natural language than binary code.

With all due respect, I think that's a terrible idea for a couple reasons.

First, a model that produces binary code would be impossible to vet. This is already an issue with "trusting trust" in compilers [1], and would be made worse in the case of companies that constantly tweak their black-box models however they like. At least my compiler can work offline.

But then there's the idea of programming as "turn data into data" that forgets that software has to interface with humans too. If I ask my LLM to give me sentencing guidelines that are "fair" given a set of parameters, no one on Earth should trust a binary they cannot check themselves. In fact, those affected may even have a constitutional right to source code.

[1] https://dl.acm.org/doi/10.1145/358198.358210

Imo you wouldn’t even have to check the binary. Only that the output is what you expect and the performance is reasonable. No different than say agriculture where we measure success of a new varietal on output and resource use and have a very dim understanding of the inner workings that might have led to those observables.

Or to take it further, you hire a consultant to produce some work in a given amount of time. You accept the work over time, you don’t feel the need to ask the consultant to turn over all their notes or to plug their brain into a mind reader machine to vet the source. They could indeed be lying and take whatever they learned about your company elsewhere, but we have no protection over that possibility lacking a mind reading instrument today. And society also doesn’t collapse today given that risk.

You think something which gets 70% of stuff right on a very good day when the planets are all aligned can write assembly that works? I have news for you.

I think it sure can. Randomly modify a binary function over a million potential permutations and measure the distribution of output, subset from the distribution the binary function(s) that output ahead of your threshold. Should give you a fast function that does what you want with absolutely indescribable code, which isn’t an issue if it does what you want with reasonable performance.

I suppose the issue is a data problem, there being relatively little high quality data explaining how things should be solved in binary. As such making the learning mapping between prompt (english) and good solution (binary) difficult.

The data is easily generated by compiling the code.

But compiled code loses a lot of the "extra" data. Also these are "language" models so I would be surprised if training on binaries was much more efficient versus writing in some kind of language.

Besides, how do you even check the result now without running untrusted code? Every run of the model you need to reverse-engineer the binary?

You don’t even need a training set imo just enough permutations to get a distribution of putative functions and outputs.

Reply on news.ycombinator.com