Hacker News Reader

Another way of putting it is that predictions based on an assumption of "exponential growth" of anything (which is presumed to slow down at some point) are almost always wrong. It's saying, sure, we may be at 1 right now, but in a year we'll be at 100, and that's the future we should prepare for! But if you believe that, then if you're off by only a little about the stopping point, we could just as likely have to prepare for a future where we're at 10 or 1000 for a while.

Once you talk about exponential growth of some capability, by definition a small error in the timing of when that growth plateaus translates to an error of the plateau capability by an order of magnitude. It's likely to be be much smaller or much greater than your prediction. In other words, you can believe in exponential growth and you can believe you have a good prediction of the future, but you can't believe in both simultaneously.

Yegge believes that "the future" will arrive by end of year. If he's off by only a few months in one direction, such a future may not arrive for a while; in the other direction, and "the future" will be quite different.

I have to agree, I cannot quite pinpoint the fixation on LLMs doing away with software engineers/developers. I know teachers who rely on ChatGPT and Claude to do so many parts of their jobs; marking work, providing feedback for a kid's homework, lesson planning and even answering subject matter questions they don't know how to. LLMs work so much better so many professions, right now, than it does for software development.

Clearly, even assuming they are wrong far less often and don't hallucinate in the very near future, there is a human element that LLMs certainly can't replace (unless semi-autonomous robots become a thing soon). Why isn't the same thought process applied to writing software? It's obvious if you've worked in this industry for any amount of time just how paramount that human element is to producing useful programs.

If current and/or near future AI will replace us, then it's surely replacing almost everyone else, no?

There are two ways to answer your questions. You are asking how do we choose between (1) generate+run (AI generate software for some task, then we run the software to do that task) and (2) agentic execution (AI simply completes the task).

First way to look at this is through the lens of specialization. A software engineer could design and create Emacs, and then a writer could use Emacs to write a top-notch novel. That does not mean that the software engineer could write top-notch novels. Similarly, maybe AI can generate software for any task, but maybe it cannot do that task just as well as the task-specialized software.

Second way to look at this is based on costs. Even if AI is as good as specialized software for a given task, the specialized software will likely be more efficient since it uses direct computation (you know, moving and transforming bits around in the CPU and the memory) instead of GPU or TPU-powered multiplications that emulate the direct computation.

> Similarly, maybe AI can generate software for any task, but maybe it cannot do that task just as well as the task-specialized software.

Yes, maybe, but assuming that is the case in general seems completely arbitrary. Maybe not all jobs are like writing software, but why assume software is especially easy for AI?

> Even if AI is as good as specialized software for a given task, the specialized software will likely be more efficient since it uses direct computation

Right, but surely an AI that can "build pretty much anything" can also figure out that it should write specialised software for itself to make its job faster or cheaper (after all, to "build pretty much anything", it needs to know about optimisation).

Yes, closely related to the question why AI services insists on writing software in ineffective high level languages intended for humans to read, such as Python. Which then needs to be compiled, use large standard libraries etc. Why not output the actual software directly, intended for a computer to run directly?

Same reason humans write in higher-level languages instead of machine code? Each additional unit of program text costs energy at write time, so there's a bias toward more compact _representations_ of programs, even if they're less efficient at runtime.

Because the written code, variable names, structure, and comments also serve as context for the LLM.

This is why LLM written code is often more verbose than human written code. All of those seemingly unnecessary comments everywhere, the excessively descriptive function names, the way everything is broken down into a seemingly excessive number of logical blocks: This is all helpful to the LLM for understanding the code.

Maybe for existing models, but I don't think that's necessarily the case. AI tools that generate or manipulate images or video don't need human words in the format. Also, I'm not sure how much meaningful names help beyond a certain program size. Assembly is quite understandable locally - even for humans, but especially if you're an AI model trained specifically on it - and globally, even codebases in high-level programming languages are too large to grasp just as code.

> AI tools that generate or manipulate images or video don't need human words in the format.

AI image manipulation tools use the image as context just like text tools use text for context.

You can get an LLM to modify assembly if there’s enough context about what it’s doing, what the input means, what the output means, and so on provided manually.

A chunk of assembly by itself is missing a lot of context that is naturally embedded in higher level programming languages, though. Like variable names.

But a coding agent is free to write any relevant context elsewhere, where it is not necessarily constrained by the syntactic requirements of a programming language. When we write Assembly, we also tend to write comments. It's not like people can't program in Assembly. Most console games in the eighties were written in Assembly, for example. I used to write programs in Assembly in the early 90s.

The difficulty with Assembly is local reasoning, like register allocations (which was particularly annoying when we had few registers, some of which were specialised) and jump targets, which doesn't require a lot of context, but is still tedious compared to C. Global reasoning is pretty much identical to C.

AI doesn't "want to" control air traffic. It doesn't have any desire or ambition. That's what the humans are for.

It is merely a tool like a hammer. The hammer doesn't build the house, it is the human who wields the hammer that builds the house.

I didn't say it wants to do anything. It also doesn't want to build software. But why would it be the case that an AI told to build an air traffic control software could successfully do that, but an AI told to make sure planes arrive where they're supposed to safely and on time won't be able to figure out the rest?

Now, I'm not saying it's impossible for there to be something that makes the first job significantly easier than the second, but it's strange for me to assume that an AI would definitely be able to do the former soon, yet not the latter. I think it could be reasonable to believe it will be able to do neither or both soon, but I don't understand how we can expect the ability line to just happen to fall between software and pretty much everything else.

This somewhat falls apart the second you realize that current models are already choosing which tools to use all the time. You can argue that's not "desire" but I'm not sure you'd convince me.

Frankly - even the other end of your argument is weak. Humans don't particularly want to control air traffic either (otherwise why are we having to pay those air traffic controllers their salaries to be there?). They do it as a function of achieving a broader goal.

I’ve heard this called “technical deflation” and it works similarly to how economic deflation can play out, causing you to forego actions in the present because you think they’ll be easier in the future (or in this case, possibly not needed at all). Time will tell if this results in a “software deflationary spiral” or not.

That was my first question. Somehow people thought that a tool able to generate an entire fullstack app from a few sentences is not capable of inferring needs based on context or customer directly.