There's no reason for software to be slow anymore
danluu.com
danluu.com
Anyone not in the US feels this even more since so much online is US hosted, 300ms for every little interaction adds up quick.
If your software has the affordance of a waiting dialogue or loading wheel for many of its UI controls, you are building with this default blocked assumption. Even if you are building something web based, ask yourself if that's actually necessary for your software or if you could build it differently to avoid constant UI blocking.
* Must feel very responsive * Network requests can take up to 500ms end to end
This gave me a chuckle because I personally hate things like watching the browser jump through 50+ redirects when logging into a website.
Or just pirate the software and let them sue you for pirating, which you will win since you have a license, even though the computer thinks you don't.
I would guess Canva already does something like this.
This might seem wasteful, but if the web were content addressed instead of server addressed you could then be serving that cache to your municipality even after it became disconnected from the rest of the internet. Which sort of recasts it not like wastefulness but instead like fault tolerance and preparedness.
We could maybe even dispense with the servers entirely.
There are so many different ways to build a web. Why does it feel like we've landed on the worst possible one?
Welcome to Next.js
The .js part makes me think not.
<script type="speculationrules">
{
"prefetch": [
{
"source": "list",
"urls": ["/checkout.html", "/thank-you.html"]
}
]
}
</script>I should be able to get the lay of the land without trusting the site enough to blindly execute whatever code it points me at. It's needless attack surface.
Also it's not really pointing me at data, its pointing me a certain kinds of requests which I have to trust will be responded to consistently. I'd much rather have a hash so if I have that data lying around I can just forgo the request entirely and use what's present locally.
This is already possible without content addressing with CDNs. They can serve content from a local cache even when the host is disconnected from the internet.
1. You need write access to the server if you want to add one
2. The server could change its behavior at any time and there's no way to know that caches now need to be invalidated
3. If something goes wrong with connectivity or name resolution, there's no fallback since the authoritative thing was not something durable like a trusted human via a public key but rather an ephemeral thing: a named server which has pinkey promised to stay online.
It asks the user to treat a server like a trustworthy source of perisisant data.
But there's no reason to couple these kinds of trust. The skills necessary to persist and traffick data are orthogonal to being trustworthy about content. Coupling them creates needless load on single sources of failure which are simultaneously single points for corruption to target.
Trust people, not servers. Use digital signatures to validate that what you're seeing came from those people.
<a> tags are the opposite of this. They encourage us to trust servers by name, which isn't really working out.
I don't think the current crop of fullstack engineers would be hard pressed to know what a "markov chain" is, but in theory yes, you could emit a bunch of speculation rules[1] based on your predictions.
I should also say that markov chain based approaches have been used for fraud detection, e.g. identifying checkout anomalies by detecting the sequence of web pages that they clicked on, amongst other factors.
[1] https://developer.mozilla.org/en-US/docs/Web/API/Speculation...
https://nextjs.org/docs/app/guides/prefetching#hover-trigger...
In my view, websites should not take seconds to load with gigabit fiber. Whatever happened to "mobile first"?
Funny thing is it apparently predates JSON. It's a bunch of data[foo][bar] = baz; - go look.
(Website's in German obviously, and a surprising number of German electronic terms are very different from English. They use two different words for stranded and non-stranded wire.)
Try turning off wifi and see if it improves.
I have been using the proprietary NVIDIA drivers for more than 2 decades on various hardware, both desktops and laptops, mostly with Gentoo Linux.
Opening an OpenGL context or any other OpenGL operations have always been instant.
In similar style I recently wiped a device that I thought had firmware that was slow to boot but it turns out that a hang and subsequent timeout due to something I had long ago misconfigured had been obscured by the previous setup that defaulted to hiding all details during boot.
Obviously agent access goes away with internet failure but the state doesn’t need to… we use CRDTs and a virtual FS. There’s a toy-ish version of the harness at https://ourhearth.ai … if local first is interesting to you I’d love your feedback
With a fast backend ~1-5ms response times (not even that fast). Streaming compression over something like SSE to keep your response sub 1kb packet (roughly an ethernet MTU).
With a push based model, pushing data to a user is half their RTT latency. They will only experience their full RTT on actions they trigger.
Now the network to you is distance to the server (not your rail/nextjs backend taking 400ms). Things like 4G and 3G are fine. The real problem is when you have such bad signal you effectively have no down or up.
When it comes to improving performance by a few ms, or implementing a new feature, business people will always choose a new feature, unless the current performance is unbearably slow (we're talking regular 1.5s+ wait times for BE response).
And it's not even a modern problem, legacy software written 20 years ago has the same latency than most modern backends from my experience.
Even though benchmarks suggest this sort of performance should be trivial, most real-world servers I have interacted with do not reliably managed to process a request, make a roundtrip to the DB, and return a response in <5ms
Hell, for a lot of projects you don't even need to get that fancy. Run a single server with an embedde database, running Go or Java and you're good to go.
Don’t know about that, pretty much everything is hosted on Cloudflare, Azure or S3 these days and all of them have at least one CDN on each continent.
But if I were to pick one single thing that would speed up the most UIs across the board, it would be poor handling of the UI in networked systems. As you noted, that doesn't mean eliminating them, it means handling the inevitable in a way that doesn't tank the UI feel.
Somehow I don't think I'm the only one who presses a button and when nothing happens presses it repeatedly until something happens, or I kill the app, or even power off whatever piece of shit computer I'm using.
I'm making a simple plugin for Gimp that sends the active layer to a model with a prompt; on Macs by default the UI waits for the request to come back or timeout; on PCs it doesn't, so the modified layer appears unexpectedly. The Mac experience is better IMHO and I will replicate it in the PC version rather than the other way around.
We've come full circle.
On top of that: Creating software that isn't outright wasteful in terms of performance isn't even hard, it's just a matter of not doing ridiculously dumb things. The problem I've observed in teams I've worked with is that the majority of programmers don't even know what the dumb things are, and wouldn't know how to even approach making something that's halfway fast.
Edit:
Unfortunately I think posts like these are only going to make the problem worse, because now people are going to ask for voodoo solutions to performance issues, when the answer to their problems was usually just "Maybe stop creating wasteful intermediate structures and just walk an array like a sane person" in 99% of cases. The first leg of any optimization journey in the average programmer's code will likely net tens or hundreds of times faster code, and that's actually all people were asking for.
The knowledge required to make those changes and understand them is fairly minimal, but the kinds of people who have to create spinners for webmail interfaces, have their application add 150ms on top of whatever round-trip you have for processing things counted in 5 digits, etc., have never bothered to even learn those things.
What you point out is slowness once you hit the entry point. Go, or similar languages, as a server language platform could have solved that problem from a computational perspective. But it did not for the most part. In my opinion people choose the faster stuff because it's cool and they have more wiggle room to cram in to get back to the slow status quo.
Everything is overengineered, software or distributed architectures, sound to naive human logic but alien to computers. It's an cultural problem, development is so deeply entrenched into "business logic" that the minimal viable and computational economic solution isn't even on the table. I don't even think it has to do with cost or feasibility, it's just that your random e-com manager wouldn't know what to do with you, if a programmer really starts talking about hardcode tech stuff.
Another friend, worked at a company, that got a new manager (I think as a result of a merger), and that manager halted all work on "yak shaving" projects. These "yak shaving" projects were things like logging, and debugging, and some kind of integrity-verification. When asked why they were being halted, the new manager said: "none of our customers asked for any of these things". When told that these things enable the team to produce a better product for the customers, the manager (I am told) looked at them with confusion and suspicion. Those projects were never improved since, and the product stopped improving as well. I am not sure if it affected their business (the pandemic was much more distortive).
What you call "business logic", is not even logic, and it has little to do with business. It is what Feynman called a "cargo cult". The obvious name for it is "cargo cult business management/logic".
It truly is embarrassing and shameful that after decades of idiotic decisions, it took a _trillion_[1] dollars of investment into a chat-bot technology, to finally crack open _one_[2] door to slightly less idiotic decisions, while opening dozens of new doors to decisions of an unknowable character.
Most companies (and, consequently, their engineering organizations) are simply _cosplaying_ as the things they are supposed to be.
I do not see how an AI assistant (or any kind of assistant or consultant) can save these fools from themselves. The only logical explanation is that most software companies are cursed -- you would have much better luck engaging a witch-doctor.
[0]: Nobody else knew C or C++ either. The fact is, that nobody cared. In fact, even Go would have been a better choice than raw Python, but nobody cared. Even Common Lisp (which is at least as abstract as Python, and has native execution speeds (GC and runtime type-checking can be turned off for compute-heavy workloads that mutate data in-place)), is a better choice, and yet, an _abundance_ of obviously superior options (all implemented and maintained by obviously superior engineers) was not enough to prevent the organization from choosing an inferior one, and using it stupidly (without a fast native-code component).
[1]: I see estimates from hundreds of billions to a trillion, depending on how you count it.
[2]: The performance door, if Luu is correct.
God forbid you say anything about it in a code review lest you get shouted down about premature optimization.
"This other unrelated thing in the code is slower, so writing this correctly is a poor use of time!"
Non-tautological rephrasing: When optimizing code, first actually just de-slowify it, and that should most often be plenty enough.
You are starting to get software with ticket driven development layered around glue code for existing libraries. I see no problem with libraries, it is just the architecture and vertical understanding that leaves a lot of performance on the table, because refactoring insanity takes resources and a lot of talking, understanding and convincing to be done.
Doing this across big teams starts to have downsides. So one team doesn't have a particular use case implemented or understood it and does not want to support it and you run code to compensate for this.
Best example in a monolith case is oracle...
I feel this is so much real. Like if there were two kinds of companies: the ones who deliver, they care about their product but most of all they care about their customers, the managers get their hands dirty and everyone pushes towards the same direction; then there are companies in which you open a ticket and wait for two weeks for something that should take 5 minutes, customers and product don’t matter because you’re focused in cost attribution and no body does anything if it doesn’t come in your JIRA board, the managers are all coming from consultancy companies and all they do is finding someone to blame.
"Do not fall into the trap of anthropomorphizing Larry Ellison. You need to think of Larry Ellison the way you think of a lawnmower. You don’t anthropomorphize your lawnmower, the lawnmower just mows the lawn - you stick your hand in there and it’ll chop it off, the end. You don’t think "oh, the lawnmower hates me" – lawnmower doesn’t give a shit about you, lawnmower can’t hate you. Don’t anthropomorphize the lawnmower. Don’t fall into that trap about Oracle."
“…the structure of a designed system mirrors the communication structure of the organization that built it”
The problem usually comes from inappropriately arranging the systems of record such that information needs to be communicated beyond the scope of one computer in order to satisfy a single logical request.
Moving information between physical processors tends to be significantly more expensive than local computation over that same information. JSON serialization is a really good example of this. You need a network with bandwidth in excess of 10 GbE to begin overtaking simdjson.
SSR or SPA doesn't really matter if the server still takes a minimum of 300ms to compose any kind of response due to how its database or other infrastructure is set up. Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful. Stringing the user along with psychological tricks is a lot cheaper than hiring a skilled developer to do it the right way.
> Information theory doesn't care how clever your loading indicator is. If the information isn't available, we can't do anything meaningful.
Yes we can - we can fix why the information isn't available. If someone said to you "sorry, we don't have the info because the other thread is doing Sleep(5000);" you'd call them an idiot right? You'd go and delete the sleep call to make it faster. Most real problems are harder than that, but there's no fundamental rule saying your database has to be slow. Ping time across your LAN is probably under a millisecond, so where are the other 299 milliseconds going? Is your database doing a full table scan? Is it using spinning rust for frequently accessed data?
As to your database comment, IME most of the time the bottleneck is the ORM and/or language. The amount of work an ORM does to generate a representation of a row is frankly shocking. Not understanding the cost of context-switching is the language half of it: Python, of course, is single-threaded, but you can use greenlets to cheat, because they’re I/O bound — except for all of them serializing behind a single process handling serdes for the queries.
I wish Atlassian listened to you.
second is on a smaller timescale. so many features and crap that is never asked for and never used is crammed into software so the systems that execute it are just juggling pretty much dead code in and stale data in their caches all days long.
The optimisations the article is talking about would help even with this problem if backends responded faster - although not as much as actually avoiding unnecessary network requests in the first place, of course.
Things like this really pushed me to everything local systems. I’ll move actual files around if I want to do anything on the network. Or sometimes even use cables! Shock, horror!
The original design had the speakers setting up a private mesh network, and the app would send commands directly to the speakers via your LAN. Then, they got the brilliant idea to route commands via their cloud service. The app would send commands to an endpoint, which would send them back to your speakers. Imagine trying to smoothly fade volume with a WAN hop. This went over as well as you’d expect, and they’ve since promised to work on performance. Thus far they seem to have been doing so; it isn’t as snappy as the original, but it’s quite a bit better.
Device developers, stop doing this! You people realize that LANs exist, don't you?
I argued breathlessly against it for days. I’ll never forget the sales chad raising his voice to shut me down with a cop-out:“This is the way the industry is going!”.
It’s not the developers making these changes.
Not saying that’s what you could or would have done, I’ve just gotten to the point in life where I’m alreaover it, ready to throw it back at them. Maybe it’s from living in this part of the world, but our plumber said “always be ready for people to be mean to you“ and while it doesn’t make for a very peaceful life, it certainly makes for fun ripping heads off.
Unpopular Opinion, but if you have absolutely zero say in the content of what you're developing, and just take orders from JIRA, you shouldn't call yourself an engineer. You should also keep your eyes open for a better job.
Again this is mostly our biases playing us. Within HN and similar communities there are a lot of above average and caring developers. Those who don’t care aren’t going to be here to look like they do.
- Access points that don’t let WiFi devices talk to wired devices.
- Office networks that don’t repeat broadcast discovery traffic into different segments.
- People in McMansions who don’t understand extenders and use two different SSIDs for each half of their house or for indoor vs poolside.
- Phone is on cellular network, speaker is on WiFi, “controls don’t work”
- Network teams who won’t “open the ports” or configure mDNS or Zeroconf or Bonjour but will let port 443 to the internet.
- people who want to turn their speakers on at home while they are at work, for their dog.
- people who want to turn their speakers off at their second home after they returned from holiday and left the pool party music playing.
- diagnostic logs that would be nice if they went somewhere.
- People who lost their phone and want a website on their laptop to control their speakers.
- people who have their speakers nicked and want to login to your website to see their speakers checking in and “remote wipe” block them.
- People who want the status of all their speakers live in their phone app, even the ones at other properties.
- tons of buggy low quality, low cost, network devices (switches, hubs, cables, routers, firewalls, NATs).
Just about the only thing which is used enough to be reasonably trustable on any random customer network is HTTPS to the big CDNs and Cloud providers.
It's one thing not to e.g. spend the extra time to ensure everything is cached and mutations are queued up. It's another thing not to do the bare minimum to ensure what is already available and working locally is gated on the network being up.
Case in point: The other day I was checking our train tickets in an app, and the network was awful, and the train tickets which the app has local copies of took 30+ seconds to appear when the network went down. Everything I needed worked once the timeouts had been hit, it was just ridiculously slow waiting for timeouts for functionality I wasn't trying to use to be hit first.
PING news.ycombinator.com (209.216.230.207): 56 data bytes
...
--- news.ycombinator.com ping statistics ---
40 packets transmitted, 40 packets received, 0% packet loss
round-trip min/avg/max = 154.980/155.832/156.990 ms
round-trip min/avg/max = 164.218/164.860/165.764 ms
rtt min/avg/max/mdev = 201.881/245.543/350.600/48.713 msWe don't turn faster hardware into faster programs, we turn it into more program. AI isn't going to change that. We'll just get even more program because the optimisation has freed up space for that.
Unfortunately most of the time, the more program isn't for our benefit. I note that by far the heaviest program I use is my web browser. The one thing I don't get to choose what code gets thrust upon me.
If I want to watch streaming videos, I don't have a choice about how I do that.
Fine Firefox is basically a bloated YouTube app. That doesn't change the fact that it is inefficient (from the pov of my CPU) for doing that.
After traveling around in places with very poor wifi/phone data speeds. I couldn't agree more with you.
It's infuriating when it's obvious that the developer of an app only ever tested it in a simulator on their dev machine on their super fast WiFi. It never seems to connect with those people developing a mobile app (or web app) that the "mobile" part has a meaning more than just on a handheld device.
We as an industry should use AI to enable a standard of software quality that was previously uneconomical.
The issue I think you’ll run into is that they’re lacking users and kind of just worse versions. Usually because people who say things like this, don’t actually want to be personally inconvenienced to switch.
If there's a connection to your services outside of the user's machine you can control all of that.
"In the Hypertext Transfer Protocol (HTTP) for the World Wide Web, servers insert a MIME header field at the beginning of any Web transmission. Clients use the content type header to select an appropriate viewer application for the type of data indicated."
A fan of "let's enable the program to do everything" philosophy I am not. This idea is embodied in the so-called "modern" web browser and a countless number of other "apps". Alas, this design, perhaps justified on "convenience" grounds (or so-called "user experience"), has been abused, e.g., for commercial purposes. One casualty of the abuse might be speed. Other sacrifices might be reliability, resource usage, "privacy", "security", etc. The most important sacrifice for me in using "do everything" software is _control_
Instead I use a number of small command line clients for making HTTP requests ("web requests").^1 I can edit the source code and compile these applications quickly with low resources
The clients are request makers, not response viewers. The historical "select an appropriate viewer application" step remains, as I prefer it
This software is not slow. I seem to avoid the dissatisfaction that I see from commenters who use software that can "do everything"
1. Generally this is one application that accepts URLs on stdin and generates HTTP on stdout and another that accepts HTTP on stdin, makes connections and sends it, typically a TCP client. But since I use a local forward proxy that has a built-in httpclient I don't necessarily need those programs to make requests, e.g.,
x=https://danluu.com/perf-opt/
echo "@1;expert-mode on;httpclient GET $x"|socat stdio unix:/path/socket
> A stochastic search process with an executable optimization objective over space of programs S can only maintain or improve the objective
This is superoptimization. We've known this since the 80s (Massalin, STOKE is more recent: https://github.com/StanfordPL/stoke) The only novelty is that the proposer is now way better with LMs.
Further, there's a large number of reasons for software written by agents to be slow:
- LMs still don't do data or hardware-oriented design well out of the box, and therefore if you're engaging in any sort of serious novel work, beyond porting an extremely well-understood program with extremely well-understood workloads, you're going to be spending hours tracking down bad allocation decisions (c.f. why TigerBeetle doesn't use agents), which are often the root of evil (before you'd reach for anything further)
- The knobs you'd need to get serious performance are nearly unreachable in languages which LMs are good at (even Rust requires a discipline that the default language doesn't enforce). When you drop into the lower realms, you're trading consumption context for access to these levers. The levers are also "soft": you find yourself writing a bunch of skills, and tools to try and enforce the discipline.
The reality is to get performant code (quickly) out of an agent, you need to know how to write performant code (and you need to know how to surface the information that you'd use to create a verifier for such a thing to the agent), which 99% of developers do not know in 2026.
Sure, agents can teach you how to do this -- but it's one of these things where iykyk.
Experience: I've poured 10s of billions of tokens into Zig with the best agents and I have the time and space to try these things.
If you want to start learning the discipline, I'd recommend matklad's + TigerBeetle blog -- as well as hardware-oriented design.
Hasn’t been my experience at all. The latest LLMs can knock out assembly optimized subroutines and benchmark 100 different variations faster than I ever could dream of.
To correct those behaviors, you're going to write tools and skills, and that's going to help, but it is still clear that you are fighting the distribution (today).
If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things, just code fulfilling what the specification says.
Granted, this does require you to not use obviously dumb models, like anything you can run locally today, and at least within reasonable range of SOTA models. But they been able to do this for 6 months or more at this point.
What is the most complicated thing you’ve built with LM agents? Have you done it with a single spec? How novel was it?
This comment is so laughably “you’re holding it wrong” I can’t respond to you seriously.
> If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things
The set of software that has followed this process is measure zero.
Bro, I don't know you, and have no idea what you do, except for the parts you share in your comment, like talking to a LLM :) If you say "if you ask an agent to write a compiler" and don't expand on that, I'll take your word for it.
I'm glad you got a laugh, and didn't take it so serious :) Not entirely sure why you got so defensive, but it's all good, again I don't know you, sorry if it felt like I called you stupid or something. At least some other people found it helpful I suppose, so in the end it sucks you're unable to get value from a process I've found working for me, hope you'll find something that works out for you in the end, regardless of what it is!
> If you instead spend a day writing a proper specification, then ask the agent to spend a week implementing that, you'll need zero tools and skills afterwards to clean it up, because there won't be any misunderstandings, assumptions or other things, just code fulfilling what the specification says.
Are you claiming that you reliably get good software out of agents using this process?
My experiential claim is that I've followed this process for complex pieces of software (and, often, taking significantly longer than a day -- more like several weeks, as if I was writing an academic paper) and they still get it wrong ... even with an "academic paper level spec"
Models: only the best, Fable 5 max, Sol 5.6 max, yada yada -- I'm not convinced this process works for "serious endeavors" (for instance: let's build a novel accelerator compiler or let's build a new incremental linker) where "serious" here means "something where I'll likely be spending multiple weeks working on it"
Yeah, for sure, I tend to be really strict about the overall design, keeping things minimal and only the abstractions needed, nothing more/less.
I don't know what "most complicated software" even is, I've only been writing software for 20 years or so, but mostly around startups, smaller companies and as a consultant/freelancer for companies of all sizes, so I won't even claim to know what "really complicated software" is. What I've done so far ranges from implementations of various ML models, native cross-platform applications, various types of games, simulation software and a bunch of tooling to help me with music production and VFX/animation/3D work.
Some of those I guess I wouldn't even have approached without LLMs, and probably would have taken me years if I did attempt, but following that process I basically end up with one spec, maybe 2K-3K lines in total, including step-by-step how things have to be implemented, then fire off Codex + GPT-5.3-Sol max with "/goal" saying exactly how things should be implemented, verified and so on, and update the specification file as it goes along, and not stop until everything from the specification been confirmed to be implemented correctly. Longest been 4-5 days until the goal is completed.
Always a lot of iterations afterwards of course, as I realize some things I put in the specification ultimately didn't made sense once it's implemented, so revise specification, figure out how to change, fire off "/goal", and repeat this until I have code that is better or at least equal to how I'd write it myself. It was a long time ago this process lead to something completely different being implemented, I usually get exactly what the specification says.
Of course, all I’m saying is that you need to refine your sample! For instance: the allocation architecture is not correct, and one has to run a bunch of performance investigations and resolve it.
My responses are intending to convey that I don’t believe this is possible, no matter how good LMs get — and it seems like we are in agreement.
If your goal with building this compiler was performance, and this wasn't part of the initial specification, and the agent didn't assume it had to, is this what you're saying is a failure on the agents side?
There is no distribution to fight, is my hypothesis at least, if you're just a lot more clear exactly what you expect up front. Hence the whole "To correct those behaviors, you're going to write tools and skills" thing isn't even needed in the first place.
I don't believe the problem is "ah, you didn't write the spec clearly enough" -- which is why I'm asking about your own experiences.
A story.
I have a friend, who is - like me - interested in the CRDT / collaborative editing space. He asked ChatGPT to write him a CRDT. Then he grabbed every good CRDT implementation, and asked chatgpt to benchmark and optimise his CRDT, using tricks and techniques from existing hand-optimised CRDTs. He got massive performance gains by doing this - which is really interesting! I think it helped that he had a clear objective function, and chatgpt could look at other projects for ideas on how to optimise.
He proudly boasted that his resulting code outperformed my diamond-types library. I asked him if he was comparing against the native implementation, or the -Oz webassembly build, running in a wasm vm. It was the latter. When he tested it properly, his CRDT was - and is - significantly slower than diamond types. As far as I know, chatgpt still hasn't been able to catch up. I tried myself using fable. Even with reference to my source code, Fable still doesn't understand what I did in diamond types and why. (... Maybe I should document what I did!)
I think his technique itself is solid though. I tried it myself. I asked fable to write a custom binary serialization format & parser. Then optimise. Then optimise, with explicit reference to existing libraries. Optimising with reference to other code made a huge additional difference. It is now nearly as fast as those libraries. (But still not faster than them.)
My takeaway is this: I think LLMs are exceptionally good at reading and understanding code. If you guide them to do so, they're good at profiling and benchmarking. But it seems like they're not very good at coming up with novel optimisations. If you have an obviously slow program (for example, some slop claude wrote), you can often get big speedups by asking it to benchmark and optimise. But if you have a complex, already well optimised codebase, like the zig compiler, claude doesn't seem very good at figuring out novel ways to improve things on its own.
This is good news for the 95% of slow software out there. But bad news for the 5% of us who write fast code already, but want our code to go even faster.
Of course that doesn't mean they have a human level mental model and associated novel ideas. Brute force can be effective but remains entirely unsatisfying from an academic perspective.
TBF as an earlier commenter noted it's "just" superoptimization using an LLM as the proposer so there's a lot of relevant prior art.
A story for a story. I had my CRDT implementation in libdog, which does per-token CRDT weave/diff/merge over a DAG of git blobs. It was written by Claude 4.8 I believe, in several iterations. It was, as you may guess, a piece of neuro-slop that passed the tests by some miracle. Once I had some time to look into it, I used a trick: I supplied it with my article on Chronofolds and some helpful kicks in the butt. It implemented everything correctly on its k-th attempt, k<5. Then I used it with full intensity for three months without thinking twice. Now I have started mass-using it to resolve permalinks in the code. Like, tens of files to chronofold-ize per one commit. It is now showing up in the profile, so I may look into it once more.
Conclusion: it sort of expands its context and prompt by association. It has no sense of direction of its own maybe. Once you know what you are doing, you can ride it fine. If not, it makes a misstep so later things go haywire, and you are left guessing why. (And how can you guess if you have not had the experience. We all grew with Commodores and suchlike. I recall Spectrums, Robotrons, and Poisks. My friend sits on exam committees, says the youth arrives flatlined after 3 years of GPT. His words. Whatever.)
This is a perfect description. Last week I asked claude opus to get AAC audio working in davinci resolve on linux. It managed to add aac in mov and mp4 containers very quickly. But mkv was another matter. For mkv files, resolve doesn't use ffmpeg. Instead, it has its own parser. Claude got totally lost down a weird rabbit hole trying to add aac support to resolve's mkv code. It was really struggling. Claude even knew it was lost - it kept telling me we should cut our losses and I should just release aac support without mkv.
Eventually I gave it the executable for davinci resolve on mac, which has aac support. Claude found the corresponding part of the code for the mac version and used it as a reference. Turns out, claude had made some much earlier mistake. Just like you said, it was going down a wrong path. Then it couldn't stop itself, and it kept making it worse.
Using the mac version of the binary as a reference, claude figured out how to get everything working very quickly. But - I'm left wondering. Maybe my real mistake was using Opus and not Fable. I wonder if fable would have been smart enough to figure out the mistake and course-correct.
> Maybe I should document what I did!
Please do!
Me and a bunch of friends worked on a project that used diamond-types as the backing CRDT engine. It does indeed go brrrr. But damn, it took way too long to reconstruct what it was doing (we needed some more fine grained knobs, so we were playing with the frontier directly). We eventually moved out to something a bit better documented, which was a real pity. I really liked the general architecture and simplicity (of the text-only based version at least).
Another possible confusing thing for an LLM is that getting close to max doesn't necessarily require any "tricks." A big part of getting in the ballpark is just not doing anything you don't have to. If program A is faster than program B, most of the time is not some magic algorithm. It's that program A just did less stuff.
Reasons this doesn't follow: (1) Benchmarks never match real world use, and many optimizations the improve benchmarks degrade cases that aren't measured (think about how CPU cache behavior can be surprising) (2) In software performance optimization, frequently there is significant noise, from many sources. This makes it difficult to guarantee that a measured change is actually an improvement.
https://github.com/eaftan/safere
https://eaftan.github.io/safere-intro/
Mine is for Java and is intended to be production grade. The first goal is to guarantee linear-time behavior to prevent ReDoS attacks. My collaborator and I have recently been optimizing it to try to surpass native RE2 in performance.
It turns out optimizations are incredibly well suited for an agentic loop. You've got concrete acceptance criteria (must show a meaninging improvement on a benchmark case, must pass tests). The agent is really, really good at using tools like a profiler and disassembler, better than I am (and I've been doing this for 20 years). It also papers over things that would take me a while to learn, like how the in-incubation Vector (SIMD) API works in Java. I understand the concept but it would take me a while to understand Java's implementation. The agent can just read the docs and go.
The key is creating a good benchmark suite and ensuring the agent doesn't ship optimizations that are too narrow or too focused on the benchmark cases. You also need a really strong test suite to make sure you're not regressing correctness. SafeRE has billions of tests; a subset of several million run on CI, and the others run on-demand.
My Java profiling knowledge is... let's call it "antique". I was really not looking forward to ramping back up for this work. Turns out, I didn't have to do any of it. The LLM chose the tools (flight recorder) and even built a JMH (also new to me) harness to experiment with different algorithms.
About half of the optimizations were things that I would have figured out on my own; the other half were definitely "wow" moments.
The whole thing was done in a couple hours, with just a few back-and-forths. Sans AI, it would have taken a week, with nowhere near the same gain. I'm impressed.
"Figure out how to make this process fast" is really a perfect activity for LLMs. And the prompt doesn't really have to be much more sophisticated than that.
do you have some samples? It would be interesting to learn what it might be.
* Using spherical points instead of trig to calculate distance between two geo locations.
* Packing data to minimize memory bandwidth consumption. Converting arrays of objects to multiple arrays of their component parts I sort of expected; bitshifting to pack and unpack multiple values into a `long` I did not.
Maybe other people would find these obvious, but I don't usually have to optimize at this level. My mental model of the relative speed of some CPU operations was a little out of date.
JMH as the framework to write microbenchmarks. It takes care of dealing with JIT warmup, etc. It's the standard way to write rigorous Java microbenchmarks.
async-profiler (https://github.com/async-profiler/async-profiler) for profiling. Java has a problem where many profilers are based on safepoints, which are biased toward particular program points. async-profiler is not biased in this way.
Java Flight Recorder for memory allocation data.
One thing I've observed in all of this is that it's really useful to have expertise in the programming language and ecosystem you're writing in, otherwise it's all Greek to you and you can't really guide the agent to do the right thing. I have opinions about e.g. profilers and I can point the agent to one that I think is more accurate than other options.
It works, it’s at least 5x faster, sometimes much more, and memory use is like 10x less and even less in cases where lots of map tiles are involved.
This shit rules.
What I’m wondering now is can we reliably evolve python and have codex act as an extremely unreliable transpiler to the rust.
Why you even start with Python at this point? Just write the Rust version straight up instead of porting things?
Personally I used to use dynamic languages for most things, because development and maintenance is so much faster and easier, particularly for larger projects (granted you know how to work with those sort of languages), but now when the LLM writes most of the code, I'm able to work as fast with Rust as with I used to be able to do with Clojure or other dynamic languages.
I mean what happens with network engineers when the biggest network of them all goes down?
One little thing I spotted is you use Boyer Moore Horspool for fast literal search. This is actually not linear in the worst case, although it is almost always sublinear. Worst case would be a literal composed of the same character searching a text of the same character, where it becomes quadratic.
You can actually search strings with character classes using Horspool if you want to, and I have some enhancements to basic Horspool which could maybe help. My library, byteseek [1], implements these.
I also have a much faster algorithm, HashChain [2] which also has a guaranteed linear time version. This was published in the Symposium for Experimental Algorithmics in 2024.
This will first massively slow down the Horspool scan, and then once you have done all that work, you rescan it all from the start with KMP if it is doing too much.
One little fix might be to only add to the work counter and compare it outside of the main character comparison loop.
But it would be better to use the linear version of Hashchain. It also uses KMP to make it linear, but it is fully integrated and you would not need to track the work or restart scanning at all. And its a lot faster than Horspool anyway!
This person doesn't understand how to make efficient code. I can write code in almost any language (with a couple of exceptions) that outperforms "super-optimized assembly". Writing efficient code isn't about the language, and often isn't about the best algorithms either (but sometimes it is). Its about optimizing memory and cache use. And that's orthogonal to anything the author is writing about. Also, LLMs are terrible at optimizing memory utilization. There is just too little training code that does it well and far too much that doesn't.
As proof, I'm can literally feel the web getting slower and I bet many others feel this as well.
The author is one of the most knowledgeable people about performance there is
If you actually know what you're doing in $language, and you know how $language wants to emit the assembly or whatever, there's no huge advantage to just programming directly in assembly.
The app and service are measurably and subjectively faster. Because I chose to have the LLM focus on solving those problems. It obviously can. It described the issues in big-O.
It is a priority problem, as it always has been, not a knowledge or skill problem, like it always has been.
I've already posted this elsewhere, but here it is again: https://news.ycombinator.com/item?id=49226923
A vibe-coded OS that runs on an 8088 with 256KB of RAM.
As proof, I'm can literally feel the web getting slower and I bet many others feel this as well.
"It's not the tool, it's how you use it..."
> The other day, I saw a viral tweet saying [...]
The points I take away are:
- Good optimization is difficult and slow work, hence expensive, but LLMs can do it so we should be able to afford it more often now.
- There’s always a risk of over-fitting to your specific problem, but if everyone is now making bespoke optimizations maybe that isn’t actually a problem.
You said:
LLMs are terrible at optimizing memory utilization. There is just too little training code that does it well and far too much that doesn't.
There’s probably something in that, but it can be mitigated by testing against a local benchmark. LLMs are good at iterating tirelessly and finding incremental improvements. And as noted above, it doesn’t necessarily matter if your benchmark isn’t fully general.
I’ve also worked on projects where the people in charge added 1mb client-side mapping libraries to render a static map and ignored my push back. Those website were slow
After crossing 40 years of age, and working for a while now, I believe it's also because of politics. You might think the goal is to deliver the best possible product, but territory grabs within companies are important, and done by people that don't have enough skill other than territory grabs. E.g. look at Trump and his behavior. No skills other than having his way, and then he gets to decide.
Of the technical reason, for sure we underestimate how much faster technology gets. Fred Brooks had this example in his Mythical Man Month book, how the os/360 got the option of a disk drive instead of mag tape, but the result was worse because everyone assumed it was much faster than it really was.
And that software is build by some of the highest paid software engineers on the planet with full access to all the LLM compute in the world.
Big, feature rich, built in quick iterations - but also, in particular if you look under the hood, of extremely poor quality if measured by traditional software engineering standards (code structure as exemplified by the leaked Claude Code source code, resource usage, "buggyness" etc.).
Toxic corporate culture can go a long way to self-sabotage.
I think the "maturity and humility" phase of the LLM hype cycle is probably still 2-3 years into the future.
And that phase probably comes immediately after the "LLMs overinvestments have caused a massive global recession".
Now look at what software they make. It speaks for itself.
Recently I was frustrated by Windows 11's seeming inability to open a context menu with acceptable speed - right click an item in the taskbar and there is nearly a 1000ms delay before the menu appears. That is unacceptable.
When I need to run old software, I now try to the "minimum viable runner" OS - start with an XP VM and slowly move upwards if it doesn't work. Obviously I'll lock it down from internet access/etc., but it really shows that modern OSes really don't have a grip on performance.
This idea is the reason why software will keep getting slower and less reliable. Because it's wrong and yet people believe it.
Not only do people believe this, even if they didn't, they would still have the problem that most people (including many engineers) are bad at telling who is a good software engineer.
So this means that code will get worse and worse; less performant, less reliable, less secure.
It will be impossible to fix because nobody would know what the problems are let alone identify who is able to solve those problems.
I don't know how bad it has to get before people understand we have to rebuild everything from scratch on much more rigid guardrails.
The essence of the problem is that the LLM does exactly what you tell it. In the hands of a skilled engineer who understands the project, this is a superpower. In the hands of a junior, this is dangerous... Because it will do things that are insecure; it doesn't push back on a user request; it just goes ahead and tries to make it work, even if it's a bad idea and the problem is best solved in a different way.
Not even close to being true. Prior to the advent of LLMs writing good SIMD was hard. Writing good SIMD for arbitrary tasks was _extra_ hard. Having the entire application be written with SIMD in mind was nigh impossible, from simple string parsing to data pipelining. Having arbitrary OpenCL/CUDA offloading to GPUs was a monstrous task. And even the people who were experts in these domains didn't have the time to do it in an afternoon.
But now you can. You can literally instruct your agent to write SIMD accelerated code everywhere, or to optimize down to it. And it just _does_.
> The essence of the problem is that the LLM does exactly what you tell it.
This is correct. I implore people to try writing a SaaS with an LLM, but instead of going the classic agentic project style of "design me an app", tell it to use C++ or Rust, use extensive SIMD intrinsics and inline assembly wherever possible. Just off those two instructions the difference in code quality is staggering.
Architectural problems are particularly difficult for it. There are many moving parts and many ways to implement so if you want certain properties like scalability and idempotent operations, then you need to have a very clear idea what to prompt.
The other day I used Claude to improve my Rendezvous hashing (with skeleton/tree) library and it managed to solve the problem. It's a pretty small library but it used up over 50k tokens and several hours to implement the optimization and it did a good job ultimately but I had written a lot of tests for it so it's not like it was implementing from scratch.
At one point it spend at least 15 minutes trying to change the code to make a test case pass to save a few meaningless percentage points of performance... After seeing it struggle with this, I stopped it and told it that it should just increase the timeout of the test case by a few milliseconds. Any engineer would have done this but Claude didn't want to take that initiative. Instead, it was starting to over-engineer the solution just to dogmatically meet that test case instead of accepting the negligible loss of performance.
Maybe if this philosophy / approach has a catchy name it might be more widely adopted and we all can have better more performant web and apps.
I somehow managed to avoid the entire SPA scene. I feel so vindicated because I am absolutely zooming with my tried and true Vanilla JS these days. I have tried React, Vue, Angular, and ever just plain TS, and I hate them all.
You can't exhaustively write down all the possible ways for software to execute, especially since many programs run forever (e.g. servers, GUIs etc)
So when you say in your spec "when the user clicks here it shows this stuff" there is the tacit expectation that this will always happen and isn't dependent on the entire execution path up to this point. You also wouldn't expect a server to just finish execution after processing a request because you forgot to write "finally, cede control back to the main loop" at the end.
These assumptions are the framework. Almost everything you read is likely written in a framework, unless you limit yourself to non-technical writing for younger readers.
You could try to formalise the framework in your specs. But you'll then end up copy and pasting these paragraphs every time you want another server program. These context frameworks will probably get names, just like today.
Or you could skip that entirely and pretend you're not using a framework, but you are. It's just that the most has already learned tons of frameworks. What worries me here is stagnation. There's no way for the model to learn new frameworks. But this might be a short term effect. Maybe software in the next few years will look quite odd to those in the future as we keep writing it in the language of frameworks we already know. Maybe they'll look back at these relics and ask "what is a view, and why did the model understand that?"
If only we had a sufficiently smart™ compiler, it could do the inkling, dead code elimination, and WPO across the application. (Oh we do have that, as long as you don’t write in JavaScript…)
That still overkill?
I find it decently fast out the box and possible to manage as it grows.
Contrary to what initiatives like the tigerbeetle team is doing with tigerstyle, or the 10 nasa coding rules, code created by llms tends to be verbose and slow.
If you want your software to be fast, stop writing it in Python and TypeScript and instead write it in Go, Rust, C++ or C. But nobody's going to do that, because humans are emotional creatures who get attached to things (like programming languages). Or if you want your software to be fast, stop spending your time and money (tokens) on features, and start spending it on profiling log replays to find a bottleneck causing a 50ms delay for the 95th percentile. But nobody's going to do that either, because how slow is too slow? Answer: when it's slow enough to scare the shareholders or annoy the developer on their high-powered laptop.
If you want faster/more efficient software, force it to run on a 100MHz CPU with 512KB RAM and a 56k modem. You'll definitely prioritize speed then.
Lol got my mac M5 128GB I'll import a JS framework for multiplying numbers
I've been really, really impressed with how effective this is. I went from a 4s load on simulated slow 4g to ~750ms: https://image.non.io/speedup-graphs.webp
Side by side vid of the results: https://video.non.io/speedups.mp4
This was for https://non.io, which is something I had purposefully written to be as fast as possible (hand wrote all the comopnents, didnt even use react).
I've been considering creating a skill / utility to do this based on learnings from the speedups - would others find this kind of thing useful?
Maybe hugged but feels really sluggish to me for it is.
Should note the server is on USW and I don't have edge servers for it at the moment.
and then you threw it all away by adding transition animation
That seriously needs some optimising. For example: on click, could the bigger image be inserted behind the small one so it's hidden; on load of the bigger image, hide the smaller image; then do the slide-bigger transition with both images together?
And there are plenty of things you did not opimize for. Like making sure certain dimensions are already known to the dom renderer so that the layout doesn't jump around.
Or progressive images so that it doesn't just popup suddenly.
- in safari, every image is loaded twice, .heic and .webp
- default.png is re-downloaded 31 times, uncached
- images below the fold are request immediately, lazy loading would avoid that [1]
But most important, you have 229 requests for tiny files being served over HTTP 1.1. Without GZIP. From a pretty slow server - 700ms+ to download the main json data. Bundling your JS, or enabling HTTP2 or QUIC/HTTP3 alone would massively improve performance.
If this is the result of days of autoresearch, it's not really anything to celebrate.
[1] https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/...
I currently have one running working on a multi path problem trying to make an A* heuristic that produces the same or better score for search time, paths crossed, corners taken (plus a few other metrics, like proximity of opposite direction corners of unconnected paths.)
I have another where I'm testing how well it can iteratively rewrite the same program without regressions and successively stricter resource conditions. Instead of progressively adding lines of code to make an over engineered behemoth, starting over with the previous version as a template of functionality lets it consider the problem increasingly holistically
I don't think LLMs are going to change this. They won't change what companies and individuals value. Yes, they can enable those interested in performance to write even more performant software, more easily, but they won't convince the people who aren't prioritizing performance in the first place to prioritize performance. This ship sailed a long time ago.
Performance was prioritized when it was necessary due to resource limitations. As soon as those lifted, performance was deprioritized in favor of other properties. I doubt the balance will really be restored until limitations re-emerge or consumers put pressure on companies around performance. Unfortunately, the past few years have shown that the latter is unlikely to work.
But all these things will change in time. Hell is other people's software.
There is no technical reason that animation should take that long.
Someone said, "good enough" and let it be visibly slow. The reasons for that could be anything, including non-performant code in dependencies, written by other people. But it just does not take much computing power by today's standards to composite code-generated animation.
Somehow, despite year after year of percentage-speed hardware improvements, there are cultural and structural reasons people ship code visibly slower than it needs to be. And faster compute appears to be irrelevant.
The technical reason is you will not see all the animated beauty if it's fast, so animation forces feeds min delays for states transition
Architectural decisions have always been important.
Though I must admit you can have an absolute shit ton of ridiculously complex objects in a scene, and it will run smoothly if none of them is in edit mode which is at least fairly impressive.
Legacy baggage has nothing to do with it, in my opinion.
First of all, coding speed is not the main bottleneck. I've been working on my own JavaScript framework. Most of my work is talking to people about challenges they have with web development, doing research on existing tools, and thinking about features to add.
Coding more slowly without LLM usage is a benefit because it helps me understand every line of code. The slower pace also helps me think about making things intuitive and reflect on my experiences with React.
While my framework is outperforming React, that isn't particularly impressive. On the other hand, it demonstrates how ignoring LLMs improves quality. Also, my monthly token spend is $0.
You used to buy a computer, install an OS, buy apps and run them. Neither the OS nor the apps didn't suddenly degraded themselves with "free update" that were developed targeting newer & newer machines. You bought a new computer when new software you wanted to buy wouldn't run fast enough on it, OR the computer died.
Nowadays, you buy a new computer when the same software you've been running for years or the OS itself that demands to be updated "free" makes your machine intolerably slow.
Funny how that all worked out.
This is, in my experience, not a thing on Linux.
Or the way so many people were on netbooks and the growing monolithic kernel kind of made the default gig or two of RAM less and less doable (I guess non-consumer-modifiable tablets and mostly soldered and increasingly locked down chromebooks took over most of that market, but it feels like people just take storage space, memory management, and hardware for granted, now).
Off-topic, but Dan, in the name of all that is holy, throw a `max-width: 40em;` on your blog!
A crude example is how, when building a large forum project years ago, I had a checklist I ran on every new http route: authentication, authorization, validation, etc. just because it's trivial to forget one of them. Yet now the LLM can come up with a lintable/verifiable way to ensure every route has the whole kit by construction.
Almost every way I feel like I can provide unique value to the automated AI workflow (aside from deciding what to build at all, and taste) feels like it can be automated by better markdown.
It's pretty much solved, though I only use sota models.
I think for these convos, we need to see concrete fail cases so we can see what you're talking about and whether the truth matches up with the claim.
I paid for a wide screen monitor, let me use the whole width of the screen.
LLMs have caused such an increase in ram prices that pine64 is no longer making linux machines. When you can't afford RAM, as a direct result of LLMs, it's going to make your assembly app slower too. Sorry, this is the future you chose.
But if you're going to be relying on the LLM to do all the final-stage implementation work, and you've already got all the guardrails in place to ensure that the final output is performant, secure, correct, etc., then why would you still target TypeScript, which is entirely a kludge to shoehorn type safety into JS for the benefit of human programmers?
But then why target JS, if it's just going to sit on top of a complex runtime like Node? Why not just target the language the runtime interpreter itself is written in, and just output clean C? But then C needs to be compiled down to assembly anyway, so why not skip that step and just output assembly?
LLMs still have a lot of pitfalls and risk factors, and will always need human oversight, but in this case, what's on the horizon is something that can eventually eliminate all the intermediary bloat that only exists to optimize the development process when human programmers are doing all the grunt work. Eventually, we might be able to do away with the tradeoff between development velocity and runtime performance.
1. JS is a privileged language and allows you to tap into web and node
2. Type safety is good
But TS is not a good language overall. It's just much slower than alternatives and much less safe.
But either way, token consumption is only a relevant concern if you still expect to be using SaaS LLMs by the time you're ready to have them target lower-level code for you.
And much of its convenience comes from the fact that it has a huge set of off-the-shelf libraries for doing things that are properly part of your higher-level business logic, rather than lower-level foundation code, anyway -- those kinds of libraries are popular because they maximize developer velocity, not because they should be third-party dependencies.
The kind of stuff that makes sense to include as libraries, e.g. code dealing with foundation-level building blocks, like standard encryption algorithms, audio and video codecs, compression, etc. is exactly where the reference implementations and baseline libraries are all in C in the first place.
The point here is that, ultimately, C won't take longer if you're using an LLM. Eventually, you'll get more performant, more secure code targeting the lowest level possible, and mucking around with stuff like NPM is what will both slow you down and produce worse results.
To help the hapless humans who have to fix it, silly!
Maybe we’ll get there but I don’t think we’re there yet.
I still regularly run into bugs that agents struggle to solve.
And they often churn creating more and more convoluted solutions which are then more error prone and harder to reason about
On the other hand it does mean there's no excuse for writing your program using Python or Electron or whatever, which will have a big effect on performance.
I get that LLMs would make it faster to write distinct copies for different operating systems and maintain them both but thats still more costly than a single system.
Sorry if I’m misunderstanding the goal of Electron but it still seems cheaper and easier (although admittedly worse than multiple native apps)
Besides a bootup that is no longer than your browser's, everything else inside an Electron app can be identically fast to a native application.
Please be a better filter - do not spread bad memes.
HTML layout and rendering is almost never as fast as GUIs like Qt. Also Electron apps usually use JavaScript which is slower than "fast" languages like C++/Rust/Kotlin/etc.
The reason people use Electron is because it's easy, especially if you have to have a website anyway. Slack is a great example of this. They already have a full website so it's kind of silly to redo the whole thing...
Unless AI makes it easy. We'll see I guess.
So going on a tangent: an additional mechanism you have that becomes feasible with LLM assistance today is proving your code correct.
High end Windows 10/11 machines (like my Alienware I9 w/ 64GB mem + RTX4090) seem a bit sluggish. Low end machines with Windows are absolutely painful to use!
I've install debian onto my fairly ancient Lenovo X201 Tablet and it is exceptionally snappy.
I'm running Debian since Debian 1.1 (1.1, not 11). My install has always been very lean: a minimal WM I love.
Computers have only ever gotten fast and Linux more optimized.
It's extremely snappy indeed and feels like it. The problem is what happens inside browsers: are great many websites are just lame. So many pointless connections to so many pointless "telemetry" services, trying to track your every single habit/clicks. Just open the dev tools and look at all the trafic exchanged: that's where the bloat is.
The SV JavaScript "punch the monkey" ads bros at work.
Not sure how either of those things are conducive to performance
I disagree with the premise that this is the desired outcome. If every piece of software is bespoke and everyone’s instance of it works slightly differently, then it’s impossible to get support or a shared knowledge of how it works. There’s no “just share the excel file”, there’s no “press the this button on the left”, there’s no “oh I use program X to solve Y” (instead you have to know what you need so the custom software can solve it, but my time in startups taught me that most users don’t know what they want or need).
- Your boss/most of the market doesn't care if your software is slightly slow
- Your boss/the market wants new features
It's easier now to slip in fun optimization features when no one's looking, but the incentives are still off.
If your boss doesn't care about your software being slow he's incompetent.
https://services.google.com/fh/files/blogs/google_delayexp.p...
https://business.google.com/ca-en/think/marketing-strategies...
https://medium.com/ft-product-technology/a-faster-ft-com-10e...
https://www.pingdom.com/blog/page-load-time-really-affect-bo...
It’s often been hard for me to make coworkers care as well which is even more frustrating.
Unless your job really prioritizes performance and builds in benchmarking people often focus on finishing a feature as fast as possible and ignore performance (and accessibility, bugs, UI polish, etc.)
It obviously depends a lot on what you do, but often that doesn't matter.
There has never been. We are running supercomputers. And somehow every app is a dog-slow barely moving monstrosity incapable of showing a page of text without stuttering.
Whether it can apply it directly from the codebase, I’m unsure and depends on many factors.
The author seems to be in a situation where you can burn as many token as you want. I don't know if that's a general situation.
(Even if you don't care about the environment impact of your computations, there is a dollar bill associated, and _someone_ cares very much about that.)
I can foresee a situation where devs will have to decide on how they allocate a fixed token budget - and then, faced with the option of "burning tokens to add a new feature requested by a customer for tomorrow's demo" or "burning tokens to maybe make the app faster in some edge case", the trade off will look a lot like the ones organisations made with human dev time.
This assumes that tokens are not going to get dramatically cheaper. I can't predict the future, but I don't see a path to that (or, are local models, and "a TPU in every machine" going to make the question irrelevant?).
I can definitely see a path were tokens get massively more expensive (let's meet six months after anthropic's IPO and check :D)
The point is: Code needs to be maintained if it is to be relied on. I haven't heard good things about LLM's on that front.
Love the concept of agents adding epicycles to fix bugs. It maps closely to my experience.
I haven’t seen as big of an impact on the “good” side. People with attention to detail and craft can use it with great results but they’re not pumping things out at the speed of the slip slingers.
Woke up today with 2x improvement and fully tested.
The code optimized was mostly written by humans over the years. That code was too cumbersome to optimize by humans so we never touched it.
Anything else kind of just feels like "make GTA 6, make no mistakes". The ability for you to write fast and correct software is really going to depend on well you can specify your problem, and having the ability to specify correct programs in such detail an LLM can reasonably performance tune it is a big enough reason for me to believe there will still be slow software
See https://deepseek.com/harness/en/
Is anyone really impressed by this gimmick anymore? Just give me a blank HTML with <table>. Its fine. I dont think anyone care.
Fun story, I was writing some 3D graphics animations for a website, and the only dependency I allowed was three.js. It ran well, but the site was loading slightly too slow so I told the LLM to optimize. It essentially hit a hard wall and told me something along the lines of "I can't optimize further, three.js is the bottleneck due to blablabla, but you don't need it. If you want I can rewrite it with pure WebGL". I said sure and load times were cut by like 90% and framerate went up too. Staggering.
There is less of a need for super convenient, yet bloated languages now. For software that matters, and burns the most joules in the world, I hope we can also spend a few more tokens instead of doing it in interpreted python just because it's a bit easier for the LLM.
That's probably one of the biggest sources of software slowness, and we can just stop using it now, with almost no trade-off. Neat!
Like we have OLAP and OLTP, there are still batch jobs that need to happen before you can see your dashboard, we have multiple users using the same resources, there are still DB Locks when writing needed, there are "first load" issues
— heck now we have to compete for resources with even more bots than ever.
Transferring 10GB or 1TB of data still not going to be instant laws of physics still apply.
Make it as harder as possible to communicate your message...I am not sure if this is supposed to pass for minimalism...but looks more lie readability hostile snobbery. Good content does not excuse contempt for the person trying to read it.
If your argument is worth attention, presenting it legibly is basic respect for the reader....
You want code that’s fast, correct, stable, memory efficient, secure, usable, and more
The reason code might be slow is about product tradeoffs. It’s certainly easy to have an LLM optimize performance only, but what do you sacrifice? Is that an acceptable tradeoff?
Maybe slow code is a good idea if it’s the only way to get correct and secure code?
Take slack for example. It's not fast, it's not correct, it's not stable, it's not memory efficient, and it's not particularly secure. There's a reasonable argument that it's usable, but that doesn't seem to have much to do with tradeoffs in their technology choices.
Do you not add sleep to your programs?
They could have worked in 8GB of RAM but chose not to, and now the rest of us suffer as a result because we couldn’t afford $5000 dev laptops as daily drivers.
But you need a really solid workflow, solid benchmarks which run quickly, and a lot of tokens -- plus a rigorous profiling workflow.
Interesting.. so how much do you think his package was worth?
And Munger's Law never cares if you hate its consequences.
Eg. JIT compilers are rare because they are difficult to write. JIT compilers come with tradeoffs, they aren't an absolute boon. JIT-compiled code isn't good for short-running scripts, for example. Even normally JIT-compiled languages try to special-case parts of the code that would not benefit from compilation and interpret them instead. Another huge downside of JIT-compiled code is debugging it. JIT compilation is better for languages that don't deal with memory allocation directly, and, in general, expose fewer low-level primitives to the language user. It would be a fool's errand to make a JIT-compiled C... I bet someone tried it, but, seriously...
Another one: a solution to slow search is building an index... Boy do I have bad news for you... Even a beginner DBA knows that an index can be a curse or a blessing. There's no one size fits all solution to this.
And the author continues to misrepresent similar problems only to make his solution make sense.
You are a rust expert. Make only 5 mistakes.
:) Interesting quote though - I guess one does get to decide how much testing/debugging effort one wants now
I think engineers building very complex systems now have a lot of performance knobs to twiddle that would have just been too costly for human effort. Since we constrain the responsibilities of the agent slop is less of a problem. We relegate it to defined tasks with clear API boundaries and test harnesses.
Those at least already exist, without the need for any LLM: Sublime Text, BBEdit, Notepad++
It's messed up. Sometimes it feels like everything in the universe is aligned to ensure that skilled software engineers lead horrible lives full of anguish, frustration and powerlessness.
It feels like the economic system creates incentives which go against efficiency. So then it makes sense that software engineers would be most impacted.
There's a weird duality to our job; it's supposed to be about automation and efficiency but in reality, engineers who make the most money are those who create unnecessary complexity because it creates better lock-in and thus gives them more leverage in negotiations. And the tech monopolies they work for never seem to run out of money no matter how much enshittification occurs as a result.
Same perverse incentive for open source projects; engineers who lead projects that are full of bugs can charge more consulting fees and sell more enterprise licenses to work around the issues.
And engineers are often forced to use shitty tools by their employers so we're always stuck between a rock and a hard place with limited control over our output... Which we are blamed for when things go wrong but never given credit for when we pull heroic feats because those achievements are often overshadowed by missed deadlines resulting from being forced to use shitty software.
Adoption and quality are orthogonal, unfortunately.
Worst part is that nobody listens to us. Business people with money only listen to engineers who have money; who are mostly merchants of complexity. Engineers who care about efficiency have no voice in this market.
The author of the article definitely seems to think LLMs are what enables this to happen, but I personally am much more skeptical of that. I think what it really needs is bringing engineering back into software, not just throwing LLMs at it and calling it a day.
LLMs operate in abstractions. "Reasoning" is bolted on.
1. Identifying the cause of poor performance. 2. Implementation. 3. Architectural impact (performance is a classic case of a cross-cutting concern)
(I am specifically leaving out the case of realtime systems, hard or soft, where additional factors come into play.)
But the article seems to focus entirely on the second aspect, while largely ignoring the other two.
Most performance bugs are not difficult to fix [1, 2], but can sometimes be hard to identify. Implementation effort is not the driving limitation in those cases.
Conversely, other performance improvements may affect the overall design, e.g. 27% of all bugs identified in [3].
Having an obvious, self-contained optimization target with a benchmark where algorithmic optimization within a module is also the key problem seems to be the exception, not the rule.
Also, not all performance issues are created equal. Many have trivial cost.
In contrast, some of the most challenging performance issues are the ones that affect the design and architecture of the system. After all, the hard part of software engineering is not writing a small, self-contained application. It's managing system complexity, while maintaining (in the words of Fred Brooks) conceptual integrity. Fixing performance issues is at least in this regard not fundamentally different from fixing other software defects.
Unfortunately, this is an area that is also full of trade-offs, such as performance vs. architectural simplicity, or performance in one part of the system vs. performance in another part, all of which requires judgement.
For example, you may need to bypass an abstraction boundary or reorganize abstraction boundaries to improve performance. Or you may have to special-case something while keeping duplicated code at a minimum and easy to maintain.
This is not to say that agents cannot help here, too. In fact, agents can be very helpful at e.g. identifying bottlenecks that are not directly visible in a profiler or can be used quickly do comparative evaluations of the various options for an architectural change. But solving these issues is not, like with the regex example in the article, about hillclimbing towards better performance, but involves a combination of puzzle-solving and design skill, IMHO.
And finally, even a self-contained algorithmic improvement may come with an increased maintenance burden, especially around edge cases and through increased code complexity.
[1] Jin, Guoliang & Song, Linhai & Shi, Xiaoming & Scherpelz, Joel & Lu, Shan. (2012). Understanding and Detecting Real-World Performance Bugs. Sigplan Notices - SIGPLAN. 47. 10.1145/2345156.2254075.
[2] Selakovic, Marija & Pradel, Michael. (2016). Performance issues and optimizations in JavaScript: an empirical study. 61-72. 10.1145/2884781.2884829.
[3] Zhao, Yutong & Xiao, Lu & Bondi, André & Chen, Bihuan & Liu, Yang. (2023). A Large-Scale Empirical Study of Real-Life Performance Issues in Open Source Projects. IEEE Transactions on Software Engineering. 49. 924-946. 10.1109/TSE.2022.3167628.
See: iOS 27. It’s faster than the previous version, even on very old phones.
I also think AI will contribute to removing a lot of the tedium surrounding optimization.
People are shipping all sorts of weird architectures, non-performance code, etc.
I wish that the ram scarcity would drive more performant software however I just think that the way people are shipping software currently will not lead to this
And they weren’t before AI?
Cutting edge development practice in 2025 was using a big JavaScript framework (not quite VMs, but they inherently strain the environments they run in).
That code is run by the JavaScript interpreter VM, which is running in the browser (another VM), which was (in whole or part) written in a managed language (another VM).
That program runs on top of the OS (another VM), which runs on another VM (the default hypervisor for Windows 10/11, or the desktop environment for Linux/macOS).
Each VM lets you have an entirely new managed programming environment- you have a new program that lets you write software even quicker than before- but that comes at the expense of memory usage and (most importantly) latency. Which is why it takes modern systems a few seconds to figure out you pressed a button in your React application that's running in an isolated browser instance consuming 1GB of RAM, where systems in the 90s were instant by comparison even though they had a fraction of the processing power.
It's conceivable that in 10-20 years, LLMs could be used as VMs in and of themselves, which will still likely be using 2020s development languages. So your software is going to require at least 128GB of RAM to compile, use another 32GB to run, and it'll take twice as long to respond to clicking a button.
But there were some sea changes from time to time.
When the iphone came out, phones became very responsive to fingers. hate the control apple gave to customers, but their guidance made apps respond in milliseconds. Before that phones were clumsy. Wasn't a desktop, but was different.
Will we have voice/chat interfaces that will be bloated... but responsive like a conversation?
Yet when I ask for code it writes, by default, both slow and insecure code that mostly works. Kinda.
As I try to get AI to rewrite it into more secure, less bloated and optimized code is when it starts to randomly crash.
Then I read articles about how AI is "moving too fast" and cry.
func Benchmark(b *testing.B) {
for range b.N {
runCodeUnderBenchmark()
}
}
Also the architecture really matters now because a poor architecture will lead to more weird edge cases and require more exotic tests to plug all the holes.
Even if you assume that statement is true, there are techniques other than exhaustive checking/model checking. Proof assistants/theorem provers/etc. like Rocq/Isabelle/Lean are quite capable of formally verifying programs without needing to exhaustively explore the search space.
I'd question the accuracy of that statement in general as well; model checkers like CBMC/TLA+ are handy for proving properties about interesting systems. The latter, for example, sees use for verifying concurrent/distributed systems, which I think can be reasonably described as more than "small and unimpressive"
It's s about subpar models trained on subpar data doing subpar work.
The only reason why anyone takes it seriously is that we've had a glut of subpar developers for 30 years so it all balances out in the end.
I agree that there has been a glut of subpar developers for years, and that has lowered the bar significantly. This is mostly because core values shifted. So let's keep our eyes on what really matters rather than acting elitist.
In practice, no one rigorously "proves" that their program works at present, and well-written tests do suffice. The definition of "well-written" here is circular, granted, but there isn't really an alternative. Even strong programmers should be writing high-quality testing suites.
Only a moron would then use them to code up that spec.
i want the code to do this --> write the code to do this
i want the code to do this --> write the tests to specify this
the hard part is not the code or the tests. it's knowing what 'this' is supposed to be, exactly.
if you have the code, it is trivial to write the tests that will perfectly test the code. if you have the tests, it is trivial to write the code that will perfectly pass the tests.
however, none of this has anything to do with whether the code or the tests are good, and having a bad version of one will not help you write a good version of the other.
People get entrenched in closedminded thinking. My guess is that this has been your canned spasm for the past year. Repeating these things won’t make them true or protect your job.
You protect your job by learning how to use new tools as the technology advances.
This sort of lazy thinking isn’t acceptable anymore. Whether this is a reflexive Luddite response or a reflection of your work ethic, neither is a good look.
> Yet when I ask for code
Engineers don't ask a tool to write code.
Most of these things are just true as of today.
> You protect your job by learning how to use new tools as the technology advances.
Sure and the more you learn the more you see how hyped they were and reality untangle
> Engineers don't ask a tool to write code.
And what they ask? They create loops or graphs? These tools ultimately vomit code anyways
Maybe pick languages that are more boring. That's what works for me. It worked for hiring crappy outsourced programmers and it works for AI. It seems to do well with PHP - no long running processes / side effects. it works great with golang, simple language spec, and a very common style of writing.1
Regarding security - i wonder: can we develop a framework that is responsible to 100% of the security, with zero responsibility for security on the app developer?
The fact is that with the right spec, agentic guardrails, and evals, frontier models can now one-shot secure, performant code.
This is the job of software engineering now. Learn to properly use the tools for the job.
> one-shot secure, performant code.
I call it reverse skill issue; it feels secure and performant because you can't verify properly, it feels right but it's actually broken in many nuanced ways. You can measure performance and produce a broken measurement as well. It's not as simple as you make it
Can you give a concrete example of that being the case? I’d love to see the prompt and harness. And understand the scale of problem you’re talking about solving with a one-shot approach.
How long will it take to write the right spec? How will you know ahead of time that the spec is correct?
Which agentic guardrails do you mean? How can they be set up?
-
I’d say the job of software engineering is still firmly being able to stand by the work you present and being reasonably confident that it’s correct according to yours and others’ understanding of the problem you’re attempting to solve.
That is, there’s a misunderstanding of why software (traffic) was slow in the first place, and it has nothing to do with our ability to generate code (number of lanes), even if that code is “high quality”.
At the end of the day its a mindset.
In reality, if more people were able to go wherever they wanted to go, that's actually a good thing regardless of if the existing people didn't get to go somewhere faster.
It's not the only reason. I think that traffic engineers lack/lacked a feel, a certain gut understanding , an inability to feel through the skin. Just look around you. See and hear them at municipal meetings. They are unable to sensually experience the effects of their work, their praxis.
The people are to blame. You get what you vote for.
Both the traffic engineer and the author of this blogpost are well-qualified enough in their fields of expertise to be careful about the claims that they make with regards to speed and efficiency.
A big issue is how those claims are interpreted by non-technical people, especially in leadership. Would hate for my CEO to stumble upon this blog post and believe, based on the title and a brief skimming, that his engineers just aren’t using AI correctly and that’s why the website still takes a few seconds to load its content.
Understanding.
Simplify, then add lightness.
[1] hn favorite word
As you imply, some of them are not because spending time on optimization seems relatively expensive compared to value. But some of them are.
Maybe the author should be a bit more careful about what titles they give their blog posts, lest they be misinterpreted by mere mortals such as myself.
Imagine traffic not getting any better after adding 1000 lanes in a medium sized town… not just to one area but to every single damn road.
Yet we’ve done this with RAM, CPU, and storage…
Big tech funds startups (through obfuscated legal and financial means) and that funding goes straight back into big tech cloud hosting and SaaS services.
AI is the perfect tool for enshittification, because it can get you 80% of a finished product with a fraction of the costs it used to take, but the last 20% takes 2-5x times longer. So you just ignore the last 20%, make it up in volume by producing 100 mediocre products instead of 1 great one.
And that’s basically the state of the tech industry right now. I don’t think it’ll be like this forever, I think there will be better ways to collaborate with AI rather than wholesale delegation and prompting. But for now we’re stuck with mass enshittifation. People who have no sense of quality, taste, or craftsmanship don’t even understand, they skim over some generic blog post from Claude that takes five paragraphs to say one thing or watch the Coca Cola AI ad and don’t event notice it’s a different train in every shot and they think it’s so amazing. I hope this isn’t just the way things are now.
AI SDLCs simply compress the same incentives that were always there: you could invest in optimizing your architecture and improving your UX, neither of which make for big bullet points that close sales, or you could invest in new features that do.
The investment that was previously measured in months of developer time may now be measured in token spend across days, but the underlying dynamics shaping where it’s directed haven’t changed.
problem solved
What was important for this was the fact that I can run JSON-defined e2e tests[2], also good to find frame-based leaks, stale-ID problems and general "program shows its using lots of memory in the task manager" (task managers are wildly inaccurate for this, as I found out).
So, I can just tell it "okay, loop this e2e.json test over and over again, use heaptrack, samply, find out why and exactly where it's slow, memory-hungry - find the section in the codebase, figure out a solution" and then let it run overnight. The biggest difficulty here is that many perf tools are still written for humans and that things like "how much memory are we using" is a wildly complex topic (lots of problems actually getting the correct number without over-allocation, memory allocator slack, OS-level page size, memory fragmentation, etc.).
But Fable was able to track down things like "LCD font hinting causes 90k allocations that are unnecessary", etc. etc. - which then also improve frame time, usually. Memory optimization + better perf pretty much go hand in hand (less allocations = more perf). I could track them down manually probably, but it would take way more time.
Having some basic understanding of data-oriented design, cache locality, memory tiering (L1/L2/L3/main RAM), does massively help with architecture decisions (e.g. Azul can use a single buffer for the entire DOM node list, in difference to normal browsers which do the more "object based" allocation model, which massively helped page breaking performance on html-to-pdf use cases[3]). Pure-functional style also helps (f(State) -> UI) because then it gets very easy to drill down exactly where things are slow and where caches are needed.
[1] https://azul.rs/ui/release/0.2.0#demos
[2] https://github.com/fschutt/azul/blob/master/e2e/css-anim-per...
[3] https://github.com/fschutt/azul/blob/master/layout/benches/f...
Note: UI Toolkit is still very WIP, docs + code are still slop, etc. - working on it. But I just wanted to add this.
Here in hackernews we have a bubble of people who are very smart so they process information at high speed in general, and have lots of experience with computers, so they process computer-related information with extra speed. We're like race drivers, squeezing every single bit of performance from the car, while majority of use cases for a car is being stuck in traffic on your way to work. Those people simply don't care about performance above 150km/h because they'll never reach such speeds so putting expensive, high-performance engines is a waste of resources.
The point I'm trying to make is that if software is fast enough for average user, and most users are fucking slow, there's no business need to further optimize it. You might keep arguing "but but but back in my days apps were instant and now they suck" but users simply do not care. This is why companies put shitty, time-wasting animations even if nothing is being done in the background.
Otherwise, I would have left immediately.
I'm all for speedy, simple, plaintext websites, but it is a negligible amount of work in 2026 to throw some barebones CSS in and make it approachable.
I think he might subscribe to the philosophy of everyone should style websites however they like and not be prescribed by the author.
He's had a usable website in the past. This one is explicitly made to be shitty. "Look at me, I'm such a geek" energy.
> Not everybody knows how to use sql or css or java or whatever thing it is that lets you pick the font.
lol at the idea that Dan Luu can't add css to his website by copypasting some links into his html. He could make the site readable in about 3 mins and never have to worry about it again.
<!-- Google Fonts -->
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300italic,700,700italic">
<!-- CSS Reset -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/normalize/8.0.1/normalize.css">
<!-- Milligram CSS -->
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/milligram/1.4.1/milligram.css">
Opinions are dime a dozen, but readability (and usability in general) is a well-studied field and having to move your head from side to side to read text that stretches the width of a screen is objectively bad for readability. That's just one example.
And making the website readable for the majority does not prevent one from using the built-in reader or an extension.
> Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting.
Also, perhaps you should learn how to make use of the user stylesheet feature of your browser, or ask an AI to tell you how...
But reading small white text on black background that's 100% width is nearly impossible to read.
LLMs can now write perfect code in low-level languages, that is frankly _safer_ and _exponentially_ faster than implementations in high-level languages.
When will we start using LLMs to write my web backend directly in UEFI and assembler? (Although uefi is an OS, so maybe more fair to exit the boot services)
Your experience does not match mine.
It writes ok code, it does not write good code. I see it leaving a lot of performance on the table, doing silly things like wrapping everything in a global mutex. Or certainly doesn’t write idiomatic code in Rust or C++
Also, how does this perspective help for (A) existing applications, or (B) frontend applications, which are almost never provided via low-level lanaguages (think web apps using JS, or Android apps using Java/Kotlin)?
the OS makes a huge baseline difference