Simpler code is generally faster code
twitter.com
twitter.com
It's great though when someone says it's not possible to make something faster, and you say “here, I made it” and leaving the existing console to scroll while you do the rest of the video is great showmanship.
I interviewed Casey[1] and I learned a lot from his perspective on development, particularly not being afraid to just build things yourself from scratch and to try to understand what is going on at the level of hardware.
The problem is that if performance isn't a priority, and then you finish all of your features and ship your 1.0, it begs the question: when exactly are you going to make it fast?
Never. That's when. Once you've shipped something, people get assigned to other projects, or new features.
Worse, you're now locked in to bad architectures that too much code depends on for changes to be easily made. Casey makes the point that the Windows Console IO API is inherently bad, and using it guarantees an unfixable bottleneck.
I've seen this attitude in Microsoft's software everywhere I look. They simply flat out ignore performance as a feature, and almost never come back to fix it. Easily less than 1% of the time, no matter how important it is. Even when people make a lot of public noise about some performance issue (like in this case), it quickly turns out that it is now unfixable because of the endemic apathy towards optimisation throughout that organisation.
E.g.: Windows Updates. I have gigabit internet and a high-end SSD, yet it takes something like an hour to apply patches! https://borncity.com/win/2018/06/01/windows-server-2016-slow... That to me is simply unfathomably bad programming. How hard is it to download a file and copy it to a folder?
The simpler code here would be orders of magnitude faster than whatever craziness they're doing. Like Casey points out, I could sit down right now and write a Windows Update that runs a thousand times faster. It wouldn't take much more than a weekend. (I'm not kidding: Prior to Windows Server 2019, all versions of Windows Update had O(n^3) algorithms operating on hundreds of megabytes of XML data that could take nearly a day to process.)
Remember the Grand Theft Auto game loading performance issue that was posted here where some guy cut minutes out of the game loading time by binary-patching it? People here were estimating that Rockstar may have missed out on a billion dollars of revenue because huge numbers of players gave up in frustration and moved on to other games: https://news.ycombinator.com/item?id=26296339
Microsoft had their own version of this. When they released Flight Simulator 2020 a lot of people called it "download simulator" because it would take a solid day to download and install. Many people made excuses for it, but they shouldn't have. Just by inspecting the progress bar I found at least five critical performance mistakes. Again: I have gigabit internet and an SSD, yet that install took 8 hours!
Where software gets out of control really quickly is wherever you start to express a thing that is not essentially true on its own premises, and requires the introduction of a great number of assumptions. Commercial software falls into this pit unthinkingly since the business requirements create all kinds of assumptions: "it needs to support as many languages as possible" or "it needs to have calendar features" or "it needs to be compatible with this other system" are three super common ones that can create huge dependencies. So our software deviates from "truth" in some respect to get the job done of being presentable to other humans and doing the thing they expect.
At every layer of our current systems, we have little Swiss Army Knife components that can be set up to do anything to resolve these kinds of issues, which means that they do all of it somewhat poorly. Old software is often more like a chainsaw - it doesn't have much to configure, so it's inadmissable for a larger spectrum of tasks, but the thing it does, it does quickly.
Long run, though, I believe that true software crowds out false, even if there's a short-term advantage in getting the market to adopt something that does everything poorly and mostly serves the business, the next phase of that is a flight to quality, which currently manifests as everyone depending on some unmaintained open source code that works and does the job.
Note that this whole twitter thread is in reference to Casey lodging an issue on GitHub about how slow Windows Terminal is, wherein he was told that his simple suggestion for making it fast would be a PhD level thesis worth of work. He was being polite and explaining things to them until then, but after that statement he was so disgusted he closed the issue and created refterm to prove his concept[0]. Then a bunch of people came out of the woodwork to make excuses for why it still couldn't be done. It's a fucking terminal emulator, a piece of 80s technology[1] that mimics a 60s technology, running in 2021, and people still think it makes any sense at all that it could be slow!?
[0] which is not anything groundbreaking, it's pretty much the obvious way to do things. He has intentionally done no optimization since it's designed to serve as a minimum performance benchmark.
[1] I'm guessing here, since I can't readily find an answer to when the first terminal emulator was released.
I had one where tuning the code to fit the cache size (by using a simpler, smaller, "less efficient" algorithm) led to a ludicrous performance gain ... and several burnt 85Mhz Sparc CPUS, they expected pauses occasionally on the data bus apparently.
Most of the time the optimizations work as advertised but there is no problem contriving examples that show ridiculous improvements by removing speed optimization.
There are various limits that have huge effect on your code performance. For example, if your inner application loop does not fit instruction cache its performance might be impaired. If you can reduce the size of the code, ironically by removing speed optimizations, this might just be what is needed to dramatically speed it up.
On macro scale years ago I have been using Gentoo and, generally, compiling almost everything with size optimization had very positive performance improvement on a platform that has been deficient in RAM.
I guess "size" improvement should not be thought as opposite of "speed" improvement but rather an alternate way to possibly improve performance. The compiler does not know which alternative may or may not bring effects.
All the rest of the time is bookkeeping and memory transfers.
GPUs are FAST, but frame data is BIG.
My older RTX 2080 Ti has main memory bandwidth (not on-chip cache bandwidth!) of 616 gigabytes per second. The more modern 3080 Ti has 912 gibabytes per second.
Let that sink in: ONE TERABYTE PER SECOND!
They really are that fast.
My old work machine has a GTX 980, and even that has 225 GB/s of bandwidth. It's a 7 year old card! Let's use that to make things fair and realistic.
I have a 4K screen plugged into it, so the frame buffer is 31 MB. That works out to 7,282 frames per second, or 137 microseconds per frame.
The GPU hardware is not the issue here. It's entirely possible to do it in microseconds even on very old hardware.