Hacker News Reader

Comment by klabb3 | Hacker News Reader

Alternative hypothesis: (brace yourselves) people don't care enough. Any vendor will prioritize requirements, if performance is not in there, that CPU and memory is going to be used if in any way it helps the developers. Conversely, by looking at a system you can infer its requirements.

For commercial airplanes it may be safety first, ticket price second (passenger capacity, fuel efficiency) and speed third. For most software, functionality, signing up for a subscription, platform availability etc are usually prioritized higher than response times and keyboard shortcuts.

Game devs worry a lot about latency and frame rates and professional software care a lot about keyboard shortcuts. This proves (anecdotally) that performance isn't unacheivable at all, but rather deprioritized. Nobody wants slow apps but it's just that developer velocity, metrics, ads etc etc are higher priorities, and that comes with a cpu and memory cost that the vendor doesn't care about.

View on HN

I'm a game developer and game performance is better than ever. 144 Hz monitor? 4k? We got you covered. Even ray tracing and VR is on the way.

Most games render frames of a 3D world in less than 17ms, but most websites take 3-7 seconds to load because of all the ads and bloat, and things shift around on you for another 20 seconds after that, so when you go to tap a link you accidentally tap an ad that finally loaded under your finger. If you optimize those websites, they run super fast though, but it's quite a pain to do with the dependency bloat in modern tech stacks...

(note: games lag as well when you drag in a million dependencies you don't need)

The thing is, most sites and web apps try to solve a user problem, and if you are the only company in town that solves that problem, then performance barely matters - what matters is solving the problem. The users will put up with some pain because the problem is even more painful.

With games, it's all about the experience of interacting with the software - so performance is (hopefully, depending on your team and budget) amazing.

That, and... performance tuning is hard work, and I think most people don't know much about it. It's a fractal rabbit hole. Cache misses, garbage collection, streaming, object pooling, dealing with stale pointers, etc. Even I have a ton to learn, and no matter how much I learn, I probably still will have a lot more to learn. It's easier for many teams to hand wave it I guess as long as they aren't losing too many customers because of it.

The 3d world has long been a curious space to me. Even in late 90s, and early GPUs, you knew how responsive things were compared to 2d applications. It was hard to reconcile in my head.

In my opinion it's because games are art, and art is about feeling. I made a browser game, and recently optimized it heavily. Now it loads a game and huge 3D area in 2-3 seconds, faster than most mainstream websites. Like grandparent said, it is possible...and in art, how something feels is a priority.

I wish more game developers optimized for space. I stopped playing video games almost altogether because I'm not going to download 70 GB every time I want to play something. The size has gotten absurd.

My first computer had a 7GB hard drive. At that time we had CDs with ~700mb capacity. Now we routinely have 1TB storage and games are 70GB. As a percentage of total storage these are similar. I'm not sure how the download rate of 70GB compares to the read speed of a CD but installs took a while back in the day as well.

"Patch day, no play!"

I had a cdrom drive in a 486 with 8mb of RAM and ~200MB of disk. I could either have a game or two installed (which at the time invariably ran in DOS) or Windows 3.1 installed. Not both.

At the time it was like… wait I just bought this fancy CD-ROM… why do I need to install 30 or 40 whole megabytes to disk to play it?

At that time it was pretty common to actually install all the game files to disk (drives were slow… I think it was 4x read speed) and then the music was encoded in the cd as additional tracks of plain CD audio.

I don't have the fastest internet connection but downloads from steam sit happily at 30MB/s so in an hour I can download 108GB of data.

The earliest cdroms that I remember were 1x drives - so it took an hour to read that 700MB. Obviously drive speeds increased tremendously over the lifespan of cdrom (did it reach 48x?), but compared to the earliest generation it is about the same.

The limiting factor of the CD/DVD drives was seek time, not read speed. Seeking a spot on the disk was on the order of 150ms. You basically got 5-7 read OPS. The best drive I remember stood out at 120ms seek, some would be over 200ms seek. Sometimes this was a hard number to track down. Even with a 48x drive, installation could easily still take 30+ minutes particularly if the installer wasn't optimized for sequential read off a flat file.

Sometimes you got a really bad installer, and it could take a pretty grueling 2+ hours. Some companies chose to optimize for space (thus cost..) rather than user experience.

As space got more plentiful it would sometimes be easier to copy the disk and remount it and then install.

Textures are big... bigbly big! And to be fair, game devs even have middleware solely dedicated to compression (such as Oodle http://www.radgametools.com/oodle.htm). I am sure that some % of that space can still be recovered, but it's definitely not significant. I'm sure you can't drop to half and maintain the ultra settings.

You would be surprised, actually. This [0] shows an example of some excellent space savings. I used to work for Epic and the results we got on other projects were in this ballpark too.

[0] https://dev.epicgames.com/community/learning/tutorials/ry2D/...

My assumption was that you get down to 70gb packages after using tools like oodle. But hell, I may be wrong there and may be giving most AAA devs too much credit.

> But hell, I may be wrong there and may be giving most AAA devs too much credit.

I don't think this is a fair comment. Theres an enormous tradeoff to make when compressing game assets, and uncompressed game assets are _absolutely enormous_. My current project has about 30 people at the moment. The game content is ~50GB, the client itself is ~4GB but the source substance files/fbx files/photogrammetry assets are tipping on 500GB. We're a _small_ AAA game. My previous project was closer to 500GB for the game content, and the source assets were multiple TB.

Tools like Oodle are third party licenses that need to be considered. Most games are compressing but switching to better compression methods can make the differences shown. Theres still huge tradeoffs to be made though. Most GPU formats for consoles require platform specific texture formats, and some of them don't compress well (hello dxt). You might have to trade a 20% file size reduction for a 30 second increase load time. Is that worth it? Maybe.

Either way, it's not just "dumb Devs". We care, we work hard on this.

> it's not just dumb devs

I may have come too rough in my comments and lost the original intent. I was hoping to defend that devs _already have_ access to good compression tools and that they already use them, and 70gb games are that big after compression is applied. That's where my assumption started.

I understand well that there are custom formats for different platforms, and that the whole reason these tools exist is not to squeeze an extra 10% smaller files, but to get fast load times while maintaining small file sizes.

> My previous project was 500gb

That wouldn't surprise me to be honest. It makes perfect sense.

Question: why can you not separate the different graphics tiers as optional addons? That way, I only have to download high resolution textures if my PC can actually handle them.

Edit: never mind. Saw this being discussed elsewhere in this thread.

But that's kinda the point, when you are on a low to mid tier setup you now have to download 50GB of 4k textures you're never gonna see.

Just get up close to a model, and you can still see the difference.

And there are YT videos that call developers out for mushy textures on every new game release.

For the steam deck valve added support for games to only include the relevant texture quality tier to save space. Hopefully that expands and rolls out more broadly, but it's also a bit awkward to specify your game quality settings before launching the game.

Both the current Xbox and PS generations support this in a way as well. The developers need to do some work to make use of the features, but they are available.

I looked through the generated binaries from unreal engine, and they inline sort algorithms almost everywhere in their code, so even their binaries becomes huge not just the assets. Inlining sort algorithms is in my experience a very common cause to binary bloat, and not at all helpful for performance, forcing the cpu to load that much code everywhere hurts performance. It will look good in local benchmarks though since then the CPU can keep all the code in its cache.

> I stopped playing video games almost altogether because I'm not going to download 70 GB

This is why some people love game streaming. Esp true if you only play a game once in a while.

Game streaming presumably needs a high bandwidth, low latency internet connection? If so that's exactly the connection that makes a 70GB download something that can happen in the background while you start playing

Or why others still love FITGIRL repacks.

> Most games render frames of a 3D world in less than 17ms, but most websites take 3-7 seconds to load

Most (many?) games take minutes to load. Perf is nice once they're loaded (assuming you've got sufficient hardware), but loading is a drag. Modern load times are worse than all the fiddling it took to get NES games to start.

Maybe on consoles. The only PC games I've played in the last decade that took minutes have been GTA5, Destiny 2, and heavily modded Minecraft. Most games load in under 15 seconds.

If anything, it's the other way around. PS5 games often don't even have loading bars, since they assume almost-instant streaming of assets from the SSD.

I like how 15 seconds is considered a decent bar.

15 seconds is my upper bar here, and it includes known, very poorly optimized, alpha games. Not once did anyone use the word decent here, you are imagining unsaid things.

Subnautica takes ages to load.

Makes it scarier!

Just check how much GPU memory games are using, it take a while to load a process the assets before they are ready to use, so the bottleneck is mostly IO I believe, AMD and Nvidia were working on improving that by loading assets directly into the GPU instead of first go to the CPU+RAM and then the GPU.

IO is also the bottleneck on the web, unsurprisingly.

I don't think so.

My Internet connection saturates my PLC at 10MiB/s, my ping is under the 15ms, and Twitter still takes ~2.4s to load and render a random tweet.

Twitter is a terrible piece of garbage website, isn't that why everyone uses it as the ultimate example of the web sucking?

Twitter wants you to download their app and sign up so they intentionally neutered their web experience. It would probably require 0.1% of their 7500 employees to spend 3 months to fix it entirely, but it's business that drives the bad tech, not the developers

I build internal apps that load collaborative an entire app with MBSE capabilities that have a time to interactivity of < 10s. Google docs takes like 7s to spin up a new doc and interact with it. I can promise you Twitter could reduce rendering some stylish HTML/CSS and a 280 character tweet to a single round trip and see maximum 0.5s browser parse & paint if they wanted to

Taking a quick look at a random tweet through the network tab, about 2/3rds of the time spent is on downloading files. Now should Twitter preload a bunch of videos from recommended tweets it shows below the tweet? Probably not. Does Twitter need megabytes of JavaScript just to show a tweet? I'd hope not. Those things do appear to be the main bottleneck however.

I don’t know, is it? I get it, sure data has to be loaded, but the problematic part is the work being done on that data, this is what takes much more time than it should reasonably take.

Just take the example before, if twitter were server-side rendered, it would likely load in basically a blink of an eye.

The problem is code, which will later do more network calls which again have to be waited on, this is multiple round trips. Also, the fault lies probably a bit on web tech as well, html+css is just a terrible abstraction, layouting not being properly solved.

Network calls are IO reducing them greatly speeds up websites.

A network call waiting on the result of another call is still two times as long as it should/could be, no matter how efficient you make it.

Optimizing your IO is indeed a good idea. Back in the day we used to order game data on CD/DVD for efficient streaming.

Maybe a partial solution will be to use qwick approach, load js on demand when needed, instead of all at once, while ideally will be use strictly the necessary js (that in some cases could be none).

That's not true.

I dunno it definitely seems that way. In our own product reducing IO significantly sped up time to live.

> Most (many?) games take minutes to load.

What are you loading them off? HDD? The only time I experienced a load time longer than a minute for any game was on a PS2 game that was scratched to high hell. It's possible that I'm simply interested in games that don't have long load times but I've played enough of modern AAA on console and PC to never have experienced this first hand.

F1 2020 on Xbox is horrible, loading all the time. Loading once in the beginning would be ok, but it’s loading between every step. Sometimes it feels you spend more time waiting for the loading than actual racing.

That's bad design then. The Witcher 3 has no loads and rhe world is massive. As OP said, priorities.

A minute seems like a long wait for software that brags about its performance

I haven't noticed loading times in many years, basically since SSD replaced HDD.

That has never been different. Games push the envelope a lot. Especially if you don't have the best of everything.

Some awesome games were small. Other used a full 1.44" FDD. Some used 4 FDDs. Then the same happened with CDs. Finally only one CD needed. But there's space left over so later games made full use of it and soon there were games that needed 2 or more CDs. Lots of loading even going from one place to another. Decisions like "do I really want to go there now, I'll have to put in CD3 again". Or you bought the bigger HDD for much more $$$ and could install all 6 CDs onto that instead.

Can't run "Apache Longbow", need more RAM. But I can either buy the RAM or the game with the money I have. Damn!

What's been a blessing for me is not having the time to actually keep up with any of this time wise. Not enough time to really warrant spending money on new games and new hardware. Now I play games that are years and years old, I can get for cheap on GOG and that run fine on Intel Integrated graphics chips. Coincidentally those games usually load just fine in almost no time on modern SSDs as well and since I can't crank up the graphics full throttle, loading all that stuff into memory is relatively fast too etc.

It does depend on the game, but it's a good point. We do this 'cache everything at the start and avoid creation/destruction' during runtime thing that can lead to slower startup times.

After initial startup though, it depends a lot on the game itself how much loading you will wait for. I think SPAs are sort of the equivalent of zero-loading-screen games in some ways: both are trying to eliminate that loading pause.

Who is we? BR games have one map and reload it every time they start and even loading the menu takes extremely long. Then they go on top make the time between matches solid minutes to wait for people to be pointlessly loading the map. Fortnite for example (haven't played it in years but it was like that in the first 2-3 years). There's literally no excuse. Now that I got myself ranting on BR games I'll note that their quality is an order of magnitude worse (bugs, performance) than games with 5 minute rounds, while they have 30 minute rounds. Meh anyway I guess I'm attacking your random thought too hard.

Definitely correct me if I’m wrong, but games have thousands of little cores generating those frames every 17ms, but websites usually rely on a big slow cpu to direct the workflow of tiny packets from some far off network, no?

Yeah, but the list of things to render still has to be presented to the GPU in less than that time, from the CPU (draw call).

Games also rely on the CPU to do simulation and feed the GPU. And Browsers also use the GPU for the actual rendering where possible.

And for the love of God, if it's a single player game let me pause while it's loading (which shouldn't interrupt the loading, but should make it start in a paused state).

most websites take 3-7 seconds to load because of all the ads and bloat, and things shift around on you for another 20 seconds after that, so when you go to tap a link you accidentally tap an ad that finally loaded under your finger

The problem with remarks like this is that if this was really your experience of using the web you'd have installed an ad blocker years ago, and you wouldn't make that argument now. Consequently either it's easily dismiasable as hyperbole or you're a masochist who enjoys terrible websites.

There are bad websites. It isn't "most websites" though. Just like some games drop frames horribly, but not "most games".

I'm using an ad blocker, and with a cold cache, clicking a Twitter link shows a blue bird for a second, then two blue spinners (center and top) and then after 3 or 4 more seconds I can finally see a message that's a maximum of 280 characters long. News sites don't load much faster. Pages like HN are the exception, and mostly come from the tech corner. I've been on some brand's online store a few days ago and it felt like they want to drive customers away.

But fair enough, it might not actually be most websites, but enough to not be a rare experience either.

I just checked twitter's network log on a single tweet by John Carmack.

- 131 network calls

- 280 kilobytes of data received

- 4500ms total network time

Thankfully, the vast majority of other websites aren't nearly this bad, but we're trending towards twitter as an industry, not away from it.

I try to write API calls that provide broadband utility and return lightning fast by always making sure my critical path gets a fair shake for optimization, but I know there exist calls out there in react land that take my 220ms response and spend an extra 7 seconds transforming it to populate a (paginated) table on the UI.

Amount of data transmitted seems fine...but 131 network calls? No wonder it always feels so slow to use Twitter.

Keyword and. Ads and bloat. Adblocking doesn't get rid of all bloat.

And GP's point still stands. 17ms vs the typical 2s minimum a site takes to load is massive given the differences in what is demanded. Games actively got faster despite being more intense, websites active got slower despite relatively similar output.

Webdev's performance is shocklingly bad relative to most other subfields.

Comparing the loading time of a website to the frame render time of a console game hardly seems fair.

Plenty of websites scroll at 60fps or 120fps just fine (and those that don’t definitely deserve to be criticised).

How many console games can load from scratch in only a few seconds?

Your comparing gigabytes of data to a few MB. Now that hardly seems fair.

This is a reasonable point, _but_ how much data do games need to load versus a website. Using Twitter as an example, when I open it I can see 4 tweets. Assuming each tweet is 1MB (for easy math), I have a 500Mb WiFi connection, I should be be able to do 100 tweets in under a second. Meanwhile most/many games are pulling in gigabytes of data in textures and audio data. Spiderman on PS5 is under 15 seconds from the home screen to being in game, playing which is about how long it takes for the Facebook mobile website to load.

Comparing latency and throughput of local and remote assets says nothing.

GP's point ignores the load times of a game (the time it takes to have a usable game-no progress bar), and instead compares FRAME time (17 ms to render a single frame) with the LOAD TIME of a website.

You too ignore this.

You might know this already but in case you didn't: ad-blockers are the reason browsers are considered greedy pigs when it comes to resources.

It's no longer a universal truth that a adblocker will save you memory or CPU, ad-blockers contribute greatly to the per-tab memory cost. They also slow down page rendering greatly.

But, hey, the web is an unusable mess without them so we all use one and it's considered the cost of doing business.

noscript can block most ads without much memory overhead

> I'm a game developer and game performance is better than ever.

It could be, if users would have more control over the graphics settings of the games that they want to play and they were allowed to scale back further to accommodate older hardware.

For example, the Unity engine by default has a setting to allow downscaling the texture resolution that the game will use 2X, 4X and 8X, yet many games out there actually disable that. Same for some options menus not having framerate limits, dynamic render resolutions (though most engines support that functionality in one way or the other), particle density/soft particle options, options to disable SSAO/HBAO or other post processing like that, as well as enabling/disabling tessellation.

The end result is that many games that could run passably on integrated graphics or hardware from a few generations ago (e.g. GTX 650 Ti) instead struggle greatly, because the people behind the game either didn't care or didn't want to allow it to ever look "bad" in their pursuit of mostly consistent graphical quality (and thus how the game will look in the videos/screenshots out there).

The only real exception to this are e-sports titles, something like CS:GO is optimized really well for performing across a variety of hardware, while also giving the user the controls over how the game will look (and run). Games like DOOM are also a good example, but they're generally the exception, because most don't care about such technical excellence (though it's useful when you try porting the game to something like Nintendo Switch).

Most other games don't give you that ability, just because they try to always do more stuff, which isn't that different from Wirth's law (software gets slower as hardware gets faster). Of course, this is also prevalent in indie titles, many of which don't even have proper LOD setups, because engines like Unity don't automatically generate LOD models and something like Godot 3 didn't even have any sort of LOD functionality out of the box.

Engines like Unreal might make this better with Nanite, except that most people will use it for shoving more details into the games (bloating install sizes a bit), instead of as a really good LOD solution. That said, Godot 4 is also headed in the right direction and even for Godot 3 there are plugins (even though it's just like Unity, where you still need to make the models yourself), for which I actually ported the LOD plugin from GDScript to C#: https://blog.kronis.dev/articles/porting-the-godot-lod-plugi...

How prevalent is a lack of graphics options really? To me it’s an expected piece of functionality in a PC build and in most games the first place I go to and turn DoF and motion blur off!

It's extremely common in PC ports of console exclusives/console first games.

If the game is popular then someone usually comes along with a DLL that allows you to tweak at least some of the settings that weren't exposed.

I absolutely hate motion blur and depth of field, even if my computer is fast enough to render them.

You'd be surprised. Here's a Wiki that attempts to document various configuration options, tricks and utilities for many games and they have tables with what features are supported in the games that are featured on the site: https://www.pcgamingwiki.com/wiki/List_of_lists#Video

While it's not an exhaustive list, you can open any of those features and sort the table by whether the game supports the functionality itself, needs some sort of a config/tool fix/hack or does not support in any known way.

(the list is so long that you might have to view it with 500 items per page and even then jump through multiple pages, I wonder whether the dataset can have aggregate queries against it done)

> Most games render frames of a 3D world in less than 17ms, but most websites take 3-7 seconds to load because of all the ads and bloat, and things shift around on you for another 20 seconds after that

I know what you want to say (and I agree), but... website rendering depends on the network mostly, 16ms latency is already the top 0.5% of fiber users and you have to add that on top of every new connection... GPU rendering happens on a bus that is thousands of times faster, and it needs to cover a minuscule distance. You can't really compare the two.

Agreed. Why is it taking my garden hose so long to fill my swimming pool when tsunamis can do it in a matter of milliseconds?

Differences in order of magnitude.

That said, websites and even web apps really should be faster.

I have 1 ms latency to 1.1.1.1 and a 10 gbps connection - faster than a SATA SSD - and most websites are still slow.

Bandwidth =/= latency, and congrats, you are a very small percent.

I mention both independently. And the point is that transport is not the bottleneck.

That's the protocol and dependencies. Game assets are sequential and can be paralllelized most times.

True, although even without ads nor a bloated stack, you start with network latency, code running in a VM that has to use hardware with terrible embedded gpu, accessibility requirements, seo, and so the obligation to use the dom api, the latter not being accessible throught a compiled language.

Also, the website teams is usually smaller, and have a 10th of the time to ship.

But it doesn't excuse everything, for sure. Most websites should be fast given how little they do.

> That, and... performance tuning is hard work

Most 'performance tuning' out there is eliminating quadratic functions and putting stuff into simple dictionary caches. This should be easily teachable to any competent developer.

The problem isn't 'hard work', it's nobody cares about this.

That is not the lowest hanging fruit (where I would put trivial changes like inlining functions or switching an of-the-shelf library by a faster alternative). But it is medium-low hanging fruit.

Getting measurements that help you identify where the bottlenecks really are can take time. Often these have to be retrofitted into existing complex system, when the project is already underway.

Sometimes the only way forward is replacing the current way of doing things with a more efficient way of doing things. This requires understanding the current way well (it can be the result of many teams working together for months) as well as coming up with a new design. Both of these things take time.

Finally, there's bug hunting, where you get into rabbit holes. "There's a slowdown when we're on this particular step. You investigate for days, until you realize that the slowdown only happens when the user mouse is near a section of the screen. It turns out that there's a non-visible UI element that should be inactive but it isn't because it is a custom element instead of one provided by the UI library, and it is attempting to render a special effect but finds no graphical context, so it request for a new context once per frame." And one week of dev time has passed.

Yes, last part of performance tuning is relatively simple process as you described, but that's only because apps where performance matters are developed from the ground app with performance in mind, so entire architecture is designed in a way that supports performance.

Now try to take some React developer that has been working for years in completely different abstraction level, and help him design more performant code base.

You really think all you will talk about are cache misses?

no, but you can definitely talk about it.

i also had a chat with a C++ dev that created a "Database"-class abstraction for our MongoDB instance and implemented methods to access each field of every documents separately - didn't care about side-effects, didn't know about ACID transactions and didn't care.

finding a good abstraction that allows for performance tuning later on is hard, no matter the language, and react devs should make use of good abstractions as well.

That’s just good software development practices. Game developers and scientific software developers borderline abuse the software (language, drivers, etc.) and hardware to get these performance numbers.

For example, I need to reorder a matrix and change my access pattern to it to extract more performance and reduce cache trashing by exploiting cache prefetch dynamics of a CPU.

I need to test this with every CPU generation and test whether it works as expected performance-wise.

To achieve this, I need to rewrite half of the said software.

The reason I’m not doing this is it’s still way faster than other software I’ve seen in that category, and extreme performance tuning can hurt flexibility of the code down the road.

Isn't it another example of misaligned incentives? Your website renders on my computer, making rendering performance mostly my problem. Your website is only financially viable because it includes Google adverts and tracking, and Google again isn't paying to run those on my computer.

Backend developer here.

Most (non FAANG) sites are a mess because they're made by inexperienced developers turning everything they touch into a tangled slow mess.

The process goes like this:

- Devs ditch the legacy (often monolithic) code

- They start creating microservices fron scratch

- They don't see the need to optimize or even make readable code. It's a clean slate! the project is small, the new code is just whatever they write.

- They do that with the idea that the legacy code is "bad" and the new code is "good"

- Requirements either scale or downright change

- To the code made before, they add a "tiny" special case and patch here and there. After all, the code is "good"

Repeat

Now you have a new messy project. The dev will either quit or work in a new project. The next dev will do the same. And the new team will probably push for another replacement from scratch, so they can feel that they re creating "good" code again

> Most 'performance tuning' out there is eliminating quadratic functions and putting stuff into simple dictionary caches. This should be easily teachable to any competent developer.

That does happen, but some of the more severe performance issues I've seen are memory related. Every byte allocated has to be freed. If you lower the memory profile of your function then you reduce the overhead of running the code significantly.

I recall a web service that ground to a halt and would sit at 100% CPU, after I put it in a profiler 97% of the cycles were in the garbage collector. Dig a little deeper and I found that the caching library used was cloning the data before sending it back. Ahh, junior devs.

I rewrote a library with the same interface that did not clone the data, this was safe as the original data wasn't being changed, 15x speedup from 10 req/s to 150 req/s.

Then I dug deeper and changed which data was getting cached. Cache post-render and post-compression instead of the pre-render database calls. 150x speedup from the original or 10x from the previous point. The system went from being able to serve 10 requests/s to 1500 req/s. This change also was safer since nobody is going to ever try to alter compressed data, no such guarantees from the cached database calls.

GC has not been an issue in Java for years. I haven't seen memory issues, except for services running in ridiculously CPU or memory starved kubernetes containers.

This could be Python or another language.

I know, just sharing my specific experience.

A sibling poster also comments that dictionaries are too slow. They might be in a game setting, but definitely not in a Java backend service.

Well, for starters... dictionary hashing is too slow to be ideal. We don't swear off dictionaries, but we try to get our constant-time lookups without needing a hashing step when possible. The dictionaries can also lead to extra cache misses, since the memory often won't be compacted as you step through your data.

In most games this doesn't matter, but occasionally you have a frame time target and you need to take extra steps like that. Those extra steps can be hard work when they come up!

> dictionary hashing is too slow to be ideal

Even when you use SIMD intrinsics?

For example: https://en.wikipedia.org/wiki/Intel_SHA_extensions

A repo I just found illustrating those instructions' usage: https://github.com/noloader/SHA-Intrinsics

Not the OP, but those extensions implements cryptographic hash functions, they are not very suitable (too slow) for hash-tables and other hash based data structures.

4 CPU cycles is too slow? Because SHA256RNDS2 is 4 clock cycles on Zen3 CPUs. 6 clock cycles on Intel Alder Lake (12th gen).

Source is the measurements here: https://uops.info/table.html

Yes, because SHA256RNDS2 aid the implementation of the update function of SHA256. We should invoke it multiple times on the message blocks, it performs two rounds out of the 64 SHA256 rounds.

> We should invoke it multiple times

That'd make SHA256RNDS2 even faster because of the throughput numbers.

I'm curious, what's the faster algorithm for hashing you use? I don't want to waste time on intrinsics and inflict complexity on my colleagues if there are alternatives I can use!

I’ll just use xxHash family, murmur hash family or even FNV. For crypto, BLAKE2 is a cryptographic hash that’s faster than SHA,MD5, etc and is at least as secure as the SHA-3.

> (note: games lag as well when you drag in a million dependencies you don't need)

No they don't. Your compiler will eliminate dead code.

It think the implication was that the dep is used but not actually needed.

That is a flattering interpretation.

Parent did write that "game lags as well when you drag in a million dependencies".

That is hard for me to interpret like you did.

Optimizing Websites is easy: turn on an ad locker. I was surprised how my fans of my pc went up once I turned off uBlock Origin once

Older readers will remember when you'd buy a computer magazine printed on paper and at least 80% of it was ads. Which often didn't change from month to month.

Massive slabs of wood pulp had to be printed and shipped to all to the stores that stocked them, at huge cost, just so you could manhandle one of them home.

And the content was mostly physical bloat.

That's the modern web. Except that you don't just get the ads, you get some of the machinery that serves them - at you, specifically, based on your browsing profile, which is used to decide which ads you see.

I always refuse all cookies. When I forget to do that for some reason it's obvious just how much slower the experience gets.

> That's the modern web.

Harsh but true.

You don't have uBlock Origin running?

> Most games render frames of a 3D world in less than 17ms, but most websites take 3-7 seconds to load because of all the ads and bloat

Well now, "most games" will take longer than 3-7 seconds to load. That's fair - they're doing a lot more than websites!

Most websites also render frames in less than 17ms.

When I'm working on a web project, like my "PlotBinder" WIP, I spend way more time trying to hit 60fps than I do in my game(s). Adding little css animations or transparent overlays cut the framerate down to 40fps on my old laptop until I tinkered for a while with it and made some compromises. The way pages render on the web is finnicky to optimize for.

Figma is an excellent example of performance being a top priority to set themselves apart, since they weren't the only player in town at the time.

We had Adobe Illustrator, XD was still in beta phases, and Sketch for macOS was showing signs of bloat and major performance hangups in any considerably realistic document. Affinity Designer was also coming into the scene with stellar performance, but had a high learning curve and wasn't well suited for interactive prototyping.

Figma swooped in and solved 3 problems:

1. Availability in the browser on any platform, where native apps can't reach (and thus 10x easier sharing of documents). 2. Incredible performance on first release through their WebGL renderer, rivaling all of the native apps listed above. 3. Stayed flexible, and yet, barebones enough to get at least 80% of a designer's needs covered initially.

Performance (and stability) were primarily what won me over to it, and I'd argue probably the same for many who switched over.

Illustrator's bad performance amazes me. I don't know about newer releases, but CC 2020 is basically single-threaded for everything. It's crazy to see it wait for 8 seconds to make a png vectorial, while using 5% CPU on a 16 core 3950x.

Bingo! That about sums up my memories of using it every since it went CC-anything. I think CS5 was the last version I used that felt pleasantly performant -- and on older machines, no less (2013 Macbook Air w/ i7, 16GB).

> I'm a game developer and game performance is better than ever.

I'm a game developer (and player) and I strongly disagree. What game can even do P95 144FPS on affordable hardware? On lowest graphics, with populated lobbies (there also can't be a that ONE map where this all goes to shit). And are we talking about AAA games or not? Because non-AAA means a minecraft knockoff cannot meet 60FPS.

Websites can def be fast, but to your point it’s a priority problem.

This comparison I made at https://legiblenews.com/speed that got upvoted here a while back shows just that—most news websites care about ads first before their users speed experience. Only Legible News, USA Today, and Financial Times have a reasonable speed score, which is kind of depressing.

> and things shift around on you for another 20 seconds after that, so when you go to tap a link you accidentally tap an ad that finally loaded under your finger.

Why don’t browsers mask out events during render of page regions or note their time and bounding boxes so click events reach the correct element?

Steps to reproduce this annoyance:

- visit Twitter mobile site: mobile.twitter.com in Safari on a slow iPhone (eg iPhone 6 stuck on iOS 12.X). - scroll and view tweets for a while soaking up memory. - visit a tweet - press tweet share icon - popup menu takes forever to render fully to include the Cancel button. - clicking Bookmark tweet menu item will trigger item after that, Copy Link to Tweet, when it finally loads and moves that menu item under your finger.

> For most software, functionality, signing up for a subscription, platform availability etc are usually prioritized higher than response times and keyboard shortcuts.

This is why I love working in fintech. The engineering is paramount. Customers will not accept slow or buggy software.

I get to solve hard problems, and really build systems from the ground up. My managers understand that it is better to push back a deadline than to ship something that isn't up to standard.

Sounds like places Id love. May I ask what company youre working for?

Now I want to work in a fintech

Yes, but all in service of what amounts to automated stealing (cough) supplying liquidity.

Better than adtech, anyway. Or nukes. Lots of things, really. (I would have said weapons, last year.)

The only really defensible tech activity these days is things to help get off carbon-emitting processes. Making factories to make electrolysers. Making wind turbines better. Adapting airliners to carry liquid hydrogen in underwing nacelles. Making robots to put up solar fences on farms and pastures. Banking energy for nighttime without lithium. Making ammonia on tropical solar farms for export to high latitudes.

It's even money whether we can get it done before civilization collapses. I guess we will need plenty of liquidity...

Those are worthwhile problems to solve, but they seem well on their way to getting solved, though it's not guaranteed. Your civilization is collapsing because you're fighting each other, not because of climate change. Fighting each other is also why it's not guaranteed.

Other problems are also worthwhile to solve. I agree that zero-sum HFT and negative-sum adtech and arms races are not the most prominent ones.

If we spent on those what we spend instead on adtech etc., it really would be guaranteed. Even with fighting invaders. Albeit probably not invaders using nukes.

I think 228 GWp of solar power generation capacity is being installed in 02022, which is about US$50 billion for the solar modules and another US$50 billion for balance of plant, not counting things like battery storage, pumped storage, transmission, and Fischer-Tropsch. Wind energy generation capacity is getting installed at a similar pace, in terms of peak gigawatts per year, but I have the impression that it is significantly cheaper. Tesla's revenues are US$53.8 billion per year, of which about US$0.07 billion is solar panels and therefore getting double counted. I don't know how to measure other automakers' EV production, housing switching to heat pumps, housing adding insulation, and R&D, so I'm just not counting them.

Alphabet's yearly revenue is US$258 billion, some of which is earned from things like Google Play, Google Cloud, and Google Workspace, and some of which has been spent on things like Makani, Verily Life Sciences, DeepMind, and Google Fiber. Meta's revenue is US$118 billion.

https://www.globaldata.com/data-insights/internet-services-t... says, "Google earned nearly 81% of its total revenues in 2021 from advertisements," and it's talking about that US$258B number, not some smaller number that excludes other Alphabet companies.

We can maybe estimate the whole adtech spend, generously, at US$400 billion per year [edited: was US$300 billion]: most of Alphabet, all of Meta, plus a handful of smaller fish. (I say this is "generous" because it's counting all the effort that goes into running Google Search, YouTube, Instagram, and Android development as "adtech", because it's monetized through ads, even though most of the people working on those projects are not directly concerned with ads; if you omit that, you might size adtech at only US$50 billion.) And maybe US$200 billion per year goes into the renewables transition. I'm not convinced that doubling that to US$400 billion per year would guarantee a win, particularly when the humans are going around bombing each other's power plants, snarling up power plant construction and energy-efficient housing in red tape, and publishing disinformation that "debunks" global warming.

These are of course very rough estimates, but how would you estimate these numbers?

As far as I know, there aren't any invaders the humans are fighting; they're only fighting other humans, plus the occasional rogue elephant.

Maybe boosting $200B to $500B would provide a win, but ok, maybe not.

Invasion is an activity, not an identity. Those so engaged could stop anytime. Many will.

Nobody is, as far as I know, invading the humans' territory from someplace else, such as Delta Centauri. Humans invading other humans' territory is just a sort of humans fighting among themselves, but not, currently, the sort that is the biggest obstacle to the renewable transition.

Agree. And maybe it will encourage Europe to build out renewables and heat pumps faster.

Apparently it has not had that effect in Romania, yet. But they are by report still happy with their Canadian CANDU reactors.

And fission

> Your civilization is collapsing because you're fighting each other, not because of climate change

10% of all deaths each year are due to air pollution. There's pretty much no place on earth where unfiltered rainwater is still safe to drink. Ocean acidification is accelerating and killing off the organisms that are both the biggest producers of oxygen and the biggest sequesterers of carbon.

Air pollution is not due to climate change, unless you count CO₂ as air pollution, but that's eargrayish misreasoning — that's not the kind of air pollution that causes 10% of all deaths. The sort of local air pollution we're talking about amounts to factories strip-mining local children's lungs for profit; it's a form of fighting each other.

Rainwater harvesting risks seem to be primarily due to birds pooping on your roof and other fecal contamination, and they can be adequately mitigated by boiling or chemical sterilization: https://www.nature.com/articles/s41545-019-0030-5.

Climate change very likely would collapse your civilization, given enough time, but that isn't what is happening now.

I think we have more than just environmental issues in this world...but sure.

Not for long, unless...

Who would they be stealing from?

The body politic.

People program computers to exchange messages furiously, to no external effect besides power consumed. Then they go buy yachts. It's all perfectly legal, of course.

I'm not sure how this 'stealing' is supposed to work.

The stock market is a zero-sum game. Every single penny taken out was a penny put in. Money going in is people's pensions, largely.

As a society, we like to reward people who do things that are useful: feed, house, clothe, warm, transport, educate, protect, medicate, lately amuse people, and get money that may be exchanged for those services, or yachts.

HFT traders provide none of those, nor anything comparable, but get yachts anyway.

Well traders do provide liquidity which saves pensioners enormous sums compared to selling retirement assets at will into illiquid markets

Granted traders can play other games, but their primary benefit to others is significant

>The stock market is a zero-sum game. Every single penny taken out was a penny put in. Money going in is people's pensions, largely.

Ah, so you lack a basic understanding of how the stock market works, but think you're qualified to make very strong statements about it.

Trades batched by second would provide as much liquidity as anybody could use, without the yachts.

You can start a new exchange that does this batching.

Btw, you need to be careful, because people might still race to be the last (or first) to contribute to a batch.

They'd race to be the first, if tied prices within a batch are resolved in favour of the earlier bird. Otherwise, they would race to be the last, so they have the most information available when deciding on their price (and so that they can change their mind until the last nano-second.)

And if ties within that second were resolved randomly, what perverse optimizing would you predict?

Well, there's education - that's net good. And I think entertainment is pretty neutral (even slightly positive) if it doesn't involve gambling or rely on player addiction. Fintech and adtech are where people bright enough to contribute positively to society go to make money instead, kind like the oil industry. All IMHO, YMMV, etc.

> The engineering is paramount.

You must be lucky, not everywhere in the sector that’d be remotely close to true unfortunately…

He's probably referring to the trading desk or similar teams.

Stock traders forced a lot of advances onto software. Random examples of high-perf stuff from that space include the new garbage collector in the JVM with a minimal pause time and LMAX Disruptor. Multi-threaded GUIs are relatively common in that space as well, to ensure that one hung control or window won't stop anything else.

I've known a handful of software engineers who have stopped through The Trade Desk for a year or less. It doesn't sound like a great place to work to me, despite the high throughput their software demands.

> This is why I love working in fintech.

The prototypical nerd would happily work on building a nuke capable of destroying a continent if that entailed letting him work on “hard problems”.

I wouldn’t work on a continent-destroying nuke unless the project used Rust. Wouldn’t want it to be unsafe.

Obligatory - https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98...

Missiles don't really need GC or RAII. They have their own special resource clean up.

Oh you mean the Manhattan project?

I don't know what you mean by "fintech" but from my experience, bank and other finance apps are usually not that great, neither are the websites. So maybe some parts of fintech are nice and clean, but the part that the end user faces, not so much.

If I am not mistaken, fintech refers to the software/hardware surrounding trade execution on the various exchanges (stocks, bonds, commodities, etc.) where speed is of primary importance.

Yeah I work in finance too and it's a mixed bag. The stuff I work on right now has a very high focus on correctness and stability which is nice.

But I've also seen productionized jupyter notebooks written by just out of school data scientists and 1000 lines long SQL queries that encoded important business logic and weren't version controlled.

I mean the actual tech than enables financial markets (fintech) - high frequency trading, matching engines, market data, order execution, etc.

This has pretty much been the exact opposite of my experience in fintech. But it's a startup so maybe that's the difference

The way that game developers get their performance is more or less orthogonal to the way many other applications are expected to function.

It is impressive that they can draw so much stuff so fast, but there are actually very few objects on the screen that the user can directly interact with.

A specific example: in a DAW, you might have tens or even hundreds of thousands of MIDI notes on the screen. These look like (typically) little rectangles which are precisely the short of thing that games can draw at unbelievable speed. But in a DAW (and most design / creation applications), every single one of them is potentially the site of user interaction with some backend model object.

All those complex surfaces you see in contemporary games? Very nice. But the user cannot point at an arbitrary part of a rock wall and say "move this over a bit and make it bigger".

Consequently, the entire way that you design and implement the GUI is different, and the lessons learned in one domain do not map very easily to the other.

What? What are you talking about? Games handle thousands of colliders and raycasts just fine. I thought you would mention that games get to use dedicated GPU hardware and apps might not be hardware accelerated but colliders? Very odd take.

You've missed my point entirely.

Games (generally speaking) do not involve allowing the user to interact in arbitrary user-defined ways with every object represented on the screen. The interactions that do happen are part of the physics engine not the "user clicked that and dragged it to the left" engine.

No I think klabb3 has the right of it. They're really not that different and it does just boil down to priority and what is fine to be outsourced to a huge stack of slow libraries and what is prioritized as a core competency.

Yeah, go check out some Factorio screenshots. Thousands of interactive objects animating on screen at once. Not only that, there is a complex sim running in the background. On my machine it runs well past 100fps.

Or 2d strategy games. Which had very decent number of animated units each with their own actions and which you could click and select...

It took the devs years to optimize it to that point though. And I'm quite sure that a screen full of trees or smoke emitting entities (thankfully you can turn smoke/steam off in settings) still makes it go below 60fps on my laptop.

Optimize, yes. But then... It's doing several (hundreds) of orders of magnitudes more than a typical app.

Several hundreds of orders of magnitude more? That doesn't sound possible.

That was an exaggeration but only slightly I guess :)

Considering how many objects and interactions Factorio tracks and at the same time lets you move around, inspect any of those things etc.

A typical app on the other hand does a CRUD operation every once in a blue moon.

> Games (generally speaking) do not involve allowing the user to interact in arbitrary user-defined ways with every object represented on the screen.

Ah yes, and which apps actually allow you interaction in arbitrary user defined ways?

IIUC, in a game, let's suppose you pick up an object. We'll ignore the fact that you probably provide some sort of fairly broad movement to indicate that you want to pick it up - the game just notices this, and your on-screen avatar takes care of the rest. But now that you have it in your hand, you do not actually have control over in a direct sense. You may indicate that you want to lift your arm - all fine, but that information will be fed into a physics engine that will work on actual speed & trajectory, obstacles and collisions etc. What is shown on screen is not the result of you dragging a GUI object across the screen, but the indirect result of your indicating "lift my right arm" etc.

By contrast, most design/creation applications give you the ability to point with the mouse (or touchpad or whatever) at some object on the screen and then move it (sometimes along 3 axes). The speed of the motion, the axial extents and the destination come (at least initially) from you. This is true of drawing applications, design applications, DAWs and lots more.

This is nonsense. First, no one can interact in "arbitrary user-defined ways" with anything. Second, anything you can imagine on the web and more has already been done in games. Please don't tell people that running a 3d strategy game or a AAA multiplayer first-person shooter is somehow less complex than scrolling an effing window with a bunch of text and forms in it.

Eh, the actual number of interactive items in a modern game is significantly less than thousands. Probably more like a few dozen when all is said and done.

A user of Blender can absolutely point at an arbitrary part of a rock wall and say "move this over and make it bigger". Blender sacrifices rendering quality to make sure that interaction is reliably responsive. Cube/Sauerbraten forgoes some of the rendering optimizations provided by some other 3-D game engines to make sure you can always edit any part of the environment at any time, but it was already delivering interactive frame rates 20 years ago. And of course Minetest has very little trouble with arbitrary sets of nodes appearing and disappearing from one frame to the next, but Minetest isn't that great at performance, and its expressiveness is a bit limited compared to Cube and Blender, so maybe it's a less compelling example.

As long as it doesn't cause a glitch in playback, it's acceptable for your DAW to delay 10 milliseconds to figure out which note you clicked on. That's about 100 million instructions on one core of the obsolete laptop I'm typing this on. As you obviously know, that's plenty of time to literally iterate over your hundreds of thousands of little rectangles one by one, in a single thread, testing each one to see if it includes the click position.

But (again, as you obviously know) you don't have to do that; for example, you can divide the screen into 32×32 tiles, maybe 8192 of them, and store an array of click targets for each tile, maybe up to 2048 of them, but on average maybe 64 of them, sorted by z-index. If a click target overlaps more than one tile, you just store it in more than one tile. When you have a click, you bit-shift the mouse coordinates and combine them to index the tile array, then iterate over the click targets in the array until you find a hit. This is thousands of times faster than the stupid approach and we haven't even gotten to quadtrees.

A different stupid approach is to assign each clickable object a unique z-coordinate and just index into the z-buffer to instantly find out what the person clicked. This requires at least a 24-bit-deep z-buffer if you have potentially hundreds of thousands of MIDI notes. But that's fine these days, and it's been fine for 25 years if you were rendering the display in software.

Blender, as a design/creation tool, is something I would classify as closer to a DAW than most contemporary video games. The fact that you can use it to create games isn't central.

Its performance is very gamelike, I think, and your point seemed to be that design/creation tools like DAWs can't get gamelike performance, so I was offering it as a counterexample. I agree that the fact that you can use it to create games is irrelevant, and that wasn't a fact I was attempting to focus on.

You might be underestimating modern games a bit I feel, but as well game engines usually have editors just as complex as a DAW. Yet both games, game engine editors, DAWs and all the examples you mentioned all run vastly more performantly than many modern simple pieces of software. Which tells us that performance is possible regardless of domain if we build properly.

Yes, but the editors are (to the best of my understanding) not written using the same software techniques as the games themselves.

No not all of them. The godot game engine editor is built in the godot game engine though so that is interesting.

Don't know about the others, but I think Godot's editor is built using its own in-game UI system.

And Godot's editor is very snappy and responsive

UE5's editor is completely built in the engine's native UI toolkit, Slate.

With a modern data driven or ECS architecture, you actually can do that. There are demos out there with hundreds of interactive things on screen at one time. It's kind of amazing.

It's not like games don't have complex UIs either, where each button or field has a lot of logic to them. Many games have very simple UIs, but others get more complex than many complex web apps. Some are even multiplayer, and the server code... it's nuts how much work goes into this. The number of updates you send each second to keep players in a game in sync compared to what is needed to keep a chat app in sync is really impressive to me.

From a technical point of view, games are really cool!

> It is impressive that they can draw so much stuff so fast, but there are actually very few objects on the screen that the user can directly interact with.

> > With a modern data driven or ECS architecture, you actually can do that.

At some point people forgot that there are synonyms and different ways to phrase “actually”.

I feel like the reason a lot of games are slow right now is because they are built out of these huge abstract "everything-doers". For example, each button can be configured in millions of different ways even though the game usually only has a few different types of buttons. It's not an optimal data structure by any means.

It's not like in the olden days where you would blit this stuff out to the screen using specialized routines that scream along as fast as the CPU will let you. Now, everything is dynamic and multipurpose and configured using properties on the object... engines like Unity get away with C# and way too much runtime reflection because it's convenient in the editor. Developers like it, it's less difficult than writing specialized routines.

There are lots of games with destructible environments and thousands or tens of thousands of interactive objects moving around, the static world you're describing hasn't been the Only Way for quite a while. Ignoring the obvious cases like Minecraft, Red Faction and Fortnite, even games where it isn't relevant to the gameplay still implement it sometimes - for example, The Division (2016) had fully destructible walls in many locations and if you fired enough rounds into a concrete wall you could punch a big hole in it that could be seen through and fired through to attack enemies or heal allies. This sort of thing doesn't have to come at the expense of visuals either, modern rendering tech along the lines of Unreal's Lumen & Nanite can adapt to changes in the environment and handle massive dynamic crowds of people or groups of cars.

You can quite literally point at an arbitrary part of a rock wall and move it over a bit then make it bigger depending on the engine you're using. It's why Unreal Engine has got a foothold in TV production.

>All those complex surfaces you see in contemporary games? Very nice. But the user cannot point at an arbitrary part of a rock wall and say "move this over a bit and make it bigger".

The Red Faction series, building games like Minecraft and 7 Days to Die, and games like Factorio are some pretty obvious examples where you're completely and utterly wrong, so I'm not really sure why I should trust anything you said in the rest of your comment.

He's a pretty experienced and expert programmer, but I don't think he's much of a gamer.

> But the user cannot point at an arbitrary part of a rock wall and say "move this over a bit and make it bigger".

In some games, you can.

Could you give some examples of such games? I'm a gamer and I haven't seen anything like that, so I'd love to check it out.

As I mentioned in my other comment, two/three such games are Cube/Sauerbraten and Minetest.

Games have come a long way since Space Invaders and Pac Man.

They routinely do have hundreds, or thousands, of interactive things. Especially things like RTS games. But also, even if you look at turn-based strategy games, which have a much more application-like interface. Every hexagon, terrain feature, and unit is interactive, along with a full application-like menu, statusbar, and UI system.

> Alternative hypothesis: (brace yourselves) people don't care enough. Any vendor will prioritize requirements, if performance is not in there, that CPU and memory is going to be used if in any way it helps the developers. Conversely, by looking at a system you can infer its requirements.

I think most of all, it isn't sufficiently visible. Most development is done on high powered hardware that makes slow code very difficult to distinguish from fast code, even though you can often get 10x performance improvements without sacrificing readability or development effort.

Individually it's just a millisecond wasted here and there, but all these small inefficiencies add up across the execution path.

Here's a fun benchmark to illustrate how incomplete beliefs like "compilers are smart enough to magically make this not matter" can be:

https://memex.marginalia.nu//junk/DedupTest.gmi

It's a 50x difference between the common idiomatic approach and the somewhat optimized greybeard solution, with a wide spectrum of both readability and performance in-between.

If you put zero thought toward this, your modern code will make your modern computer run as though it was a computer from the late '90s.

The sad thing is that a computer from the late '90s running software from the late '90s is extremely fast and snappy basically all the time. Old computers are just so much more responsive.

Honestly, what I want is for more developers to test and optimize their software for low-power machines. Say, a cheap netbook for example (like a Chromebook). I've heard that if you do that, you will be faster than basically every other piece of software on the system. And that speed will persist (multiply, even) on any more-powerful computer.

I've heard of one person who does that for their Quake-engine game/implementation (don't remember which). They get thousands of frames per second on a modern machine. I am guilty of not doing that myself, though. Might pick up a cheap netbook from eBay for around $30.

> Old computers are just so much more responsive.

Because the software on them predates the explosion of GCs, JITs, and hundreds of layers of abstractions & dependencies. All those techniques that improve developer iteration & velocity came at the cost of runtime performance.

Yet the only reason those things exist and are allowed to exist is because we have been training users that snappiness is not to be expected. We have been training them to wait, because their computer is busy and has a lot of work to do. The only reason these things are socially acceptable is because nobody is doing anything about them, and the average user does not care.

Also, improving developer iteration and velocity is possible without doing this.

And on the mobile side you cjisyvthroe your hands in the air and claim network latency is the root cause of your performance problems. I spent some time with a team that did this, and what I saw, and tried but failed to fix, was that the problem is usually an abuse of ORMs and a complete and utter lack of software design.

GCs and JITs are absolutely not at blame here. You could in general write just as snappy programs with them than you could with assembly in the 90s. Hell, a hot loop in Java can be as fast as C code, and it has support for vector instructions as well if you want to write even faster (and platform-independent) code than you could with only C.

Bloat usually happens at the top 2 layers (actual program code and its dependencies), not from the runtime/OS/hardware.

JIT runtimes usually result in (very) slow first execution, and the post in question is specifically about app launch. Responsiveness is not isolated to only after the app has been chugging along for a while.

> Hell, a hot loop in Java can be as fast as C code

Only if the Java code in question is written like C code and not like Java code. Which is sometimes possible, yes, but very very rare, and almost never done.

> and it has support for vector instructions as well if you want to write even faster (and platform-independent) code than you could with only C.

Absolutely nonsense. C compilers are more than capable of cpu-specific vectorization, too. CPU detection at runtime is ancient technology and perfectly compatible with AOT languages. And since offline compilers have the benefit of getting to spend all the time in the world on it, regularly and consistently have much better auto vectorization outputs.

JITs are normally under such a time crunch that it's common for their initial output to be quite mediocre, hence why you end up with multi-level JITs. Which then take even longer before the code stops being slow. Great for benchmarks, but pretty shit in the real world for general app performance.

Now you can sometimes have your cake and eat it too here, such as what Google is doing with cloud profiles on Android to basically run the JIT ahead of time. But that is very far from the norm for JIT'd things.

I’m not talking about autovectorization, but explicit one. Autovectorization simply doesn’t work reliably enough no matter how much compiler time you spend on it, and leaves behind huge performance gains. With C, your best bet is compiler-specific intrinsics, or inline assembly, while java - strangely - has an explicit, cross-architecture Vector API, that can reliably compile down to the architecture’s SIMD instructions.

You mean the brand new Vector API? The thing that's only, oh, 25 years late? That's your example of JITs not being a bottleneck?

And it's not like the JIT actually provides any advantage here anyway. There's no shortage of similarly high level SIMD libraries for the AOT-world.

Many of the game studios near me (I live in Irvine, there's quite a few) test on older hardware as part of their QA process. I don't know what their limits are these days but I remember being impressed by Blizzard when I was visiting their campus once with a friend who worked there. It was fairly comprehensive at the time. According to him, anyway, many of the studios around here had similar practices.

Frankly I'm shocked that deduplicateTree is two orders of magnitude slower than deduplicateGreybeard, despite having asymptotically better-or-equal performance (TreeSet runtime is O(output log output), whereas sorting an array is O(input log input)). I'd say that perhaps boxing the integers is slowing the algorithm down, but deduplicateGreybeardCollections has only a <3x performance hit rather than >100x. Is the problem due to allocating large amounts of internal memory and branching heavily (binary trees as opposed to B-trees), or less predictable/vectorizable code execution than array traversal, or virtual comparison dispatch slower than `items.sort(Comparator.naturalOrder())`, or some TreeSet-specific slowdown?

Very good question. I'm away from my work station so I can't do additional profiling, but my overall experience is that TreeSet is hella slow in almost every case.

My hunch is that the extreme indirection makes mince out of data locality during traversal, since TreeSet is backed by a red black tree, so conversion to list is O(n) cache misses, whereas the sorting algorithms are reasonably well behaved from a cache perspective (Java uses a dual pivot quicksort for primitives and TimSort for objects IIRC).

Correction: Performing O(input) failed insertions into a TreeSet might be O(input log output) runtime, which is slower than what I stated above if input > output.

  > Most development is done on high powered hardware that makes slow code very difficult to distinguish from fast code, even though you can often get 10x performance improvements without sacrificing readability or development effort.

true, but compilers are also doing a lot more, and one big driver of high-performance machines (at least for native dev) is compilation taking waaaay too much time...

I looked at this source when you shared it and had planned to show it to a few engineers on my team. Now, sadly, the file is gone. Any chance you can put it back up?

> This proves (anecdotally) that performance isn't unacheivable at all, but rather deprioritized. Nobody wants slow apps but it's just that developer velocity, metrics, ads etc etc are higher priorities, and that comes with a cpu and memory cost that the vendor doesn't care about.

I saw a project that had a very clear N+1 problem in its database queries and yet nobody seemed to care, because they liked doing nested service calls, instead of writing more complicated SQL queries for fetching the data (or even using views, to abstract the complexity away from the app) and the performance was "good enough".

Then end result was that an application page that should have taken less than 1 second to load now took around 7-8 because of hundreds if not thousands of DB calls to populate a few tables. Because it was a mostly internal application, that was deemed good enough. Only when those times hit around 20-30 seconds, I was called in to help.

At that point rewriting dozens of related service calls was no longer viable (in the time frame that a fix was expected in), so I could "fix" the problem with in memory caching because about 70% of the DB calls requested the very same data, just in different nested loop iterations. Of course, this fix was subjectively bad, given the cache invalidation problems that it brought (or at least would bring in the future if the data would ever change during cache lifetime).

What's even more "fun" was the fact that the DB wasn't usable through a remote connection (say, using a VPN during COVID) because of all the calls - imagine waiting for a 1000 DB calls to complete sequentially, with the full network round trip between those. And of course, launching a local database wasn't in the plans, so me undertaking that initiative also needed days of work (versus something more convenient like MySQL/MariaDB/PostgreSQL being used, which would have made it a job for a few hours, no more; as opposed to the "enterprise" database).

In my eyes, it's all about caring about software development: either you do, or you don't. Sometimes you're paid to care, other times everyone assumes that things are "good enough" without paying attention to performance, testing, readability, documentation, discoverability etc. I'm pretty sure that as long as you're allowed to ship mediocre software, exactly that will be done.

> In my eyes, it's all about caring about software development: either you do, or you don't.

Yes, that's the difference. I'm lucky enough to work mostly with colleagues that share a certain level of craftsmanship about the software we ship. We all care about the quality of the product, although we may not always agree on what an ideal solution should look like.

I think people care but there are no alternatives. My 3 month old Windows 11 machine with a Ryzen 5700 and an RTX 3080 regularly starts chugging on stupid things like moving a window, chrome will randomly just peg a cpu with some sort of "report" process, and visual studio locks up when the laptop awakes from sleeping. Id drop it in a second but for what? Mac sucks for game development, has the same issues with too many weird background processes, and linux distros never seem to work on laptops well, even ones designed for it (system 76 for instance). The aren't real alternatives.

By people I think he meant "product managers", not users.

(I know that this is not the ideal solution, but) I had sucesss by installing a trimmed-down 'gamer' version of Windows 11 on a similarly high-specced PC, and it's still blazingly fast nearly a year later.

(If you go this route, you've got to be careful what's been trimmeed, mind, if you want to use it for general purposes - mine doesn't have WSL and seems to have damaged printing support, which is obviously suboptimal depending on your needs.)

How does one find such an edition?

Ameliorated Windows 10 is one "distro" I use and (blindly) trust. It lacks store app support (not a problem), .appx package support (annoying when you want Windows Terminal), creating new user accounts (unacceptable on any machine used by multiple people), and has questionable "security" decisions by default like requiring a separate admin password for elevation, disabling VBS scripts, and an ugly logon screen wallpaper that requires downloading and blindly trusting a PowerShell script from the Internet to change (https://git.ameliorated.info/Joe/amecs). Frankly... if you're using it for situations that need security, you're better off running the (documented) amelioration steps yourself than downloading shady ISOs and wallpaper changers from online.

Torrent sites, ideally combined wiith a legitimate serial #.

One of the reasons users don't care is that they have no expectations of quality. It's painful to watch non-tech people use software written by incompetent developers, like Teams. You'd literally see someone type, then Teams UI would hand. They would pause typing and patiently stare, hands on keyboard. UI would unfreeze 3 seconds later, they'd keep typing. To me this is insane, but people are used to it / have been trained to expect it and consider it normal, apparently. Just like (a somewhat more reasonable) expectation that things need to be restarted in an escalating sequence if something is not working.

If I could go back in time and make one change to the history of computing, I'd add code to every consumer-facing GUI OS that would kill-dash-nine any application blocking UI thread for more than say 200ms. And somehow make it illegal to override for any consumer software, or if I were God make it so any developer trying to override it to let their software be slow gets a brain aneurysm rendering them (even more) incapable of writing software and has to find work digging ditches.

Bad alternative hypothesis. The implemented approach with recursive search is more complicated than just enumerating the files in the Sound recordings directory. A singular focus on requirements would have resulted in the easy solution that's also more efficient.

This is sheer stupidity: a more complicated, less efficient solution to a simple problem.

Why is it more complicated?

They presumably wanted to recursively enumerate all the files under the Sound recordings directory?

(In any case, for the application programmer the difference between recursive enumeration of files vs only what directly in a specific directly is likely only one flag or so.)

This is why I think it's so important that tool makers take the responsibility of performance seriously.

My old company used a proprietary VPN software (Appgate SPD) that used an electron front end - why? Well the front end could be reused on Windows, Linux and MacOS. Fair enough but it was very bloated and lacked features. Compound that to half a dozen electron apps and suddenly my laptop battery dies in half the time.

We can't guarantee or mandate that every project will be written in Rust, multiple times for every platform using their native APIs - so it's on the tool makers to bridge that gap.

Electron shows that we need a robust, efficient, cross-platform, cross-language GUI API.

Would be nice to be able to use Go, Rust, Python, JavaScript or whatever and interact with an GUI interpreter that translates a familiar API to native widgets.

GTK can run on Windows, Linux and Mac. It's open source (LGPL), fast enough and the ecosystem is fine. QT is another one.

The truth is, Electron is easy to use, quick to hire and cheap to build on, that makes it the best choice for many companies, even when the user might (highly-likely) hate it.

Except GTK looks like shit on Windows and Mac, so nobody wants to use it for those platforms. Most developers that create software with these bloat issues would not accept GTK.

Is it because they’ve not been updated to use the latest UI guidelines from the host system? Because they just look old and out of context, not particularly ugly.

GTK looks like shit on Linux too unless you are already using a GTK-based desktop.

Performance should be part of the requirements!

‘Should’ doesn’t pay the rent

It is... fintech normally choses good and fast (in this case fast == fast execution, not necessarily quickly delivered).

That is to say -- not cheap.

When your business is in the middle of financial transactions you have the privilege of picking fast, good, and expensive.

I agree. I'm claiming that most customers and/or decision makers do not. We should recognize that and focus on a solution instead of blaming engineers and local technical decision making. A first step is to avoid slow products, or influence those that buy products for us.

Not in general, no. Performance is one thing that a user might care about, but there are dozens of others. Cost, how fast to build the software, etc.

Telling users what they ought to want is not a very good idea.

No, no, performance should be part of the requirements.

Not in the sense that every project needs awesome performance. But in the sense that requirements should not what performance goals are required.

So you should note down target throughput and latency for different scenarios in your requirements.

That can be as simple as saying that any user interaction should give feedback within one second. (Be that either doing what the user requested within that one second, or giving some indication that the program is working on it.)

Yeah with that I agree, part of the requirements can definitely be what performance is required.

The way that GP comment was written though made it sound like good performance should be part of the requirements for all software, which is what I disagree with.

Why would a user care about how fast it was to build the software?

That depends on the user. As Joel Spolsky famously said, there are different spheres of software. Things like Word or Windows, which are general purpose and go out to millions at once, are one kind.

On the other hand, lots of software is custom-built for a specific end-user. E.g. an organization's internal tooling. There, you can often sacrifice a lot of things for it to be done faster, depending on the use case. In that case, I consider the user to be the "organization" or specific people/teams within that organization.

I guess, but there does seem to be a lot of things companies don't actually care about - performance, security, documentation.. there seems to be a real list of things that regularly show up with people complaining about it and the answer is nobody cares about that stuff.

That said I did work at a major media company and the performance sucked (in the help section) and I said I want to improve it and the PM said "nobody cares" but I think she meant the business didn't care. So there is one anecdote supporting your claim!

I think you are very right about it, but for certain things speed needs to be a requirement. I have tried both Evernote and notion plus onenote.

They are all much more powerful than Apple notes but they are so slow by the time they are ready for my input I have forgotten what I was trying to note down.

We do live in this paradigm of everything being in the cloud and clients are stupid and constantly need to sync, often times in a "stop the world" fashion. Last time I used Twitter web I saw three spinners in the same screen. So that's a big factor as well. It's incredibly poor engineering culture imo, but again those are there because that's what they wanted to build.

Yeah but that’s twitter so it doesn’t matter, you just open it as needed. Evernote works offline and is an actual tool, so there is no reason it’s so slow

I'd say it's also instructive to consider different aspects of performance. For example, many games start up unnecessarily slowly because nobody bothered to optimise it. Users will load the game, then spend a fair while playing it, so the slow startup time doesn't hugely impact their experience. Hence, improving frame rate is a much better investment

True. I don't think users care either. Typically I get lambasted when I bring up performance issues on vendor or user group forums.

Visual Studio and Adobe Creative Suite are examples of the two most egregious offenders. They're both awfully slow and Creative Suite is extremely buggy as well. Visual Studio is nearly 30 years old, and Photoshop is even older. That's a lot of features to have to carry forward over the years.

It would be neat if during setup, they tried to figure out what your goals were as a user and turned stuff off you didn't need. My pet peeve about Visual Studio is whenever I upgrade to a new version, my default setting overrides are not carried over.

Isn't this thread full of evidence that people do care? Most of us, talking as end-users of software, seem to care. It's annoying when notepad.exe is slow.

Most people don't know what notepad.exe means. We're an extremely narrow set of people.

Also, it's about caring enough to actually stop using it. Have you ever changed software to a competitor with better performance? I don't know if I have..

I guess it’s more that what is developed is lead by product people and those don’t know how to plan for performance issues and instead inadvertently push for more features in a shorter time.

As well: Companies pay more for dev time than they ever have in the past. There are tonnes of stories of companies buying devs the best hardware so that they can program more effectively, and of course devs with the best hardware don’t even see a lot of performance issues.

Then on top of that, companies are making choices like “we could optimize X and Y, but that would take too much dev time. Let’s just (compromise somewhere/bump up the requirements)”

People do care, just everything is silod and monopolised/duopolies with no where else to go.

We’re stuck and getting stucker.

This brings to mind Leonard's law, which is that software will be as bad as it's users can tolerate.

Reply on news.ycombinator.com