Most 'performance tuning' out there is eliminating quadratic functions and putting stuff into simple dictionary caches. This should be easily teachable to any competent developer.
The problem isn't 'hard work', it's nobody cares about this.
Getting measurements that help you identify where the bottlenecks really are can take time. Often these have to be retrofitted into existing complex system, when the project is already underway.
Sometimes the only way forward is replacing the current way of doing things with a more efficient way of doing things. This requires understanding the current way well (it can be the result of many teams working together for months) as well as coming up with a new design. Both of these things take time.
Finally, there's bug hunting, where you get into rabbit holes. "There's a slowdown when we're on this particular step. You investigate for days, until you realize that the slowdown only happens when the user mouse is near a section of the screen. It turns out that there's a non-visible UI element that should be inactive but it isn't because it is a custom element instead of one provided by the UI library, and it is attempting to render a special effect but finds no graphical context, so it request for a new context once per frame." And one week of dev time has passed.
In most games this doesn't matter, but occasionally you have a frame time target and you need to take extra steps like that. Those extra steps can be hard work when they come up!
Even when you use SIMD intrinsics?
For example: https://en.wikipedia.org/wiki/Intel_SHA_extensions
A repo I just found illustrating those instructions' usage: https://github.com/noloader/SHA-Intrinsics
Source is the measurements here: https://uops.info/table.html
That'd make SHA256RNDS2 even faster because of the throughput numbers.
I'm curious, what's the faster algorithm for hashing you use? I don't want to waste time on intrinsics and inflict complexity on my colleagues if there are alternatives I can use!
For example, I need to reorder a matrix and change my access pattern to it to extract more performance and reduce cache trashing by exploiting cache prefetch dynamics of a CPU.
I need to test this with every CPU generation and test whether it works as expected performance-wise.
To achieve this, I need to rewrite half of the said software.
The reason I’m not doing this is it’s still way faster than other software I’ve seen in that category, and extreme performance tuning can hurt flexibility of the code down the road.
That does happen, but some of the more severe performance issues I've seen are memory related. Every byte allocated has to be freed. If you lower the memory profile of your function then you reduce the overhead of running the code significantly.
I recall a web service that ground to a halt and would sit at 100% CPU, after I put it in a profiler 97% of the cycles were in the garbage collector. Dig a little deeper and I found that the caching library used was cloning the data before sending it back. Ahh, junior devs.
I rewrote a library with the same interface that did not clone the data, this was safe as the original data wasn't being changed, 15x speedup from 10 req/s to 150 req/s.
Then I dug deeper and changed which data was getting cached. Cache post-render and post-compression instead of the pre-render database calls. 150x speedup from the original or 10x from the previous point. The system went from being able to serve 10 requests/s to 1500 req/s. This change also was safer since nobody is going to ever try to alter compressed data, no such guarantees from the cached database calls.
A sibling poster also comments that dictionaries are too slow. They might be in a game setting, but definitely not in a Java backend service.
Most (non FAANG) sites are a mess because they're made by inexperienced developers turning everything they touch into a tangled slow mess.
The process goes like this:
- Devs ditch the legacy (often monolithic) code
- They start creating microservices fron scratch
- They don't see the need to optimize or even make readable code. It's a clean slate! the project is small, the new code is just whatever they write.
- They do that with the idea that the legacy code is "bad" and the new code is "good"
- Requirements either scale or downright change
- To the code made before, they add a "tiny" special case and patch here and there. After all, the code is "good"
Repeat
Now you have a new messy project. The dev will either quit or work in a new project. The next dev will do the same. And the new team will probably push for another replacement from scratch, so they can feel that they re creating "good" code again
Now try to take some React developer that has been working for years in completely different abstraction level, and help him design more performant code base.
You really think all you will talk about are cache misses?
i also had a chat with a C++ dev that created a "Database"-class abstraction for our MongoDB instance and implemented methods to access each field of every documents separately - didn't care about side-effects, didn't know about ACID transactions and didn't care.
finding a good abstraction that allows for performance tuning later on is hard, no matter the language, and react devs should make use of good abstractions as well.