Hacker News Reader

> Sometimes I think about how much money people would pay to make their computer run 20x faster.

I think people who can reliably do this are paid $250-750k/year at top paying companies.

I think there are almost 0 positions in industry for this sort of thing.

There are plenty, actually. Source: did this professionally for a bit

I'm interested to hear more. What kind of companies pay for this? Hardware companies like Intel and Nvidia? What are the roles called?

The FAANGS certainly have people doing this kind of work. For example, Andrei Alexandrescu has a bunch of keynote talks at CPPCon where he talks about 1-4% speed optimisations he makes to the std library at Facebook.

https://youtu.be/FJJTYQYB1JQ

I also like the talk by Nicholas Ormrod “The strange details of std::string at Facebook":

https://youtu.be/kPR8h4-qZdk

Algotrading companies.

There are several branches, but all of them typically run at a scale where performance costs them money. People with large server deployments (any "hyperscale" company) hire these people to save millions of dollars a year. People who do client performance attract and retain users. Those who do trading bank on being the fastest. And many other areas; these are just the big ones.

Interesting. Does it need a PHD or a specific Master? Because I feel no one needs that kind of performance at home so the only place to train is academy or industry while industry usually does not train a junior on that topic.

No. Usually you learn this stuff by doing it at a smaller scale elsewhere.

I just became an AVX2 programmer for fun. You can too, if you go to the highload.fun group chat then I or someone else will suggest blazing fast ideas for every problem that happen to be about half as fast as our own approaches.

This type of optimization isn't something you typically study as an academic. Everyone I know that does this kind of work (these jobs definitely exist) was self-taught by experimenting in their own time because they had a passion for performance optimization. It is a rewarding specialization in that there is a quasi-objective measure of incremental progress.

Much of this optimization work requires specific knowledge of the details of the operating environment (e.g. Linux) and silicon microarchitecture (e.g. AVX-512), which can be poorly documented. You have to be comfortable doing experiments and digging into system arcana to surface properties of the system that you can't trivially google. While algorithm selection is important, doing that well is table stakes and part of the role is knowing when and why an "optimal" algorithm is worse than e.g. selectively applied brute-force.

Computational efficiency and throughput is worth a lot of money at scale but most of it is designed and implemented at the scale of a single machine. Every software engineer owns the tools required to become proficient at this. It is genuinely a rare skill even among systems programmers and is a good way to separate yourself from the crowd.

Various types of hyper-scale or high-performance computing where incremental improvements in throughput, efficiency, latency, and resource utilization saves millions of dollars. Hardware companies are not where you go for this, though they do offer some limited micro-optimization support for the companies that do care about this kind of thing. You want to focus on companies that spend enormous amounts of money on compute infrastructure. The appetite to invest in compute efficiency waxes and wanes with the economy. If companies can get away with throwing money at a problem they will, but that can quickly become untenable.

A closely related area is slightly bending the scaling linearity curve on e.g. big multi-core servers or scale-out systems, so that it is possible to efficiently throw hardware at problems. However, this operates from a pretty different set of theoretical principles than classic performance optimization.

Two domains that have an almost unlimited appetite for improved performance and efficiency right now due to current bottlenecks are AI and sensor processing.

I'm certainly intrigued and I'll shoot you an email.

It should also be noted that you can also get a significant speed up by writing situation specific code, instead of handling the general case, if your problem allows it.

This kind of guarantees that custom code, written competently, will always be able to outpace standard library code. The standard library is one of the few things that probably always needs to handle the general case.

> Sometimes I think about how much money people would pay to make their computer run 20x faster.

Bottlenecks that can make a computer 20x faster are very rare. First, pay $1m for a specialist to find the bottlenecks. However, it is even rarer that it is a bottleneck of the entire service. Rather, it is more likely to be an accidental drop in performance. Thus, basically performance monitoring is the best use of money.

> Its weird to me that we'll pay thousands of dollars for faster CPUs then write new software in python, electron or a docker container running in a VM.

There are more labor costs and lost commercial opportunities due to delays in development.

>Its weird to me that we'll pay thousands of dollars for faster CPUs then write new software in python, electron or a docker container running in a VM.

Python has a REPL, Duck typing and hooks for many libraries including gui libraries.

Electron is basically a browser you can program via JavaScript with everything you'll ever need to make a gui application.

They both make a lot of sense when you consider the alternatives. However, I do wonder why there isn't yet an option to AOT compile python and electron projects into fast executables.

the semantics make aot compiling based speedup basically impossible. Python is just way too dynamic to do anything with. If you need speed and nice language features, you're much better off using a high level language that has better semantics for speed. Julia is my preferred option, but Lua and a few others fit this niche well.

    > then write new software in ... electron

Yeah, I agree. The OP's point is a bit misguided. The reason why people use Electron is it is cheaper to develop an app in Electron. By "cheaper", I mean the cost of developers plus the hardware needed to run it well. How much does an enterprise desktop PC with a 3+GHz CPU cost? Surely less than 2K USD. How much does a developer cost? It is hard to pay less than 75K USD per year.

And about Python: Almost no one is doing CPU intensive work in pure Python. They are using C-language extensions like Pandas, NumPy, etc.

> How much does an enterprise desktop PC with a 3+GHz CPU cost? Surely less than 2K USD. How much does a developer cost?

But the software runs on the computers of all of their users. How much money do their users' computers cost? If you have just 10k users, probably $10M or so? But software companies don't pay for my computer's resources. The money I need to spend on computer hardware isn't priced in to your calculus at all. They don't care.

I find it pretty frustrating and sad.