This was a lessons-learned post, but I would LOVE to read a "How we made those lead bullets" post.
Because his post goes like "guy says lets use lead bullets." -> "We hunker down and use lead bullets" -> magic -> "Boom. $1.6 billion company"
Because his post goes like "guy says lets use lead bullets." -> "We hunker down and use lead bullets" -> magic -> "Boom. $1.6 billion company"
Optimize something, reduce the running time by a few ms, optimize something else, reduce by some more ms, repeat 8 hours a day, with several people in paralel for a few months.
The concept of a "lead bullet" implies that the history won't be interesting to read, just some repetitive boring hard work.
I may be the only one, but knowing what the 'something' was is incredibly valuable bc we all, as people, pick 'something' and not everything in any sort of bullet is created equal.
- Processes which run many times.
- Processes which run long time (and block other processing).
- If at all possible: both.
The more subtle approach is to identify things you don't need to do (needless initializations, "settle" delays, doing (or allowing to be done) the _wrong_ thing requiring both unwinding that operation _and_ doing the right one, and anything you don't have to wait for (if it's an ancillary process, spin it off and handle it out-of-sequence). Redesigning a process to avoid sorting data. Eliminating global locks. Spinning off tasks to parallelized processing (especially if you have multiple cores/systems available. Pipelining data to avoid writing/reading from disk multiple times. Handling data in memory rather than on disk. Realizing you _can_ handle data in-memory, and that disk-seeks will kill you, so sorting or bucketing output. Swapping spinning rust for SSD. Specific tools such as COW, snapshots, or the like can be very useful in such cases.
Algorithms (or methods invoking algorithms) can be another big win.
I've seen processes fall in time requirements by 90% following such an optimization process.
1. Use a sampling profiler on the important execution scenarios.
2. Identify the operation that is taking the most time.
3. Ensure it is using appropriate algorithms and data structures.
4. Profile again; if it is still taking the most time, hand-optimize it.
5. Goto 1Even if it was boring or rote work, I'd love to know what it was.