Hacker News Reader

User focused metrics:

1. It's a user facing bug if I have to purchase a bigger laptop because someone decided to get the smallest element of a list by typing list.sort()[0].

2. It's a user facing bug if the use of my app heats up their phone, drains the battery, and leaves someone stranded w/o a way to get safely get home on a friday night.

3. It's a business facing bug if my AWS bill is in the hundreds of thousands per month when I could be paying a mere 10s of thousands. (This is more common than you'd think)

4. It's a user facing bug if a sudden increase in traffic (due to my product being featured somewhere) takes down my services and results in lost sales and brand damage.

I understand your point, it's not necessarily a problem if a backend feed is taking 45s instead of 30s, or if the the text input subroutine for an interactive app allocates.

As far as I can tell as a software developer and daily computer user, people aren't erring on the side of making their software too fast.

If you've a widely used program and you've never run a profiler on it, that's the problem, not using list.sort()[0].

100% agree with all of these, and you did what the author didn't and added criteria that affect people. If these happen, then make the code faster!

Abrash is a well known games/graphics programmer, and part of his point is that if your game is already running at 60fps, there's no point in replacing list.sort()[0] with something else. No milliseconds saved will help anyone in that case. (Though yes, as many pointed out, power consumption may be an important metric).

Personally, I'd say if your backend feed is taking 45s and could easily take 30s, then you should fix it. :)

> people aren't erring on the side of making their software too fast

:) 100% agree... too fast isn't a problem we have. But, over-engineering is an exceedingly widespread problem, and that's where I'm coming from. If there's a user-focused metric that shows a problem, yes fix it. If there isn't a user-focused metric, then think twice. The article didn't advocate a user-focused metric as far as I could tell.

> if your game is already running at 60fps, there's no point in replacing list.sort()[0]

If the game runs on a device with a battery then reducing that load is a reason. You already implied that though.

I suspect hard to quantify improvements in quality is not what you are trying to rail against. If the software is going to be used widely or for long duration hard to quantify improvements can easily save millions in the long run.

There are over-engineering efforts where people try to shoehorn in bad design strategies, over-abstract an already well abstracted or refactor well working code to be more "testable" when it is already tested. All of those should be opposed, but how do you draw the line be over-engineering and hard to quantify quality increases?

> how do you draw the line be over-engineering and hard to quantify quality increases?

That is a really good question. That is the question, IMO. I don't know the answer, and if I did I'm pretty sure I could make a lot of money doing it. It's a balance, maybe the right thing to do is just make sure you have forces in both directions, and don't let one force dominate for too long. Make sure in your working groups there are always people asking if there's a user facing metric, a real tangible reason or justification for the code you're planning to write or optimize. And also make sure there are people who are good at planning for the future and are good at coming up with user-facing justifications for upcoming plans.

In The Practice of Programming Kernighan and Pike say that one should estimate how much time can be saved over the expected runs of the program and allocate no more than that much time to optimization.

If you're doing something once or once per year that takes 45 seconds rather than 30 then don't bother. If you're doing it once every hour, that's six minutes a day, 365 days a year. If it's on a minutely cron on a hundred systems, that's 144,000 seconds a day. If a few hours of time can save you 144,000 seconds of runtime every day, that's a clear winner assuming other higher priority things are done. If it's run minutely by thousands of customers on hundreds of thousands of servers...

I think that was true back when he wrote the words, but less so now. Even if you're not noticing a delay, code that completes quicker is generally code that uses less power and you can still notice the effect on your phones battery life - or on the size of your monthly bill from your data center.

Great point. Efficiency comes in different shapes. To respond, I'd say power & cost are human oriented metrics, and I take his point to be use a human oriented metric, for me it's as true now as it ever was.

Ah darn, I wrote along the same line an hour ago or so but got delayed in hitting the submit button until now. Yes, pretty much this.

> I love the way Michael Abrash talks about optimizing. He emphasizes and re-emphasizes that optimization is for people not computers, and that if you aren't experiencing a delay, you're done.

In a time of battery powered computers CPU cycles spent equate to energy consumed equates to reduction in battery life.

Even if your program does not impact the user's experience of runtime behaviour (latencies, delays) it's still impacting in the user's experience on his devices recharge cycle.

Also spinning fans generate noise, which is just as a nuisance as a laggy program.

"Unseen" is relative

A performance issue that is hard to detect at 10 rps becomes an issue when you're trying to push 1000 rps. A performance issue in a mobile app (or a desktop app on a laptop) might be undetectable from a responsiveness or frame rate point of view, but burn through your battery.

If you count memory footprint as a performance issue, thowr don't become apparent until you try to run more things concurrently, and find yourself thrashing when memory should be plentiful.

Yes, true, I agree with you. And performance becomes seen when you scale, or measure power consumption and battery life, or check your memory footprint.

I think what you're saying is congruent with what Michael Abrash is saying, that you address it if it matters. And sometimes it matters.

But almost all the code in the entire world can run faster if someone spends more time optimizing. Anything in any scripting language could be compiled and further optimized. Anything in a compiled language can be further improved, use less memory, access cache faster, etc. etc.

Should it be optimized just because it can? Should it be optimized if it doesn't matter under any human oriented metric, including power consumption? (The author didn't mention power consumption or memory footprint.)

Sometimes it doesn't matter, and looking for it when it doesn't matter is what we call premature optimization. Premature optimization is also a waste of resources, including time and money, and a huge factor in why almost all software arrives late and over budget.

Performance issues cost you money: either you're losing users due to bounce rate (there are studies out there that having pages load for more than 300-500ms or so will significantly increase your bounce rate), or you're paying huge prices for your cloud hosting or you're burning battery of your users and thus losing them as well.

The issue is that usually smart design avoids significant performance issues and design which does not consider preformance at all ("we'll optimize it later") usually has to be scrapped and rewritten due to how many systemic issues accumulate over time. When you hit a scaling wall, it's usually orders of magnitude more expensive to rewrite the application than spending a bit of time getting basics right before you started.

Totally agree with you. Those sounds like measurable human facing impacts that can and should be optimized.

FWIW, I've also personally watched millions of dollars get wasted on over-engineering, that's part of the context behind my comment. I'm pretty sure I've seen more money lost on problems that don't exist than I've seen lost on avoiding problems that do exist. (Of course, that's impossible to prove, and I could be wrong, but that's my personal feeling.)

And in no way do I advocate not considering performance at all. I've done a lot of high performance code, and I think it matters a lot. But I like that Abrash attaches it to something meaningful, not make it faster for the sake of making it faster. The author didn't demonstrate any meaningful metrics like you did, he only said you should make it faster because you can.

Oh yes of course... overnegineering is as dangerious as bad engineering. Finding the proper balance IMO is one of the most important skills of senior engineers. Sadly I've also seen way too many people misunderstand the "Don't optimize prematurely" tip and completely ignore any performance considerations when designing software in early stages. Fixing that hurts.

Meh, there is a balance.

A performance issue that isn't an issue when doing a single thing can become huge when you're trying to automate doing it many thousands of times. On the other hand there are few ways which are better to obfuscate a code base than to try to optimize performance.

The rule of thumb is don't do anything stupid, but don't particularly worry about efficiency until there is a real need. And then, as Knuth advised, you start with a profiler because the performance problem probably isn't where you think.

>"In other words, high-performance code should ideally run so fast that any further improvement in the code would be pointless.

And then define pointless, to make a quantified decision on optimization you should use a performance model. Common model is the roofline model. https://en.wikipedia.org/wiki/Roofline_model

> Is it a bug if it's unseen?

It's not a bug, it's a feature. Gives you time to get coffee while the page loads.