At my company, for a web app we run in production, we strive to get every response out of our infrastructure in under a millisecond, everything above that except for a few select endpoints is considered as a bug. By using sensible technology choices, it's not even that hard to do. A RDMS like postgres can answer to queries in microseconds.
Nice bonus, we can provide real time computation features our competitors could only dream of, just by not using dog-slow technology.
Our customers are not techies and you know what? When they use our product, the first comment is usually "wow, it's so fast".
Just to expand on your point:
Sure you can let a chunk of code take a few seconds longer than it could with little overall effect.
But stack a hundred of these up and all those seconds compound and suddenly become immensely important.
A suit of armour with a million chinks isn't very effective armour.
(As an aside, depending on your use case, it can be worth optimizing the big bottlenecks, high use, computationally expensive, etc. components and not worry too much about the rest)
Counterpoint: https://en.wikipedia.org/wiki/Chain_mail
I am agreeing with you, I don’t understand your response.