1-5ms response time is clearly hard for most real world endpoints.
When it comes to improving performance by a few ms, or implementing a new feature, business people will always choose a new feature, unless the current performance is unbearably slow (we're talking regular 1.5s+ wait times for BE response).
And it's not even a modern problem, legacy software written 20 years ago has the same latency than most modern backends from my experience.