If an application gets 4 thousand req/s for an hour, and an additional 10% requests in the rest of the day, it is handling nearly 15 million reqs/day, which is completely different and of course requires scaling in most cases.

That said, even then, there are a lot of business cases where you are not constrained by the time required to sort or traverse a custom data structure, because you spend more time waiting for an answer from a database (in which case you may want to tune the db or add a cache),or the time needed to talk to a server or another user, or a third party library, or a payment processing endpoint.

There are also use cases (think offline mobile apps) where the number of concurrent requests is basically 1, because each offline app serves a single user, so as long as you can process stuff before a user notices the app is sluggish (hundreds of milliseconds at least) you're good.

What do you do with those 4 thousand req/s? That's what makes the difference between "processing everything independently is fast enough for our purposes", "we need to optimize database or network latency", or "we need to optimize our data structures".