Reasons to always try to be nearly optimal on performance from the start, that I rarely saw stated:
1) What you can specify depends on what can be done, and to know what can be done you need to have tried your best. In particular, it allows to see early whether or not performance expectations are reallistic.
2) It's more difficult or impossible to upgrade performance later if it requires to break an API. It causes a development process in O(n^2) in number of layers, instead of O(n).
3) Better lower level performance makes higher level code and architecture simpler, as you can just brute force in more places for a same overall performance.
In fact, devoting time to code optimization tends to be explicitly discouraged in my experience. The prevailing wisdom seems to be, "write shitty code... just make sure it runs on as many AWS machine instances as possible."
Those that prioritize performance upfront can find all that work thrown out if the design needs to change for some reason.
But I think that performance by design upfront should be done where possible. This is where experience helps a ton.