I think that people building end-user software are less culpable in some regard. My comment is aimed more at people producing gems and libraries and frameworks that then propagate through the Rails community. When they are slow, they cause a compounding effect that ripples down to the developer building end-user applications, who just assume that 400ms for a page is "normal", because that's how it's always been.

Ideally, in the Rails world, the people building client apps would only have to worry about the algorithmic complexity of their code, and the libraries should be operating efficiently under the covers. To that end, you're completely right in that you shouldn't have to worry about performance - the people writing the software you build on should have already have done that. But when they don't, that costs you time and money, and I think that's a shame.