If you don't think about performance ahead of time (preferably at the algorithm level) then you will build something correct but slow and it'll be very hard to make it fast without redesigning major parts of it. It's possible to make design choices that paint you into a performance corner very easily, especially with something designed to work on a lot of data. O(n^2) might look like O(log n) for small data and in testing.
"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."
It means make it fast in the code that matters, do not try to make it fast in every single line.
To say the quote is often misunderstood is the biggest understatement in modern computer science.