This is true for mathematics, but not in the same way for computer languages, because we haven't unlimited computing power and memory space.
Yes, with a more abstract language you can solve a problem easier on the implementation, source code side.
But with every abstraction you're loosing flexibility, because to get an abstraction you're putting the lower parts in a concrete combination. Thus every abstraction has it's costs.
Yes, you could use two computing languages, one for the performace/memory critical part, and one for the other parts of the programm. But in practise it's not that easy to combine the two.
There's a reason why people are still using C/C++.