I teach linear algebra in a pure math department. We like to think that freeing people to be able to daydream about linear algebra is our goal, that there's more to it than what function calls to make in MatLab.
Still, you could throw out our course and spend a semester teaching people how to really think about singular value decomposition. It's everywhere, critical for example in machine learning, yet pure math types think it's "applied".
Riemannian geometry begins with an inner product at each point (though what's amazing about O'Neill's text is that he slips by you a categorical definition first at every step). As another route up the mountain, one could now understand parallel transport as a local version of the singular value decomposition, and work out the whole theory that way. This requires a fluid, intuitive grasp of singular values, rather than the textbook definition. We should teach that understanding.