I'm sold, although I'd still start by sitting them down with a copy of Feynman's QED. When doing math about a thing, it helps tremendously if you already have some sort of intuition about what the math is describing.

Also, the historical stuff is fun and interesting. Physics students should probably be taught this stuff regardless, but maybe not as part of a class on theory.