It's true to a degree that bankers don't aim for zero unemployment - i.e. implicitly they "let millions and millions of people be unemployed" - because to do so would create inflationary effects. Such an economy would be overheating. In an economy the size of the US, you would expect there to be a few million people unemployed at any given moment, simply because of turnover.
But this is pretty basic stuff. See e.g.:
http://www.investinganswers.com/term/unemployment-rate-809
http://en.wikipedia.org/wiki/NAIRU
It is however true that the Fed has a rather ridiculously overloaded mandate. It has basically one lever - the interest rate - with which it is supposed to target multiple goals - "maximum employment, stable prices, and moderate long-term interest rates". If you think about this for a moment, you'll realize it's not possible.