Without the math, how do you know that your solutions are any good? How do you know that the algorithm you sketched out on the whiteboard will work on the 1,000,000 records that we have in production in addition to the 5 record toy example that you worked out?
I mean, sure, I agree that brainteasers are in appropriate. Knowing why manhole covers are round or how M&Ms get their candy shell tells me nothing about your programming ability. But basic math and algorithms? They're a lot more important than you think. Knowing Big-O notation, various kinds of basic algorithms and a good assortment of data structures lets me know that you can evaluate a solution and find potential flaws and bugs.
How is putting someone on the spot to build something and build it perfect a good test of their capabilities and skills?
There's a difference between perfect and working. I've never had an interviewer ask me for perfect code. Not at Amazon. Not at Google. Not at Facebook. I have, however, been asked for working code. The code doesn't need to be 100% bug free, but it should be syntactically correct and should solve the problem in a reasonable fashion (e.g. without using inordinate time or space).
Frankly, I like technical interviews a lot more than the alternative, which increasingly boils down to, "Make me a webapp and we'll evaluate your code." I'm sorry, but if I wanted to make a webapp for you on my own time, I also want to get paid for it. Even now, when I'm single and living alone, I have a hard time motivating myself to do homework problems. I can't imagine having to do something like that when I'm married and have children.