Yeah, as another commenter said, an interview is not the same as a critical sprint. I don't mean that I find coding stressful, I mean that I find the social aspect of being interviewed stressful, and it's hard for me to get in the problem-solving state of mind when in that situation.
I don't think you can argue that you're not invalidating someone's experience (within the context of their suitability for the position) if you don't look at any of the code they've written outside an interview process, because it's self-evident that there are candidates that are talented programmers who cannot demonstrate those skills effectively during an interview.
You can argue about how insignificant of a minority that is, though. I'm a senior developer who has been offered most of the jobs he's applied for because I _have_ been able to prove basic code literacy during an interview (and have typically surprised employers by how effective I can be, when not in an interview situation). So if the _screening_ bar is low and if you're taking steps to ensure that the handicap of social anxiety is diminished, it may be a negligible percentage of candidates.