I'd be fine doing a coding interview with just Notepad.exe or vi or whatever. It takes me ~2 seconds to type a line of code, but more like 10 or 15 seconds to handwrite it [0], and my brain's just not used to that kind of latency. It trips me up.
Plus, on a whiteboard, the mechanics of "oops I need to insert a line... I guess I'll just write it down here and draw a big arrow... ok now I need to rename this variable, but now the name is really long and I can't fit it... wait, what was I doing?".
It just mucks up my process. I don't precisely conceive an entire subroutine before I put fingers to keyboard--my code evolves as I'm writing it. I edit, revise, rethink, and refactor constantly, long before anything's even compiled. Keys and screen facilitate that process a thousand times better than pencil and paper. Handwriting just isn't the best medium for code [1].
[0] And thank god I have a CS education, because if nothing else I at least learned how to write legible braces, brackets, ampersands, and at-signs by hand...
[1] OP is obviously a wonderful idea but that's because it's teaching aspects of CS that aren't code.
I think your use of the term "latency" is a very good way to describe some of the problems around whiteboard coding.
I'd even somewhat prefer a chalkboard.
There are also "programmers" who's entire method of work seems to be to copy+paste blocks of code from Stackoverflow, only changing the minimum amount to make something load and then move onto the next item in their hit list.
I always liked white boarding problems.