I've seen this same thing frequently when evaluating security engineers. They often can talk the talk, but when given a simple task to perform more often than not will fail.

Coding exercises might not be a great measure, but as far as I have experienced, they are better than other items out there.