For those who don't know much about the AP CS course, I found it really awkward when I took it in high school. The exam involved a lot of writing Java...in pencil. Not an experience I ever want to repeat.
Let me be clear: I think it's a great way to test students as long as the graders/instructors aren't overly anal about syntax or even API correctness. I was lucky to have teachers who were okay with unbalanced parentheses, made-up API functions, and in upper-division classes, pseudo code was even acceptable (data structures). (However most CS classes will be biased towards imperative programming over functional programming).
To use a cooking analogy, if you have to think about how you are cutting the fish, you won't be able to think about how you are going to prepare or cook the fish. Writing code on paper forced me to think about design first; that is, it enabled me to think about what I was going to do with the fish.