This would be really great. I constantly see excuses for why no-one should bother doing user studies of programming languages because its really hard. So let's not do it at all and instead focus on approach #2 that you lay out ("10 weeks thinking about the problem in your head").

That being said, this is really hard. Observing a (necessarily small) sample of heterogeneous programmers dealing with (necessarily small) tasks isn't easy, either.

I wonder whether users would consent to having their text editors and compilers instrumented to produce a user study of every single character typed and every single build (failed or successful, especially failed). I would imagine that a language designer could learn a huge amount from seeing all the things their compiler is rejecting from real users...