Curiously, it's inherently impossible to compare what I think matters most for languages: solving new problems. To compare two languages you have to have people solve the same problem, and that means it has to be a predefined one.
The other problem with studying this question formally is that it is so psychologically rooted that people almost invariably "find" that their favorite language turns out the best.
IQ measurements seem to have a similar problem: doing test a, influences your score on test b, because you get more used to the problems presented.. but doing test a and b just marginally increases your general intelligence, which is what you are trying to measure, while it might have a big influence on someone's score in these tests.
With IQ tests, you can expect that doing test c till z will eventually saturate someone.. but that might only work because you generally do not get the answers. In coding, you do get the answer back (whether your code compiles/works or not), so eventually both programmers will just be limited by the speed with which they type.
There are many ways. Off the top of my head:
- to focus on speed of development you could use a practice or warm-up
> provide each with a practice in another language
> provide a sample solution in pseudocode in both functional and imperative style
- To focus on speed of problem solving you could use topcoder data + research of participants programming language backgrounds.
The important thing is that if the effect is large it's easy to spot. According to some, the difference is an order of magnitude. If that's true, it will be trivial to spot.