I estimate it would/should take any programmer/problem solver no more than 15, maybe 20, minutes to come up with the solution if they have experience in the language we request they solve it in. The exercise doesn't require specific domain knowledge but should quickly assess whether a programmer, particularly with experience in a different language, is able to learn and apply the syntax and semantics of another language. If they are completely unfamiliar with the language then maybe an hour. They can use whatever references they want. We will check for complete copy/pasted code and alter the requirements and/or test data over time.
The other great benefit is we get to see what/if/how they ask questions to clarify the problem and solution.
EDIT: This exercise is geared towards fresh graduates or developers that don't have experience in the language we use. Can it be gamed? Yes. Might some people not bother? Yes. Are we okay with that? Yes.