For certain data science usecases, evaluation is possible by using a CSV output bu a user, and comparing that to an expected CSV.
(I worked on the product).
I should have asked this before, but "data science" is a pretty broad term - who are you hoping to target with this? I'm guessing for pretty junior positions but want to clarify.
Oh, and one other question, do you /can you enforce the 60 min time? [edit: never mind, I answered this experimentally, you do cut it off at 60 min]
What skills would you consider important for senior vs junior Data Scientist?
For what it's worth, I think junior and senior DS roles should have fairly different evaluations & interviews.
The question then is how do you algorithmically (or even just consistently) distinguish a great data scientist from one who can accurately model answers to a question that was badly thought out?
Plus as pointed out before, the length of a take home could reduce applications from the most qualified candidates.
Wonder if this should be even shorter and more quiz/fun like so it intrigues rather than annoying more senior applicants, and still wondering the best way to identify the data scientists who ask better questions.