Hello folks, would love your feedback on our new product to assess data scientists.
Can you elaborate?
For certain data science usecases, evaluation is possible by using a CSV output bu a user, and comparing that to an expected CSV.
(I worked on the product).
I should have asked this before, but "data science" is a pretty broad term - who are you hoping to target with this? I'm guessing for pretty junior positions but want to clarify.
Oh, and one other question, do you /can you enforce the 60 min time? [edit: never mind, I answered this experimentally, you do cut it off at 60 min]
What skills would you consider important for senior vs junior Data Scientist?
For what it's worth, I think junior and senior DS roles should have fairly different evaluations & interviews.
The question then is how do you algorithmically (or even just consistently) distinguish a great data scientist from one who can accurately model answers to a question that was badly thought out?
Plus as pointed out before, the length of a take home could reduce applications from the most qualified candidates.
Wonder if this should be even shorter and more quiz/fun like so it intrigues rather than annoying more senior applicants, and still wondering the best way to identify the data scientists who ask better questions.
This meant we couldn't automatically score/rank a candidate's solution. We do provide them with an evaluation metric in the problem description (Mean Absolute Error). Here is scoring rubric we provide to the interviewers when they review the submission - https://d.pr/i/hNYY0u
Would love to hear more opinions on our scoring rubric
[edit]. Initial thoughts:
- "data wrangling" scoring difficult given this task - more weight to "rationale", that's more important the "performance", here.
- not enough focus on communication capabilities
- really need something on validation
- "proficiency" measure you use is pretty much impossible to accurately evaluate from your example question
- way too much weight to modeling section overall
Of course more abstractly you want to validate the entire process - but I was referring to those two, as it is hard to see how to address the latter in a format like this.