I am in the camp to think that this notebook judges data scientists in a way that soon will be obsolete.

If I'm given this clean dataset with all of the features properly set in columns and data types labeled, I could spin up Azure or Google Cloud's ML capabilities and have them run gridsearch and optimize my model.

To test data scientists, it seems like it's generally falling more into the buckets of people that can pull analytics and query databases to create the datasets and features OR people that can build the infrastructure to serve models, engineer pipelines, etc...

FYI though we're working on this now at https://www.interviewquery.com to try to start creating suitable tests to assess data scientists without having them do 10+ take homes every month.