Maybe I’ve misunderstood but from HN / other online discussions it seems that employers in Data Science have been increasing education requirements to MSc, PhD, and so forth for many jobs that more appropriately require a strong statistical understanding combined with rigorous data processing skills and some imagination. Is this a real “skill gap” or an attempt to create artificial scarcity of hot jobs?
I am suspicious that bootcamps and online courses will set individuals on to long term data science careers. I think there will be a large demand for individuals who understand basic data science principles such as the difference between supervised and unsupervised learning problems. I think these people will be given subscriptions to services that automate a lot of the actual data science work flow. They will use programs like Data Robot. However, I also think these individuals will be paid closer to what a data analyst is than what a software engineer gets.
I think that certain companies will always be looking for people who can actually do traditional data science work from scratch because the automated tools will not apply to their specific use cases. I think these individuals will probably generally have a master's or PhD.
I think right now the field is too new for managers to be able to tell the difference between what kind of Data Scientists they need. My experience has been a lot of companies need the former, but they think they need (or they want) the latter. I suspect in the coming decade these two roles will find different titles.
Being as how "Data Science" is such a nebulous term, what is wrong with such a posting? I'd say that a data analyst with some advanced math and programming ability is a pretty rare skillset. Is it any surprise such a person is in high demand?
5 years ago DS was hot and new. As a middle manager you would call up a consulting meat farm and ask for data science consultants: presto, you have a data science team. These people would, likely, be random BSc's with a SAS base certificate.
In 2018 data science, and data driven decision making, is still hot, but hiring managers know they need people with a formal education. So, education reqs go up and scarcity increases.
It's the exact opposite: as better tools are developed, there is less "formal education" required. In the same way that Web Devs are in high demand, and nobody needs a Computer Science degree to do that work.
The reality is, "data driven decision making" is something businesses large and small are discovering they need. And often times that sort of analysis requires a regression, or maybe something slightly more sophisticated. What these businesses do not need is some cutting edge algorithm or complex AI pipeline.
There is a ton of gate-keeping in this space, as the the PhDs learn that domain experience and business acumen are more important than a deep knowledge of the algorithms. A significant amount of the work is cleaning and processing the data. This is the way of the world. At the end of the day, most business problems boil down to inference and prediction, and most times you don't need to be incredibly precise; the easy 80% will do.
When trying to put it into practice, if we go the slightest distance off piste we are lost without the math. So it's a fragile form of knowledge.
If we take someone who's already strong at math from some other training and teach them more stats/ML concepts and intuition, then great. But the value add of the mooc learning is quite low in that scenario.
The best alternatives (barring a formal course) imo, are text books that have exercises like Bayesian Data Analysis by Gelman.
Our other tier of hireable candidates have been individuals with 4+ years of industry experience, usually with CS PhDs with a machine learning specialty.
The rest of our team came from internal transfers, people who were less qualifed but proved that they would have a positive impact on the output of the team.
First, physics has been computation driven since the 1940s. When I was in grad school, programming was vital to my project. I wrote mountains of code, and also designed my own electronics.
Second, I noticed a weird difference between physics and engineering. Engineering students were given problems that were expected to be solved within a particular domain of engineering. A physics student might be given a problem with no idea of how to solve it, much less even how to define the problem itself clearly. That was my project. So a physics student could find themselves having to learn practically any technical skill.
Third, a matter of motivation. We knew that we would need to make ourselves employable. My project would occasionally fail in some spectacular way that would require me to learn more programming, or more electronics. Imagine that. ;-)
Fourth, possibly also as a matter of employability, math and physics people have always figured out how to worm our way into ill-defined, nascent areas of technology. These are areas that have not yet created a mainstream training pipeline, so we can plausibly claim to be as well trained as anybody. "Embedded systems" was such a field when I was finishing school.
Running and extending numerical simulations with supercomputer clusters is just something you have to learn to do in order to solve your problem in some cases, apparently.
We don't work in advertising / marketing / business analytics, which is very easy to have an intuition about; and we don't strictly work with images, which, again, isn't terribly difficult to have an intuition about. So a strong scientific background is actually a major plus for us.
I could see that if we were doing purely deep learning image classification or advertising prediction then the physics degrees would be less useful, but thankfully we don't.
How would a person with a MSc in CS and 6+ years of work experience rank for you? (Converting ~4 years of PhD to 2 year MSc + ~2 year work exp)
You're right about "research", although I'd hesitate to use the word hardcore hah. We get tons of candidates with ~2 years of slightly relevant work, but rarely do we get candidates with the exactly right relevant work. The "qualified" candidates I spoke of have ~4 years of slightly relevant work, but there are only a couple of them, and they're either out of our hiring budget or currently working for our team.
I think getting insights out of massive amounts of unstructured data is a relatively new problem for most organisations, so that might explain the skill gap. Personally I also think the ubiquity of ORMs might also play a role, as they remove a lot of the day-to-day data handling that many engineers would otherwise have to do. But I might be completely wrong about that.
The intent is to get away from just merely slicing and dicing data to come up with some insight that explains causality.
Whether we are afforded the time to do a proper and thorough analysis is a different matter.
Managers want to understand what’s happening in their company; they’ve been told data scientist can make that happen, or better, fix the problems automatically. They expect the solution to come without significant investment in data collection. One, even over-paid, data scientist is a far smaller budget than having your dozens of teams make an effort in fixing data collection, and setting up a reporting team. That kind of problem is easy to detect during the interview, if you’ve been there, and any data scientist has. What’s left are fresh graduates without diploma, who were taught statistics, but not how to get a project running -- so they don’t know what to say during an interview.
In more details:
Too much focus on stats. The actual skillset required should not be that demanding on statistical side: interviews generally focus on testing that exclusively. It takes the form of Whomever has any statistical background will grill on what they know best, their own MSc thesis. The thing is: no one can know the minutia of every model, certainly not a student’s interpretation of them, so those end up being a lot of “I would have to go back to the documentation.” Unless your interviewer is mature enough to either know that’s reasonable or stop doing that, it’s a lost cause. At best, you end up hiring a pile of people with the same focus, and every problem will look like a nail to a group of people carrying hammers. One alternative is to ask about something they did recently, but that’s rarely very instructive, either because the interviewer has no idea if that makes sense, or because the implementation has a lot of properties that make the whole process tedious. The focus should be the ability to learn new techniques at speed. Because the vast majority of the time, the right model is:
a. a basic implementation of a better one, just to get things started (fix the data collection, the engineering of the input to the serving model, think about scaling the serving model, improve the actions taken from the decision, improve the decision format and the actual model objective)
b. a model that you haven’t heard about, and you need to learn about it within a calendar that is conditioned by a. not being sufficient.
Be more project-focused Description like “codes and does statistics better than either” fail to mention that the bulk of the time, a data scientist is actually more of a project manager, trying to identify blockers, get the integration of their tool prioritised, compare the impact and risk of different approaches; there’s also a big chunk (if not all of it) of being an analyst, generally to clarify what problem is being asked, or checking that the data is consistent. That’s if all is reasonably good: far too often, the time is spent trying to get management to grasp that Machine learning can only predict something, and that what to do with that, what to expose to the users, how, when, is more the remit of the product team i.e. you don’t have a real direction. Many large companies actively select against people willing to say that the kind is naked.
Some company test for “business sensitivity” but that too often means checking that aspiring data scientist can walk through a P&L. Reality is closer to: Can you listen to a project manager, frustratingly oblivious to his own contradiction and general ass-holery, rant for 45 minutes without strangling him? Because you need the support of his team to get through.
Fight for good data I have yet to see a company that had data you could use for data science on day 1. The vast majority of the time, it was inconsistent. Not a little bit, or hard to find: big, glaring, obvious contradictions like 20% of the revenue unaccounted for. If there are analysts, they generally have found a way around it, rather than address the issue. In more mature companies, there were some problem that would make basic model stumble, and more nuanced one go to very stupid conclusions. The classic case of a trip-up on good, but not well-structured: What drives retention? Completely failing to delivery your service: people will re-order within the hour, a larger ratio than any other case. Executives pay for the servers so they think that data quality if measured in peta-bytes. They get fed a continuous streamed of over-analysed data, filtered by hand from any inconsistency. They have no idea how inconsistent it really is. Any improvement is seen as a cost centre, rather than the ability for the company to be better informed.
Don’t get me started on technical debt, short-term focus, insensitivity.
What data scientists need, before they even join: clear objectives, clear breakdowns to understand what effects are correlated, clear plans to address some of the concern, pilot tests to make sure that those solutions would work, and collect some data, audited data pipeline with clear documentation to avoid misinterpretation, dedicated engineering to help with data logging, hosting computation, serving model. I am willing to bet that hardly any of those dozen of thousands of jobs looking for candidates don’t have most of that.
There is a small piece of good news: Agrawal & al. summarised their theory on how ML will make predictions cheaper in a all-access book, Prediction Machines. The predictable success of that book will help. Basically, it tells about what ML does, without ever doing any ML at all: it just looks at how it’s a lynchpin technology, like search engine, digital photo, etc.
This x100. I've seen so many places where the budget for "data" is zero. This includes funds for paying for new data, budget for additional people/software to support data collection/cleaning, money to provide to existing data vendors/suppliers to provide better data, support from legal to re-negotiate contracts with existing data vendors who historically have only provided aggregated summaries, etc.
I've seen situations where a Data Scientist discovered that a data pipeline was broken reports using that data had errors for years. Rather than trying to fix the pipeline, the Data Scientist got in trouble.