Deep learning job postings have collapsed in the past six months

There are obviously places in my company where ML is making an enormous impact, it's just not something that's fit for every single place where decisions need to be made. Sometimes doing some analysis to inform blunt rules works just as well - without the overhead of ML model management.

Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on every decision of what projects an expectations were realistic and what were not. I left, as did everybody else that reported to me, and we were replaced by people who would make really good BS slides that showed what upper management wanted to see. A year after that the whole initiative was cancelled.

Back in 2000 I was in a similar position with a small company jumping on the internet as their next business model. Lots of nonsense and one horrible web based business later, the company failed.

It's the same story over and over again. Some winners, lot of losers, many by self-inflicted wounds.

Many data scientists I knew were either sitting on their hands waiting for data or working on problems that the downstream teams had no intention of implementing (even if they were improvements). I still really believe that ML (be it fancy deep learning or just evidence driven rules-based models) will effectively be table stakes for most industries in the upcoming decade. However, it'll take more leadership than just hiring a bunch of smart folks out of a PhD program.

It was difficult to attract top ML talent no matter how much we offered. Everyone wanted to work for one of the big, recognizable names in the industry for the resume name recognition and a chance to pivot their way into a top role at a leading company later.

Meanwhile, we were flooded with applicants who exaggerated their ML knowledge and experience to an extreme, hoping to land high paying ML jobs through hiring managers who couldn’t understand what they were looking for. It was easy to spot most of these candidates after going through some ML courses online and creating a very basic interview problem, but I could see many of these candidates successfully getting ML jobs at companies that didn’t know any better. Maybe they were going to fake it until they made it, or maybe they were counting on ML job performance being notoriously difficult to quantify on big data sets.

Dealing with 3rd party vendors and consulting shops wasn’t much better. A lot of the bigger shops were too busy with never ending lucrative contracts to take on new work. A lot of the smaller shops were too new to be able to show us much of a track record. Their proposals often boiled down to just implementing some famous open source solution on our product and letting us handle the training. Thanks, but we can do that ourselves.

I get the impression that it is (or was) more lucrative to start your own ML company and hope for an acquisition than to do the work for other companies. We tried to engage with several small ML vendors in our space and more than half of them came back with suggestions that we simply acquire them for large sums of money. Meanwhile, one of the vendors we engaged with was acquired by someone else and, of course, their support dried up completely.

Ultimately we found a solution from a vendor that had prepared a nice solution for our exact problem.the contracts were drawn up in a way that wouldn’t be too disastrous if (when?) they were acquired.

I have to wonder if an industry-wide slowdown to the ML frenzy is exactly what we need to give people and companies time to focus on solving real problems instead of just chasing easy money.

This understates how awful ML is at many of these companies.

I've seen quite a few companies that rushed to hire teams of people with a PhD in anything that barely made it through a DS/ML boot camp.

To prove that they're super smart ML researchers without fail these hires rush to deploy a 3+ layer MLP to solve a problem that need at most a simple regression. They have no understanding of how this model works, and have zero engineering sense so they don't care if it's a nightmare of complexity to maintain. Then to make sure their work is 'valuable' management tries to get as many teams as possible to make use of the questionable outputs of these models.

The end is a nightmare of tightly coupled models that nobody can debug, trouble shoot or understand. And because the people building them don't really understand how they work the results are always very noisy. So you end up with this mess of expensive to build and run models talking noise to each other.

When I saw this I realized data science was doomed in the next recession, since the only solution to this mess is to just remove it all.

There is some really valuable DS work out there, but it requires real understanding of either modeling or statistics. That work will probably stick around, but these giant farms of boot camp grads churning out keras models will disappear soon.

I worked for a financial services co that saw massive gains from big data/ML/AWS. Given, we were already using statistical models for everything, we just now could build more powerful features, more complex models, and move many things to more-real time, with more frequent retrains/deploys bc of cloud.

I do agree that companies who don't already recognize the value of their data and maybe rely on a consultant to tell them what to do might not be in the position to really capitalize on it and would just be throwing money after the shiny object. It really does take a huge overhaul sometimes. We retooled all of our job families from analysts/statisticians to data engineers and scientists and hired a ton of new people

So now my advice is others is "if you can start with some first principles equation or system of equations... start there and use optimization/regression to fit the model to the data."

AND: "if you don't think such equations exist for your problem... read/research more, because some useful equations probably do exist."

This is usually pretty straightforward for engineering and science applications... equations exist or can be derived for the system under study.

In my very limited exposure to other areas of machine learning application... I have found quite a bit of mathematical science related to marketing, human behavior, etc.

However someone is trying to make robotic process automation the Next Big Thing - which I think is hysterically funny.

It is clear that for most of the companies who are investing in deep learning are tangible results are always around the corner, and maybe 1 in 100 will build something worthwhile. But here is the carrot driving them all on, it's like the lottery: you have to be in to win. The stick is the fear that their competitors will do so.

This field is more art than science, give talented people incentive to play and don't expect too much for the next decade.

> They were forcing ML in to places it didn't (and may never) belong.

I find that I spend a lot of time as a senior MLE telling someone why they don’t need ML

That's because it didn't get a chance to mature and to show how it could be powerful. People kept trying to force hadoop into it and call themselves "big data experts"

We've gotten a bit more clarity in this world with streaming technologies. However, there hasn't been a good and clear voice to say "hey .. this is how it fits in with your web app and this is what you expect of it". (I'm thinking about developing a talk on this.. how it fits in [hint.. your microservice app shouldn't do any heavy lifting of processing data])

(or) In simple terms, is profitable commercial Deep Learning just for oligarchies?

The problem is that we don't have enough ML engineers and many who go by this title are not really capable of doing the job. We're just coming into decent tools and hardware, and many applications are still limited by hardware which itself is being reinvented every 2 years.

Take just one single subfield - CV - it has applications in manufacturing, health, education, commerce, photography, agriculture, robotics, assisting blind persons, ... basically everywhere. It empowers new projects and amplifies automation.

With the advent of pre-trained neural nets every new task can be 10x or 100x easier. We don't need as many labels anymore, it works much better now.

Not because it would be a good use case (although there are some for our product), or because it would be of any practical benefit, but because it makes for good marketing copy. Never mind the fact that nobody on the team has any experience with machine learning (I actually failed the paper at university).

I work in 2D animation and we were able to design our current pipeline around adopting ML at specific steps to remove massive amounts of manual labor.

I know this doesn't disprove your anecdote, I just wanted to point out that real businesses are using ML effectively to deliver real value that's not possible without it.

Any technology too complex for the managers who purchase for it to understand fully can be sold and oversold by marketing people as "the next big thing".

Managers may or may not see through that, but if their superiors want them to pursue it or if they need to pursue something in order to show they're doing something of value, then they're happy to follow where the marketers lead.

Java everywhere, set top TV boxes, IOT devices, transitioning mainframes to minis, you name it... the marketers have made a mint selling it, usually for little benefit to the companies that bought into it.

1) AI 2) Machine Learning 3) Robotic Process Automation

They felt RPA would help them stay more competitive since there are tons of smaller health care companies who are moving faster and innovating faster because they're not buying up companies and having to integrate all their technology at a sloth's pace. They thought RPA would be a way to mitigate these issues.

18 months later and the one manager, director and VP in my org has all but said they don't care about RPA, all their money is going into ML and AI. Even though in all the presentations I've seen them put on, its all blue skies and BS about "IF we had this, we COULD do this." Nothing concrete at all about how the plan to use ML to increase profit margins or reduce overhead.

Right now, our team is basically an afterthought in the company and I'm already starting to interview elsewhere with the knowledge at some point, they're going to kill my team and cut everybody loose.

The easiest way to solve many problems is through lexers, regular expressions and plain-old pattern matching. But that doesn't sell, so, they call it AI anyways.

It's kind of batty actually, people looking for ideas to make money just been taking old ideas and attaching ML to the side of it as if that automatically made it better. And then not educating their customers on the limitations of ML both generally and with respect to their data size.

I personally think the companies that make and sell the software that the police used to make incorrect arrests should be legally liable. Yes, the police shouldn't have blindly trusted the software, but I guaran-fucking-tee you part of why they did is the marketing from the company themselves.

Btw. I don't like twitter's new feature that prevents everyone from responding to a tweet that was used by @fchollet. It no longer feels like twitter if you can't engage.

In the future, I expect ML to also fall out of the "AI" umbrella - it gets used primarily for "smart code we don't know• how to write", so once that understanding comes, it gets a more-specific name and is no longer "AI".

•"know" being intentionally vague here, as obviously we can write both query planners and ML engines, but the latter isn't nearly as commonplace yet to completely fall out of the umbrella.

Without the skillsets to work with and then understand that data, they are forced into this long process of asking for data to be put into reporting and dashboards and then once they finally get them, either fixating on the limited metrics it provides while being oblivious to other context not in front of them, or to instead forced to start another long iteration to adjust that reporting and dashboards.

We’ve gone almost 30 years believing management was the sole skill required to manage teams and companies, but dealing with the new era of data is starting to show the limits

* https://ai.googleblog.com/2019/01/transformer-xl-unleashing-...

* https://arxiv.org/pdf/1901.02860.pdf

* https://arxiv.org/pdf/1906.08237.pdf

My understanding is that a CMU student interned at Google and developed most of the pieces of TransformerXL, which formed the basis of XLNet. The student and the Google researcher further collaborated with CMU researchers to finalize the work.

(For the record, I think the remainder of your points do not match my understanding of NLP, which I do research in, but I just really wanted to clarify the XLNet story a bit).

Little things compound such as optimizers ( Ranger/Adahessian), better RNN ( IndRNN, Linear Transformers, Hopfield networks ) and techniques (cache everywhere, Torch script,gradient accumulation training)

It's not just the data scientists fault. I once heard our chief data scientist point out that they don't want to hand off a linear regression as a machine learning model -- as if a delivered solution to a problem has a minimal complexity. She absolutely had a point.

Clients are paying for a Ph.D. to solve problems in a Ph.D way. If we delivered the client a simple, yet effective solution, there's the risk of blow-back from the client for being too rudimentary. I'm certain this extends attitude extends to in-house data scientists as well. Nobody wants to be the data "scientist" who delivers the work of a data "analyst." Even when the best solution is a simple SQL query.

Our company kind of sidesteps this problem by having a tiered approach, where companies are paying for engineering, analysis, visualization, and data science work for all projects. So if a client is at the simple analysis level, we deliver at that level, with the understanding that this is the foundational work for more advanced features. It turns out to be a winning strategy, because while every client wants to land on the moon, most of them figure out that they are perfectly happy to with a Cessna once they have one.

You need not only real use cases, but use cases that happens to well with DL’s trade offs and limitations. I think many companies hired with very unrealistic expectations here.

We did have courses explaining the "around" of the whole process though, but that's not as hyped.

There are 5 ML models that we maintain where I work, and none of then are more complicated than linear regression or random forests. Convincing me to use something more complex would take an enormous amount of evidence. Domain knowledge is king.

However, megacorps do not seem to suffer much for such continuous lagging in hiring. I do not know why this is so: is it that they still hire smart engineers who can easily change groups and fields or do they work on their core technology to help build the next peak (after the debris are washed away in a fad crash there is often a technology renaissance).