Hacker News Reader

Comment by tictacttoe | Hacker News Reader

I think what companies really want is smart generalists with advanced math, programming, and modeling skills coupled with domain knowledge. That skill set will always carry high value in technical companies.

The reason it carries value is the skills are difficult to acquire. I think the recent decline in interest reflects the rise of new data science candidates that are taking the path of least resistance to a career in data science. Rather than pursuing problem solving, people are pursuing "data science" which is a nebulous term in and of itself.

View on HN

I am wary when people wax lyrical about all of the ways they love using machine learning on data. It makes me nervous because i worry that they have a hammer and can't wait to use it on anything vaguely nail shaped.

Yep, that's why I make sure to set time aside for toy problems. Creating contrived problems can sometimes scratch that itch to use a certain technology that don't really fit into what I'm doing at work.

ML makes predictions; testable predictions.

Machine learning is an area where you need to be able to produce results. Fake it ‘til you make it isn’t going to cut it for long. Either these people produce something that works, or they don’t.

> Machine learning is an area where you need to be able to produce results.

Having to produce results is one thing. Mindlessly throwing tensorflow/pytorch at problems is an entirely different problem.

It's like those front-end devs who mindlessly insist that they need to use heavy javascript frameworks with convoluted build processes such as React/Angular to churn out a static web page with a couple of paragraphs and images.

> ML makes predictions; testable predictions.

Well, no. ML solves the classification problem, not the prediction problem.

E.g.: The "is this a cat picture" problem is effectively solved, but we _still_ can't reliably predict something as primitive as a simple binary proportion.

Yes and no.

You can certainly produce some plots and numbers, and possibly even plots and numbers that look good to your boss/clients/investors. The (multi)million dollar question is whether those numbers are actually meaningful. I think this is where a lot of ‘data science’, both in industry and academia, falls down.

Some state-of-the-art models don’t even generalize to test sets drawn from the same database, let alone similar data sources or the actual business problem. Unless you run a pet shop, telling breeds of dog apart, a la ImageNet, is probably not your goal.

>I think what companies really want is smart generalists with advanced math, programming, and modeling skills coupled with domain knowledge. That skill set will always carry high value in technical companies.

What companies want is to be "in" on the data science hype, while they have no clue what they are doing and the most advanced "data science" they need are simple graphs, boxplots, and linear regressions.

Yes. Unlike software development, data science is not completely business agnostic and a fair amount of business understanding is required. For e.g. if you work with sales data, you need to be aware of seasonalities, purchasing patterns etc to understand the trends that you observe to discern between what is a true outlier and what is explainable.

What makes me cringe the most is to see flashy presentations with claims akin to 'Data Science will change your world'.For sure, it can and has been proven to automate decisions (think, credit scores), assist in decision-making (think, sales trends) and anomaly detection (think, security systems). I find so many data scientists that I interview are so hung up about the esoteric techniques they employ, often failing to even explain why was it useful or how it helped their businesses.

What has been transformational and path-breaking is the breaking of enterprise monopolies in this space (for e.g. SAS/IBM SPSS) and a variety of open-source frameworks have made it easy and convenient, apart from opening it up to software developers to build these skills. Important, though, to not lose of the sight that data science is at the sweet spot of expertise in domain, data and technology.

> The recent decline in interest...

Can you quote a source for this?

Yes I'd be quite keen to see that also, I wasn't aware there was a decline already (although it's obvious the interest can't just keep going up forever).

Reply on news.ycombinator.com