Comment by noirbot | Hacker News Reader

Comment by noirbot | Hacker News Reader

Parent Full thread

Yea, I feel like "Data Science" was always a Zero Percent Interest phenomenon. It's the software company version of Dogecoin or GameStop or AMC. If you've got free money, why not throw 10mil at the moonshot of discovering a diamond in the rough of your data that revolutionizes your business.

For the last 10 years, every data team I've been adjacent to has had a near infinite budget, hired more people than the rest of the company combined, had an abundance of "scientists" for whom any sort of data process, version control, scientific method, or cost controls were never part of the equation. It really doesn't surprise me that companies have decided they're not throwing good money after bad.

There's absolutely good work to be done in Data, but it requires people who know what they're doing, and not just "here's 10 years of postgresql databases. I dunno, find us 10% profit somewhere in it". There's not many companies that really have the data or personnel to make it happen.

I think that’s a bit harsh – but only so much, as I’ve had to explain basic stats literacy to several consultants – but there’s a common factor to many “AI” initiatives: some executives really want to be able to tell their buddies that they’re doing amazing things, and often believe there are huge untapped opportunities to make or save money which all of the employees they don’t trust haven’t mentioned.

In reality, of course, there probably just isn’t that much money at play so the high-budget approach is unlikely to break even, and more critically it’s unlikely that people are prepared to change how the organization makes decisions. I’ve known some people who worked for places where they had data showing solid improvements but it either hit politics or was simply small enough that their executives didn’t want to go with it because it wasn’t the magnitude they’d promised to justify the data science program.

It's definitely harsh, and I want to be clear that I don't think it's innately a bad idea or destined to fail, but when it's easy to throw millions of dollars at a moon shot, trying to spin data straw into gold is worth a shot. It's similar to people trying to make a self-driving car, and how many of those are struggling/ending recently. It's not that there's nothing there, but when loans aren't free, you can't just throw money at it and hope.

It's a field that's just been over-hired because it's been a field where you can go "we're 2 years away from something big" for 10 years and the money was cheap enough to just keep paying people with questionable results or prospects. It doesn't shock me at all that people are having a lot of trouble getting work in the field. Even 8 years ago, going to college job fairs, I almost had to herd away Masters students in ML and Data Science who were already having a hard time getting a job and were resorting to applying for QA positions at web dev companies.

Eh. The real elephant in the room with "ML and Data Science" is checks and balances. They take over as the eyes of the company. Guess what department is seen as kicking ass? Guess which department is getting most money? Most hires? Data. Data.

It's like having the local police department check on the local police department.

Reply on news.ycombinator.com