Hacker News Reader

Comment by barkingcat | Hacker News Reader

This can be applied as "nobody listens to the people who actually do the work" as in company hires ML/AI experts to analyze purchase records and service records, and spits back out trends that the service front line workers (tier 1) already knew dead solid.

Then the company doesn't listen to either group of people (neither tier 1 sales/support people, nor the ML people) and then fires / shuts down the entire division because "upper management didn't find value"

View on HN

Or it could be that a lot of data is wrong. It may be "technically" correct, ie the table in a database produces X. It is no surprise that executives would ignore what the "data" says because they don't trust it.

A lot of time they are right to ignore it. I've seen tables say X, but there was some flaw up the capture stack. Very few data analyst have the broad based knowledge and dedication needed to trace the data stack to establish the needed trust with the executive team.

Probably the biggest part of a lot of projects is cleaning up and normalizing the data (just like it was with data warehousing last century). A lot of high power data scientists don't want to hear that--especially if they're research-y.

If you don't at least tolerate data cleaning, you should find a different job than data scientist.

I would have expected the researchy people to be better at it, as often you'll need to collect and analyse your own data during grad programs, and thus have some experience.

I kind of like data cleaning, I'm a weirdo like that

Some of the better historic manufacturers that "made it" were known to have good managers go and visit the filthy masses on the factory floor and get a feel for what's going on. It was very valuable for me when I used to help with manufacturing testing. I always spent some time with the techs and the people on the floor assembling stuff. A lot of it was useless but a lot of it was worthwhile and we learned to trust each other better instead of the "eggheads upstairs" and the "jarheads downstairs" that seemed to be most prevalent there.

I think what you have done is similar to what is depicted in "The Goal" by Eliyahu M. Goldratt. The graphic novel depicts it in a more succinct manner.

Contempt for this kind of knowledge is almost a religion in Silicon Valley.

Reply on news.ycombinator.com