Comment by PaulHoule | Hacker News Reader

Comment by PaulHoule | Hacker News Reader

Parent Full thread

If it were my business or my team I would want an ML Engineer to relentlessly knock down any barriers to getting models into production.

The deployment of ML is still per-paradigmatic in that people today deploy products based on luck, blood, sweat and tears. Many projects fail and many more never get started.

I am sure that somebody somewhere had a team that gets results consistently, I can partially picture what it would take to do so, but that's not the usual condition. (That team is so productive that it doesn't need to hire more people!)

For instance, you might not want to do any of these things, but any of them could be essential to putting something in production:

   * manually classifying 10,000 images or documents
   * figuring out how to run a program in the Java and C++ debuggers at the same time
   * be able to convince somebody that their research programme is a dead end
   * survive a 40 minute build process
   * push back against a 40 minute build process and reform it
   * reproduce a result from a scientific paper
   * get a "data scientist" (who thinks he is too good to listen to you,  or even to the CEO) to standardize their python distribution so their work isn't wasted
   * find the scalability bottleneck in a system,  where it is,  not where you want it to be.

Many teams fail at ML because there was an essential task that nobody wanted to do that didn't get done.

People who do these things today will be able to say tomorrow that they were part of a successful team. Other people will expect great praise because they diddled around with Word2Vec for a summer.

Some day the social structures will be in place and they might have a role made for you that will make you productive but for now you must ask: who do you want to be?

I overall agree with the sentiment on delivery and needing to deal with a variety of issues, but one nit:

If it were my business or my team I would want an ML Engineer to relentlessly knock down any barriers to getting models into production.

My favorite is when they knock down the question of whether a certain project even needs ML to focus on getting ML into production.

Many teams fail at ML because there was an essential task that nobody wanted to do that didn't get done.

Many teams fail at ML because there was a task that didn't need ML (or at least anything more than a linear model) that is made opaque to anyone but an ML engineer after they implement it.

> * manually classifying 10,000 images or documents *

Ah, the 50's era sweatshop mentality! Most excellent. I would fire a person who wasted days doing such a stupid operation. Take 10 minutes, do some research, find a better way. Spend some company money to get it done today. If you can't figure a good way, start asking up the chain.

Very curious how you think ML without data is going to work. 10 minutes of research will neither give you a solution to that nor will it procure the data needed.

In the end, ground truth data is almost always going to originate from humans putting in man-hours to create it, one after the other. Whether those humans need to be ones that are paid ML Engineer salaries for this is an important business question, but someone will have to do the work.

Yup.

People complained about the cost of getting an expert to cough up secrets about their thought process in the 1970s.

The essential problem is not different in magnitude with ML because you need to design the classification to be solvable and then get the right answers for many hard cases.

From an academic perspective ML is advantaged in that multiple people can train algos on the same data, which leads to a certain kind of progress through competition.

When it comes to most projects where you need the model you have to supply your own training set. Sometimes there is a download or a weird trick, but that conversation that ‘we can’t afford you to make the training set’ leads to bad outcomes:

   * you never get a training set
   * you download the wrong data from Kaggle
   * you get an unusable training set
   * you wind up spending more time of expensive people herding inexpensive people, when….

one ‘expensive’ person who could get over the stigma of doing what they think is menial, be 4x as productive as an imagined ‘cheap’ person and then the team can move on to other tasks that are necessary and unglamorous.

The team can still win the game by default in 2021 because your competitor thinks it is too good to do those things.

Reply on news.ycombinator.com