Comment by machinekob | Hacker News Reader

Comment by machinekob | Hacker News Reader

Parent Full thread

I don't see anyone writing that ML/DS is a lot harder than a typical SWE and results are a lot less visible for a VERY long period of time.

A lot of the time you could train model prepare pipeline and test enviroment for 3-6 months before you get good enough result to push it into production. And it can get extremely stressful rly fast if you care about that, because not every model is good enough for production so once per year you could have as low as 1 or even 0 models that are working fast and good enough for proper usage and this can burn you after just a 1-2 years of work in the field (I know at least 4 people who just drop ML and go for SWE after 1+ year of ML work and all of them are a lot happier with classical backend/devops jobs).

In SWE after 2-3 days you can have small stuff working fine and after few weeks push your small code into production codebase fixing some stuff or optimising smth as there is a ton of potential in almost every codebase for "easy" upgrades in ML space everything is extremely competitive your results are "state of the art" or they are not if you want to upgrade model it better be sota or you would get asked "why we don't just implement/use ...?".

I still love my job but for sure but I prefer working with ops, classical SWE and deployment then model training, optimising and learning/collecting new datasets (I have 5 years of commercial exp in ML/DL maybe it gets better after 10 years or I'm just boring out who knows)

IME the core issue here is that there are far more SWE jobs than ML/DS jobs, which leads to a far greater variety of SWE jobs, which means people are better able to find SWEs jobs where they are able to find a job that matches their job environment. If you want to coast, there are SWE jobs for that. If you want to work 12 hour days on the bleeding edge, there are jobs for that as well.

>A lot of the time you could train model prepare pipeline and test enviroment for 3-6 months before you get good enough result to push it into production. And it can get extremely stressful rly fast if you care about that

Some of the the major job challenges in ML/DS are that the field is new, the number of jobs are far fewer than SWEs, the teams are small, and the people who can exert control of you have wildly unrealistic expectations of what it takes to build a useful model. So it's easier to land in a crappy job where stakeholders add to your stress because you are not "delivering" according to their definition of "I thought ML would solve all my business problems in 2 weeks."

I personally prefer ML/DS to what I see a lot of SWEs do. Yes, it can take months to get something working (or maybe it will fail after all that effort), but I'm also involved in the end to end of what I'm building and I get be part of defining the the "what", "why", and "how" and not just crank out yet another story on a Kanban board. What really helps me in my job is having a management chain that pushes back and educates stakeholders on why things take time and why that time is required to have a certain level of quality.

There are plenty of SWEs (on HN and elsewhere) who say they feel like factory workers. Somebody else makes a bunch of decisions and the SWEs simply churn out features and stories. So the quick wins turn into an endless flood of drudgery, just like how when you learn to fix a toilet you feel great, but if you fix 20 toilets a day, it probably becomes monotonous pretty quickly.

Reply on news.ycombinator.com