Hacker News Reader

Comment by jstx1 | Hacker News Reader

Data engineers don't work with machine learning at all. In fact one of the reasons why it developed as a job title over time waas specifically to differentiate the people who work with data but don't do any statistics or ML. If a DE who is doing "datasets for ML" decides to call themselves an ML engineer, they're just getting a bit too creative with the job titles (maybe they want a career change, more money, they think it sounds better, all sorts of reasons).

As an ML engineer you might need to do some data engineering work as part of your job but not the other way around.

View on HN

> Data engineers don't work with machine learning at all.

This very much depend on the company. From experience DE is used as a catch-all title.

Data scientist is much more catch-all from what I've seen. But a lot of that varies a lot by geography too (for example in the US people very often use DS very differently from how the title is used in the UK).

>in the US people very often use DS very differently from how the title is used in the UK

I haven't heard about this before and now I'm curious- can you elaborate on the differences?

In the US it's more common for data scientist to be similar to a product analyst or a data analyst perhaps with better technical skills. In the UK data scientist is more likely to be someone who is doing applied ML work (other titles for this are ML engineer or applied data scientist).

Obviously it's not a perfectly clean separation but it's a trend, and people sometimes end up really talking past each other. You can see on r/datascience which is very US-heavy how people often recommend to beginners not to bother with advanced ML, stick to SQL, basic Python and analytics, and in the UK data science job market that's outright bad advice (it's fine advice for the UK analytics market which is a separate thing).

This. "Data Engineering" is pretty far from having a standard definition in the wild. If someone is describing a role as "data engineering" about the only thing you can count on being true is that it involves data.

I haven't come across that. Unless the job title is being polluted like DS was (to include all aspects of data management), DE is specifically about data pipelines and not the models generating those clusters, predictions, or classifications.

Somewhat surprised that there's a separate job category for what sounds like large-scale data cleaning and aggregation work (which IMO is 90%+ of the effort involved with data science).

Anyway, I'm going to go back to my 5K+ lines of code for an upcoming conference submission - almost all of which involve data cleaning and aggregation - and think about how I could be making a 2x more than I am now.

Thanks Hacker News.

Depends. If their favourite data engineer says "Oh hey, I can write tensorflow too", then guess who get the job of to "productionizing" their crappy data science notebooks?

You have two much more likely options:

1. The person who developed the notebook is responsible for productionizing it. (No, it's not all crappy notebooks and some data scientists can indeed write high quality code).

2. You have someone like an ML engineer whose job it is to do this.

What you're describing seems like the least likely option; at least on the teams I've worked on "I can write tensorflow" would get you nowhere if that's not already a part of your job description.

Well, anyways that's what I was doing this time last year. Much too small a team to have a dedicated ML engineer though.

Reply on news.ycombinator.com