Hacker News Reader

I've been in such an org. The problem is that few decent coders want to do data engineering work, because they'd rather build software and because data engineering pays less and is kind of a dead end in a lot of org.

You end up with a lot of DBAs that self-style as Data Engineers.

View on HN

I'm in Australia and data engineers here are overpaid, afaict slightly better paid on average than software engineers who are better paid than data people.

So you end up with bizarro stuff where a $150k AUD data engineer who basically can't code or even set-up FTP telling some maths PhD data scientist on $100k what he can or can't do. Managers, unable to gauge technical ability, assume the data engineer must be highly skilled because they're getting paid so much.

The wages and lack of skill I think are symptomatic of there being no real entry-point in data eng. There's a very slow osmosis of people from genuinely skilled areas into data eng, the main drawback being that the entire culture of data eng is defective with so many unskilled hacks.

I work in the aus data space, I’ve met a lot of intelligent PHD data scientists, but very rare is it to meet one who can scale their work efficiently across the org. The amount of time wasted on models that never make it to production is ridiculous.

Reply on news.ycombinator.com