Comment by madenine | Hacker News Reader

Comment by madenine | Hacker News Reader

Parent Full thread

The market for serious data scientists is growing, but is not as dire as these articles make it seem.

The private sector has wrapped 'senior data analyst','BI developer','data base administrator', and 'data engineer' into 'data scientist', along with the ML/AI/stats roles you would expect.

So sure, when you put all the roles that touch data into one basket, and we're in a world that demands increasing familiarity with data systems and data analysis techniques... a gap is going to appear. Color me shocked

Typical data scientist role I see posted is looking for: - SQL - AWS/Azure - Tableau/Power BI/Other BI tools - Python/R - Increasingly Hadoop/Spark

With Java/Scala/C#/C++ listed as 'nice to haves' along with 'machine learning' and 'big data'. These are not research roles, and while they often want a masters degree or more they really don't need them.

If you fit that bill you can probably get a role paying $95-120k just about anywhere in the US. You'll wrangle data, make pipelines, build dashboards and occasionally deploy a model or two when the planets align.

If you want to/are capable of doing more ground breaking/cutting edge work, these roles will crush your soul in 8-12 months.

>If you fit that bill you can probably get a role paying $95-120k just about anywhere in the US. You'll wrangle data, make pipelines, build dashboards and occasionally deploy a model or two when the planets align.

If you expect to pay that little to anyone who knows the above to any degree you will have a bad time.

A number of projects I have joined absolutely look like someone treated them as a way to learn the technology before moving onto better paying jobs, leaving the business with a terrible mess that's largely unfixable without a rewrite. You're better off hiring one developer that knows what they are doing at above market rates than three who don't for the same price.

Outside the bay area, 95-120 is about the going rate for someone with those skills. Add a little for the bigger cities, and maybe 10k for every 3-5 years of experience.

>If you expect to pay that little to anyone who knows the above to any degree you will have a bad time.

I believe you misunderstood his point. The $95-120K figure is for people who do not do ground breaking research. Anecdotally, I know people who get hired for data science who are doing very little data sciencey stuff (a bit of data manipulation/slicing and lots of plots - almost no actual ML). These people should not get paid more than a typical SW developer, and decent SW developers should get paid more.

> - SQL - AWS/Azure - Tableau/Power BI/Other BI tools - Python/R - Increasingly Hadoop/Spark

Is above and beyond a typical SW developer. That's three languages and two dsls.

If you expect that on a 120k salary, you will be what you pay for, e.g. a code monkey living on peanuts.

First, I don't think he meant they expect you to know all of these - just a subset.

Second, most people I know who know R or Pandas or other Python numerical stuff (NumPy/SciPy) are horrible programmers. They're good at numerical work - but that doesn't translate to good coding skills. The bar is not that high to know it.

It's somewhat similar with AWS/Hadoop. Most people I know have taken learn these (online training, in person classes, etc) with very basic SW skills.

I don't doubt expert programmers exist who know these technologies - I'm just pointing out that it's not that hard to learn some of these. Plenty of people learn these without knowing much SW. To be a good SW developer takes a lot more.

>If you expect that on a 120k salary, you will be what you pay for, e.g. a code monkey living on peanuts.

You must be in the Bay Area. A 120K salary is above the median for SW developers for most of the country, and grants them a great lifestyle: Big house, two cars, vacation travel, etc. Hardly "living on peanuts".

The GP was clear he was not talking about the Bay area:

>If you fit that bill you can probably get a role paying $95-120k just about anywhere in the US.

Finally, the point is: If you studied actual Machine Learning, he is pointing out that these jobs are mostly not that, and you will just do a boring job.

>You must be in the Bay Area. A 120K salary is above the median for SW developers for most of the country, and grants them a great lifestyle: Big house, two cars, vacation travel, etc. Hardly "living on peanuts".

I'm not in the US currently. I have worked with teams in New York in finance and the wages expected were multiples of 120k. Likewise in London and Honk Kong.

So, some of the most expensive cities in the world.

How about the likes of Minneapolis or Frankfurt or Jakarta?

Yeah, outliers. The rest of the US where costs of living are radically lower 120k is a reasonable expectation. It's actually a bit lower for subsets of those skills here in Columbus,OH. Mostly with large corps like Chase, Cardinal Health etc.

But you only have a 1400$ mortgage for a three bedroom home here so it's all relative.

Most of the US is not California or the NE.

When someone says "most", they mean the outliers are not included. When one says "most Americans can't point out Egypt on the map", it's rather pointless to give examples of PhD's who can.

That made me wonder, though, in this particular context, are these really outliers?

Even just counting population, the Bay Area's CSA is 8.8M, NYC is 23.9M, Boston is 8.2M and LA is 18.9M. That's already over 18% of the population, just in the major urban centers of California and the Northeast.

It's not much of a stretch for the percentage of tech jobs to be in those areas, and less of a stretch to imagine that the vast majority would be in some kind of high-cost area.

I wasn't able to come up with suitable web search terms to find any numbers, but perhaps someone else has.

Throw in Seattle and there are no tech jobs outside those areas to a first approximation.

I don't know where the people above are getting hired, but the only places I've ever seen outside those major cities are incidental programming jobs to babysit VBA code that runs some business that has nothing to do with computers.

Denver, Austin, Houston, North Carolina, Chicago, Atlanta, Salt Lake City, Boise to name a few. Granted, some of these have recently become pricey, but still nowhere near Seattle levels.

Look at all big companies (and not just the recent Internet ones of the last 20 years), and look at where they have sites. Jobs in most of these places.

> Look at all big companies (and not just the recent Internet ones of the last 20 years), and look at where they have sites. Jobs in most of these places.

But how many of those jobs are technical/programming (in those places)?

Even by population alone (Chicago, Houston, and Atlanta making up the bulk), that's under 10.7%, less than CA+NE (and only half of CA+NE+Seattle?). That persuades me that these lower-cost areas are the exception, not the rule.

Not sure why you'd compare Houston to all of California?

https://en.m.wikipedia.org/wiki/List_of_metropolitan_statist...

Just looking at top 50, you've got a real geographic spread - about half of the people are in CA/NE, the rest are split pretty evenly between the four quadrants...

> Not sure why you'd compare Houston to all of California?

I believe everyone in this sub-thread has used "CA" to mean the SF Bay Area plus greater Los Angeles (and certainly I have). Similary, "NE" was just shorthand for the big cities therein, not the whole region.

Ultimately, population is just a very loose proxy for number of tech jobs.

> Typical data scientist role I see posted is looking for: - SQL - AWS/Azure - Tableau/Power BI/Other BI tools - Python/R - Increasingly Hadoop/Spark

They are looking for programmers.

Unless...

C programming makes you a hardware engineer,

Fortran programming makes you a physicist,

Ada programming makes you a rocket scientist,

and Lisp programming makes you a god.

Reply on news.ycombinator.com