Hacker News Reader

Comment by reilly3000 | Hacker News Reader

I really wish hiring managers read this. I am a data generalist, and have had no traction with obtaining even an interview for a data science job. I’ve setup a private JupyterHub where I run python ETL, interactive models, and dashboards. I deployed Metabase several times and have written hundreds of SQL queries. I’ve used Tableau with gigantic datasets. I built a front end serverless analytics pipeline from scratch with AWS that handles 30M events/mo. I've demonstrably grown revenue and margins in multiple contexts with my data products. I’m working on making a fully dynamic frontend for content recommendations. I have self-taught all of these skills in the past 3 years after a decade in sales, marketing, and entrepreneurship. What I haven’t done: a CS/math degree (mine was music), graduate work, or tech work at a household name. Lived in the Bay Area. Gotten an interview for any data job. Sigh.

View on HN

None of what you describe is something I'd hire into a data science role specifically. Some of those skills are skills I'd expect a data scientist to have (e.g. SQL skills). A data scientist in this context has to have an understanding of basic statistics generally: hypothesis testing, modelling techniques and their applications, and performance tuning and evaluation. They also need to understand how to devise and run experiments that collect and make use of data in practice (i.e. "real world" data). I wouldn't necessarily expect a candidate to be able to derive the formulas involved, but it would be the odd candidate who truly grasped the nuances who could not do so.

The one thing I see kind of missing is a math background or at least a project proving that that is in your skillset (recommendations sounds like it could fit this). There are a lot of people with a similar background to you and normally those are in "business intelligence/analytics" or "data engineering" where they are mostly writing sql queries and interacting with dashboards/OLAP cubes or setting up those dashboards/cubes.

That's perfectly fine but it's not what traditionally is referred to as data science. I'm actually quite annoyed at what has been happening to the term data science lately - it's supposed to be some stats-heavy/applied-AI role but a lot of companies hiring "data scientists" are really just hiring SQL jockeys.

Personally I've done both data science and data infrastructure and I like infrastructure a lot more anyway. And it sounds like you are somewhat qualified for that with some of your pipeline work (although big data experience is also important). A LOT of data science departments have no idea what type of business value they are supposed to be adding, are doing shitty boring work with glorified titles, or are improperly integrated with the company at large (bad productionizing processes, poor data infrastructure). There's always going to be a need for data infrastructure but the "data science" hype is going to fade once all the shitty data departments cut the fat.

Have you tried paid services/consulting arms of software or cloud companies? Teams that bill customers at an hourly rate? They generally look for generalists who can help customers tackle problems at different levels of the stack. They aren't looking for PhDs in statistics.

When I interview people who have your type of background, I tend to get confused by what exactly it is the person wants to do (Analyze Data? Build an Analytics Pipeline/Architecture? Write Software/Services? Be an Analytics IT person?). Sometimes they have to talk about all the "cool" stuff they've done and lose focus on what they bring to the role. I also become skeptical because it's really easy nowadays to follow a few tutorials on 50 different things and then boast about how you did it all yourself.

Even reading your comment, you don't sound like somebody who wants to analyze data.

As far as being a generalist, I definitely agree that it's good to have somebody with skills in ETL, analyzing data, and maybe building a software service. But what happens is that all those things happen at different speeds and then people get crushed. You're asked to investigate some data quickly over 3 days, but suddenly the software service you built is having issues and you need 2 weeks to dig in and fix it, and also your ETL job is overloading the server you need to fix it yesterday but you need help from a somebody else to figure it out. Oh, and that Metabase thing you installed is broken and the VP was using it and has a big demo tomorrow.

I don't see this as a bad thing - it's a lifecycle thing. I absolutely would want someone like the GP to start my data team from day one - there is a lot to build and much to hang together.

When I am up and running I don't want yet another generalist - or rather I will happily take one, I just will put them in a box making pins.

Perhaps the GP will do better at the consulting level - or even some level of productise consulting - and out of the box product

> I tend to get confused by what exactly it is the person wants to do (Analyze Data? Build an Analytics Pipeline/Architecture? Write Software/Services?

But this is the crux of the job-seeker's dilemma. If he/she is specific about their interests when speaking to an interviewer, they might get a response like "well, we're really looking for someone whose operational focus is [something else]".

And if they're not super-specific (I doubt anyone does data analysis exclusively without any other involvement in the project), but instead attempt to give examples where they had demonstrable impact working across a number of domains, you might hear a response like this:

> Even reading your comment, you don't sound like somebody who wants to analyze data.

I don't think a person has to be super specific or say "I am interested in X and Y." What they do need is consistency in a resume so the reviewer/interviewer can evaluate what their primary and secondary focus areas are. Sometimes you have to leave stuff out. For example, Data Scientists typically don't deploy and maintain a Data Science stack unless it's a really small company. And a Data Science Infrastructure person at a bigger company probably isn't analyzing data unless they are just playing around to validate their stack.

But if you have a resume (or say this during an interview) that gives equal weight to the data analysis and the stack deployment, it's just confusing to the person reading it. Especially in Data Science, which already confusing from a skillset perspective. Lots of resumes look like the applicants just thought 10 things with minimal overlap were cool and decided to put them on their resume.

Even if you did work at a 5 person startup and had the unoffical title of "Data Scientist, Data Engineer, Data DevOps, DB Admin, and Chief Data Officer" I'd recommend you downplay some of those based on the jobs you are applying for. Figure out what is essential and what is +1

>what exactly it is the person wants to do (Analyze Data? Build an Analytics Pipeline/Architecture? Write Software/Services? Be an Analytics IT person?).

But you realize that the vast majority of businesses in North America need someone to solve all of those problems. They aren't going to hire and cna't afford an experienced data team of specialists.

This is the point of the article. Getting the 80% is far more valuable than having some PhD optimizing the hell out of features. Silicon Valley tends to overthink things.

>But you realize that the vast majority of businesses in North America need someone to solve all of those problems. They aren't going to hire and cna't afford an experienced data team of specialists.

The vast majority of business don't need Data Scientists. They need a BI person with SQL skills and some of the skills of a Database Admin. What most companies really need is a good set of Dashboards and clean data to feed it. This enables the business people to get the information/visibility they need an make decisions.

Also, most businesses should not be building analytic services and deploying them - they should be paying for a good product with a cloud or easy on-prem install and getting support from the company that sells the product. A few licenses of a good BI product are a lot cheaper than a Data Scientist.

I'm a hiring manager who shares this view. I see three distinct but overlapping skillsets with creating machine learning - as they mention in the article: data engineering, data science, ML engineering. I would never hire a person who only had one of those skillsets. I prefer all three, but can settle for two depending on the situation.

The trouble you might be having in getting an interview is probably partly to do with your background and likely also that those job postings get A LOT of submissions. Other hiring managers in my department as well as myself have found we get 10x more submissions for DS/ML positions than software dev positions. In general it's a really unrefined and new job skills that anyone and everyone who's taken a coursera course in regression or clustering will apply.

> I built a front end serverless analytics pipeline from scratch with AWS that handles 30M events/mo. I've demonstrably grown revenue and margins in multiple contexts with my data products

When I'm hiring I care most about wins. These are wins. When I read 30M events / month I want to hear more.

The rest is fluff IMO and things that I'd expect you to play around with while you're self learning. Also, most hiring managers don't care about what you did 10 years ago in sales if you're applying for a data science role. It might be icing on the cake you can share if you get into a conversation about sales or marketing, but otherwise it can feel off topic.

I'd slim down your resume to focus on these two wins (plus any recent experience building or leading teams) and stay later focused on recent data science related work in production. That sounds like a good enough resume to get an interview at most companies. Good luck!

>>When I'm hiring I care most about wins.

Losses can be just as important, IME. “I tried to do X, tried several methods, each one failed due to...” would be an interesting conversation to have with an interviewee.

Sounds like full stack data visualisation, rather than data science. You’re an engineer applying for science jobs.

Check out the series by Jeff Leek, Brian Caffo and Roger D. Peng, “A Crash Course in Data Science.”

Hope this helps, I can be pretty clueless sometimes so you probably already know all the mathy bits.

I think you've got it flipped. A CS/math degree, and/or STEM graduate work, are very strong indicators of generalist skills and exposure to breadth. And graduate work is a strong signal about someone's ability to learn and deal with exploratory/unknown problems. Whereas the things you've listed are actually more specialized.

This certainly doesn't excuse hiring managers from lazily filtering out candidates who don't have these things. But these are strong signals.

Depends where you're applying, right? If you don't know what hyper-parameters, regularization, or cross-validation are and how they affect your work, then some jobs just won't make sense until you're able to talk about those coherently.

The skills you have are useful. I know we wouldn't hire you in for a data science role on my team. I could imagine many other places they're useful, though, so perhaps it's the places you're searching.

The skills you described are those of a data scientist/engineer hybrid. Have you tried clearly branding yourself as a data engineer? There’s a lot fewer of those, and the job is overlapping to the nearest understanding of a hiring manager or non tech person.

Please post some contact information. No idea what your geography is but I am very interested in chatting with you about some roles that I have that may be very well aligned to the skills and interests you are describing.

Are you assuming people with technical degrees don't have these same skills?

You can also train entry level for a good amount less too.

You should put some contact info in your bio

If you're interested in relocating to Amsterdam, send an email to earl at apolloagriculture dot com.

Have you tried data engineering jobs rather than data scientist?

Can you create a public demo site to showcase your multi-disciplinary skill set? Perhaps make it specific to sales/marketing, where you have domain knowledge.

How did you teach yourself all that?

You sound like you'd be a great data engineer.

It's not you, it's capitalism. There just aren't enough (well paying, decent) jobs to go around. I'm in the same boat. Welcome aboard matey.

We’re hiring.

Email me: mark at dotscience dot com

Send me your CV. If you're legit and resourceful enough to get in touch with me, I'll get you a job or tell you how to get one at least.

Reply on news.ycombinator.com