Hacker News Reader

This comment reads as full of hubris. It is easy to say you can do what someone else does, but the fact is you haven't done it, don't have the evidence to show that you can, and are exasperated that others don't see in you what you see in yourself. Having a PhD in a relevant field is an objective signal - not proof - of capability.

When these jobs are hot and candidates are plentiful, using signals to narrow the field to a group you can more rigorously interview is typically a more effective use of time than buying into everyone's self-belief. Candidly, I find most individuals from a programming background vastly underestimate the skillset required in this space. I know that is not an uncommon perception and you are likely being penalized for it, fairly or unfairly. I'll say this: anyone who refers to data science as "just linear algebra and calculus" would be immediately removed from any candidate pool I was managing.

Others have evidence of capability, you do not. Programming experience is not evidence enough to elevate you above candidates with more reliable and relevant credentials. A shelf full of books is not evidence either. You either need to find a version of this job created by people that don't really know what it is they want (hint: if the Data Science JD says "Excel" that's an indicator, it's not too uncommon) to create a work history, find a way create a portfolio that you can use as evidence (e.g., Kaggle competitions, hobbyist projects with available datasets), or network with others in the industry and academia such that they will vouch for you.

This is all very sensible as long as it's a buyers' market. What I get frustrated by is a sellers' market where the buyers gripe about how there are not enough qualified candidates. At that point, employers should be figuring out how to identify and cultivate inexperienced potential. I thought I had sensed some of this "there aren't enough qualified candidates!" hype in data science (for instance, that's the vibe I got from the article we're discussing), but the first-hand reports from people on this thread are making me think that probably isn't the case; that is, that people in the field feel that they can be very choosey and still fill their roles. If so, it seems to me that this is all working as intended.

I don't think most organizations possess the ability to cultivate these skills. I know mine doesn't, and from the outside-looking-in we should be more likely than most.

> I can do everything you can do.

Sorry but how would you even know?

I understand your frustrations well since I don't have a BS.

Still, as much as you seem to want to talk about how capable you are, you can't seem to understand the perspective of employers. Given what you've said about your history I suspect this is not due to a lack of intelligence but empathy.

Employers have to go through many candidates, each of which has some true capability but of which the employer can only see some signals. Signals have varying degrees of quality, and interviewing candidates costs time and money. That being the case, it is only natural that they try to use the strongest signals they have.

Nobody believes that there are not capable people who do not have an MS or PhD as you seem to be suggesting. The reality is that the proportion of people who have a BS and can do the job is much less than the number with MS or PhDs, and so it's one of the more effective filters they have at their limited disposal.

I'm sure companies are not happy about skipping great candidates like yourself, but they have not figured out a way to do so that is scalable and cost efficient. It's a difficult problem but maybe you can figure it out.

> You are not special because you have a PhD and I don’t. The only difference between you and me is that I had the ability to learn for free what you paid for.

How much do you even know about PhD programs? It seems like not much because PhD candidates, at least in the US, get paid. It's not much but they certainly are not paying for their education.

I apologize for this unsolicited advice but your lack of humility is frankly very off putting. You sound like a very hard working person. PhD programs are extremely difficult to both be admitted into and to finish. I'd expect that you would respect others like yourself who are very hard working.

I commend you for writing this levelheaded reply. I was about to write one with essentially the same arguments but a much more aggressive tone.

I think you are seriously underestimating the value of first hand experience conducting scientific research. It beats into you a mindset you can't get anywhere else. There are no books or manuals or docs to read and master it. There's only the scientific method. You start with a question, conduct experiments, analyze results, adjust your hypothesis, and repeat the process. You get very comfortable with saying "I don't know". You right at the border of the unknown and trying to navigate further.

Companies are using a PhD as a proxy for having research experience because it's the the only qualification like it out there. It's a poor proxy because not all PhDs are created equal.

What sorts of positions are we actually talking about here? How many companies actually need a research lab? I don't think anyone contests that a PhD is very useful if your job is to actually write and publish papers, attend conferences, present at conferences, etc. But many data scientists and machine learning engineers working in industry don't do any of that, and many companies have no need to have anyone on staff doing any of that either.

> You start with a question, conduct experiments, analyze results, adjust your hypothesis, and repeat the process.

This is missing my pet step: doing the literature review.

I’m pretty ambidextrous when it comes to Python and R, so I’m not typically a combatant in the data science language flamewars.

But... for as much as the Python community likes to assert their superior coding chops, I’ve observed that the R community does a much better job of reading about prior art.

> This is missing my pet step: doing the literature review.

One of the earliest, most important, and most useful lessons I learned from a senior grad student: "a day in the library can be worth a week at the bench."

Or this: you spend 4 months on a project and belatedly notice schmidhuber proved a much better result in 1985 and yells at you and you get kicked out of academia

You don't get kicked out of academia for that... Well maybe if you're a student and you lose your funding. It's really embarrassing and I'm sure frustrating too. I've reviewed a couple of papers where they didn't do their literature review, there were lots of obvious (to people who read) prior work, and I've had to straight reject their papers. I feel bad for them but do your literature review folks!

I was kidding!!! But people do take literature review seriously (which on the balance is good)

Of course it's possible that there are competent data scientists without graduate degrees; the parent was simply suggesting possible reasons why you might have been passed over for those positions, such as positioning your previous positions as more engineering-focused and less research-focused. I doubt the intent was to "gatekeep" you, but to give you suggestions on how to improve your search in the future.

This is a grossly defensive overreaction to the parent reply.

I don't think anyone doubts that the skills can be learned outside of school or fancy degrees.

The problem is that there are hundreds of applicants in your situation WITHOUT experience. There are usually a couple of PHD or MS applications WITH experience for every job. Who do you think the company would give preference to?

You are input to a classification routine. The job of those designing the routine is to reduce the false positive rate to as close to 0 as they can without incurring too high a false negative rate.

So this sucks if you don't fit the model well in a way that has you often end up as a false negative - but that doesn't' mean the model is broken.

No. The model has a problem with overfitting. That’s my point.

Saying it doesn't make it true.

You are claiming there is a generalization problem that causes extra error in practice. Another perfectly viable hypothesis is that the classifier is working fine, it's just tuned for true positive rate and accepts a higher false negative rate to get it. Specificity vs. sensitivity is a fundamental trade off, not a training issue (though that can make both worse)

I feel you bro. I've been there and done that. The thing is is that they are completely happy losing out on seeing the few candidates that are mixed in among the whole pool of non-PhDs (who are 85%+ posers) when they can select among the PhDs (who are 85%+ legit). It's a simple risk aversion tactic among companies and has nada to do with you personally. Like the man says, "nothin' personal, it's just business".

I'm sure you're extremely smart and dedicated -- I've only dreamed of putting in that much work -- but have you considered your mindset might be holding you back? Would you hire someone who had never worked in low-level game programming and didn't have a bachelors but told you "I can do everything you can do"?

If you're serious about getting a PhD-level job without a PhD, getting someone to recommend and vouch for you is even more important than usual. Since you're up to date with papers, why not email researchers you admire with questions that demonstrate you deeply understand their work? Many will be too busy, but some will probably be impressed by your determination. Once you have a relationship, see if you can assist with their research, even if initially it's just grunt work. It will take time, but integrating yourself into the academic "web of trust" and maybe getting your name on some papers is the only plausible way you can expect a company that doesn't know you to take you seriously.

Data science is not a PhD level job. PhDs may be well suited for it, certainly, but 80% of data science can be done by peoplenwth bachelor's degree in stats / math-heavy science.

There's always a domain specificity that sometimes comes from grad school, but data is data and industries are filled with SMEs who understand the domain.

Source: I hire data scientists for Fortune 500 companies.

>You are not special because you have a PhD and I don’t.

>I can do everything you can do.

To demonstrate that to an employer wanting PhD workers, go get a PhD like the other PhDs did. Claiming you can do what they do when you haven't done what they have done is not going to cut it.

>you won’t even call me to talk to me that means you are missing out, and you are gatekeeping

Gatekeeping = not spending unnecessary money and wasting unnecessary time.

An employer saves significant money and time by not having to interview everyone claiming they can do what PhD can do but didn't bother to get one. Your skilled workers don't have to stop producing and do interviews, your HR people don't need to spend time and money booking flights, hotels, and such for candidates. You don't have to work through 500 resumes with 30 PhDs in the pool - you sift through 30 resumes.

If you're in NYC, Chicago, or Boston we'll give you a shot! Email me sandy.vanderbleek@publicismedia.com. Thanks.

Every job is gatekeeping. You chose a path in life and it wasn't data analysis. Why would anyone interview you when there are 100's of other people that have the actual skill-set they are looking for?

Sure. Unjust as it may seem though, requiring a PhD in science is a free and simple way for employers to narrow down the field of candidates to those who are good at data science. It's the data science equivalent of requiring a CS degree when looking for a programmer. It's natural for employers to use that tool.

But if data scientist is the sexiest job of the 21st century, and everyone's going to need one or more, there won't be enough PhD's to go around. Someone's going to have to "take a chance" on the lesser degreed or risk falling behind their competitors that have data scientists of some effectiveness.

There are not as many data scientist jobs as the hype makes it seem. Maybe this will change in the future but right now there is not such a lack of qualified candidates to justify taking chances on lesser degree candidates.

All this hype makes it so everyone wants to be a data scientist. You get people who change careers to go into this new hot career. You also have a pool of people who have been working with data well before the hype with experience. The people trying to break into data science will have a very hard time competing with the people with experience over the pool of jobs out there.

What value does an advanced degree, in your opinion, bring to a data science position then? To hear you say it, it brings no value.

What value does a bachelor’s degree have if some kid can just learn ruby and JavaScript on his own and go make $100k/yr at some startup? Of course a degree has value but it is possible to provide value without one. This is especially true in 2018 when anyone can learn pretty much anything online, and download whatever papers they want online and read them. Your mistake is in thinking that somehow a degree makes you special. It doesn’t. It just means you paid for a head start. Anyone can learn anything you already know and they can surpass your skill, whether or not you have a piece of paper certifying your knowledge.

You assume that everyone can teach themselves with the same efficiency and effectiveness as quality guided instruction from real teachers they can directly interact with. Some can, but most cannot. Education is not a waste.

Thats a pretty ignorant statement. I'd wager that there aren't many people who could/would put the same effort into self studying that would be required to pass a CS degree with decent grades. They also would probably not learn a lot of stuff that is not interesting to them, while CS students have no choice than to go through the materials. As an employer this also tells a story about who is taking the easy route vs working through a complete program over the span of years. Obviously there are exceptions, but when there are a lot of applicants for a position, it's just an easy filter for employers.

Prestige and artificially pumping salaries. It’s all about brand building.

I don’t have an advanced degree, just a bachelors of math from 1995, and have been breadboarding (and more), and coding since the 80s

I can follow along with ML and have implemented toys with the ML algorithms in a couple days.

It’s bourgeois intellectualism. Like a law firm only hiring from Harvard

ML is automated schema design. And the current methodology has known limits of applicability

This is “Mongo DB”, “devops” like hype all over again.

Well, to be more precise its like a law firm hiring only from those with law degrees.

Just FYI many (most?) people with STEM PhDs don't pay for it, but in fact get a stipend.

To be fair, I didn't pay for my PhD, but I gave up the opportunity to earn more than a grad student's stipend for six years.

You know PhD candidates have their tuition paid for and get a stipend. They don’t pay for it like you made it seem or seem to think.

Actually no, a lot of PhD's are a bit smarter than you, some PhD's are a lot smarter than you