The Ph.D.-Industry Gap
chronicle.com
chronicle.com
Poor coding skill accounts for perhaps 80% of the industry no-hires. Well...that's not exactly fair. Poor coding skill under pressure. Most Ph.D. candidates are either out of practice or used to taking a long time to think about and solve a coding problem. That's great for academic software, but misaligns with both rapid development and (importantly for the candidates) the style of interviews. Most Ph.D.s wither under the expectation of producing a feature very quickly.
Some Ph.D. engineers can code extremely well. If they don't end up at the company, it's because the following dialogue happened, implicitly or explicitly:
PhD - Look, I'm great.
Company - Yes, you are. How about $X?
PhD - Given my specialized skill, I was expecting more like $1.5X.
Company - Well, $.8X is what we pay the new grads, so $X is what we'll pay you.
In other words, their ability is clear, but their market value isn't enhanced very much by their specialization, when most companies are looking for the skills that developers get during their bachelor's or master's.
Perhaps they think they are getting hired to do the later and you are hiring them to do the former ?
Well, what did they get it to become? I bet most of them don't even know. I know I contemplated a graduate degree, and if I had done it, it would have been for love of the material and love of learning, not because I knew precisely what I was going to do with it.
Back in the day, so was doing an undergraduate degree. If you wanted a job in industry, that's what you did. University was for academic pursuits.
> "It's reasonable to assume that a PhD has lowered themselves to work in industry because they couldn't get a post-doc. Whether that's true or not it starts out the interview on a bad footing."
It's not reasonable to assume that at all. Last time I checked, the majority of PhDs leave academia after their degree (they don't want a post-doc - so don't even try). I do understand why non-PhDs in industry may harbour false impressions about the degree and why that can set things off on a bad footing.
> "What if a guy walked into your company and said 'I couldn't get a job at Google so I thought I'd try you schmucks'?"
This is completely independent of whether or not someone has a PhD. Undergrads can do this too.
Let's conduct a poll. Other than you, did anyone here who's been involved with PhD holders or grad-school and academia themselves actually think this? How many of us actual academicy types actually think it's "lowering oneself" to work in industry?
I don't. I do think that both academia and industry need to drop their utterly ridiculous notions of professional passion, because most people most of the time will never work on Their Life's Defining Passion when there are bills to pay, but industry can certainly come up with problems just as big, just as hard, and just as worthy as academia.
http://matt-welsh.blogspot.com/2010/11/why-im-leaving-harvar...
It's certainly something I could have taught myself over the years, but at the time and now it makes sense to me (i.e. sunk costs are irrelevant, only a little bit left to finish).
In general it falls into the same category as other colorful ways to reduce a role either for the purposes of humor or simplicity, such as 'keyboard warrior' or 'desk jockey'
I don't know if a PhD makes you a better writer of code, but it definitely makes you more determined, focused and unflappable person. (That's pretty much all it does, actually. Everything else is a stereotype.) These are all qualities that are worth buying in an actual engineer.
While here we are talking about a profession (software engineering) that does value general problem-solving ability except it also requires additional industry-specific expert knowledge.
Haven't worked with any plumber I take it? A plumber does quite a bit of general problem solving.
My brother was "just" a welder and could look at the blueprints produced and approved by experienced architects and engineers and tell them exactly why their designs would fail under load and how to fix them on the fly without massive cost overruns.
I urge you to reconsider your preconceptions about people working in "non-intellectual" professions.
A surgeon has a backed up sink, and he calls a plumber. The plumber comes, works for 15 minutes, fixes the drain, and gives the surgeon his bill.
"Six hundred dollars?!?," the surgeon exclaims. "For fifteen minutes? I'm a surgeon and even I don't make that kind of money!"
"Yeah," replies the plumber. "I didn't make that kind of money when I was a surgeon, either."
(1) From your observations, what is the rate of poor coding skills among PhD vs non-PhDs? (ie. the PhD may still be a partial signal for better (or worse?) ability.)
(2) A PhD has 4-6 years of experience over a "new grad"... so 0.8X vs 1.0X seems off to me too. Most companies I've spoken w/ allow each year of a PhD to account for 0.5-1.0 years of industry experience depending on the individual's skillsets and the job's responsibilities.
A PhD would (likely) only have done development-programming and testing -- and only (at best) about half-time, since so much of their other responsibilities are coursework, writing, teaching, etc.
Really? I mean, anecdotal evidence and all, but I haven't found that PhDs can actually work more independently in a commercial software setting than non-PhDs. I think it's because academic settings are just too different from commercial settings. In academia, your project is your own. You're responsible for it, from start to finish. But in commercial software, your project is often just one component or subcomponent of a much larger system. This requires a much greater level of communication and cooperation with other developers and business-people. So yes, a PhD may be able to work more independently on a project that allows for independent work. But most commercial software development is a highly cooperative activity, where the sort of independence that PhD holders exercise can often turn into being a loose cannon.
Also, it is hard to value PhDs if you aren't one yourself; e.g. it took Page, Brin, and Schmidt to figure put how to leverage PhD talent to build what was basically a new industry. Other people would see PhDs as seeming "very Mickey-mouse compared to real world problems." They wouldn't have the creativity and experience to see the connections.
There are reasons for that. It's just that the research grade code IS different. And labs tend to have their own cultures and approaches. Sometimes to the extreme of writing in their own languages. But in some cases it can be very advantageous.
If you're writing code that needs to be maintained (i.e. not just run a few times to generate results for a paper, but read and written by several different people over time) this is pretty bad and leads to much more technical debt than would otherwise be accumulated.
This is the type of thing you learn to avoid by working in industry a few years and is why experienced ICs are paid more than new grads.
Her coding skills were fine, but lacked some things very common in industry experience, like relational databases, Object Relational Mapping frameworks, distributed version control systems, and continuous integration tools.
None of this has been a problem. She has only been on our team a couple of weeks, but there was no delay to her being able to contribute to our code base relative to any other developers we have hired.
Someone may argue about this but people with less advanced degree tend to obey more to management.
By and large, hiring managers will see a PhD and be like "yay, give them an interview." Its the interview with an engineer where things break apart, because the engineer expects a PhD to be solving things the same way he would despite coming from totally different backgrounds.
(This is born out in the OP's experience, as he got many interviews but few offers.)
For once, hiring managers aren't to blame.
Consider we produce twice as many PHDs as the business sector requires. This is probably accurate. From a humane perspective, what should we do with half the grads, soylent green seems a little harsh... The social perspective is they should just sit on an ice floe as it floats away in the ocean, which seems wrong.
We're moving toward a class based neo-feudalist system. The first time around with feudalism, the bottom of the barrel of knights and barons and such got wiped out in more or less continuous low-level warfare. In our new neofeudal era we don't have a solution for the bottom of the barrel of our various classes of titled nobility, leading to all kinds of angst.
To mix some metaphors it seems a waste of talent and economic costs to euthanize the bottom half of the modern Baronry after all that education and accomplishment.
Although it would screw around with modern feudalistic class based ideas, maybe we should be able to arbitrarily officially reject titles of nobility. Better to have a lower 50%-ile PHD writing CRUD apps as a code monkey than turn them into hamburger.
I have a PhD, and by now I think it's safe to say that I'm a productive programmer, but it was hard to get (back) into the industry. I've experienced the bias against PhD-holding coders firsthand.
The interesting thing is that I've now also been on the interviewer side of this equation more than a few times, and I know that a lot of programmers have a huge inferiority complex that really gets revved up when they hear that a candidate has a PhD. I've seen the hiring bar shift dramatically higher for a candidate simply because interviewers on the loop knew about the advanced degree.
Given that most companies have virtually no interviewer training, and even fewer checks or balances on the attitudes and predispositions of their interviewers, I suspect that this latter problem is a far more parsimonious explanation for the hiring gap for PhDs than is the presumption that >80% of doctoral graduates can't write code.
Combined with the terrible quality of candidates I've had apply, ">80% can't code" is probably valid over a random sample.
If you add in some confirmation bias you'd probably explain a lot. However, I'd also add that I'm not shocked at the idea there are a lot of people who are great at academic research but terrible production coders.
The average applicant can't code, but that's true across the board. Yet there's a strong and systematic bias against people specifically with doctoral degrees. There's something else going on, and IMO, it isn't explained by chance.
"I'm not shocked at the idea there are a lot of people who are great at academic research but terrible production coders."
Why? Isn't this just a gentle way of saying that you have a bias?
There's no fundamental reason that you should expect that a PhD (especially in CS) would be less able to write code than any other randomly selected applicant. More likely is that most interviewers don't have any idea what goes on inside a PhD program, and bring some very weird, negative ideas to the interview room about what the degree implies.
That's exactly what I said, and was the point I was making. Combine this with confirmation bias and you're likely to see the result we do.
> Why? Isn't this just a gentle way of saying that you have a bias?
Academic research and production code have different, conflicting, requirements. That there are people good at one rather than the other is not surprising. I know people who are fantastic researchers, but their code should never go into a production system. I know people who write beautiful code, but they'd never have something good enough for academia.
> There's no fundamental reason that you should expect that a PhD (especially in CS) would be less able to write code than any other randomly selected applicant.
A PhD versus a coder straight out of university, or a PhD versus someone who has 3 years experience writing production code?
Have you ever actually produced any academic research code? Because my direct experience with the matter says that you're wrong. I've seen plenty of "academic projects" that had far wider distribution, bigger teams and more complexity than your average "production code" (most of which is never used by anyone).
"I know people who are fantastic researchers, but their code should never go into a production system. I know people who write beautiful code, but they'd never have something good enough for academia."
I know a great many people in industry who micro-optimize for beauty and who go off on architecture astronautic expeditions with no provocation, so what's your point? Are you generalizing to all academics? I can tell you for a fact that "academics" as a group don't care more or less about beautiful code than anyone else, and that research is just another kind of production system.
You pretty clearly have a bias that "academia" is this ivory tower place where nobody gets anything done, and that's a stereotype, not reality.
> Have you ever actually produced any academic research code?
Yes. I've also maintained other peoples academic research code.
> I've seen plenty of "academic projects" that had far wider distribution, bigger teams and more complexity than your average "production code" (most of which is never used by anyone).
And there are plenty of industrial projects with wider distribution and with bigger teams than a lot of academic projects.
> I know a great many people in industry who micro-optimize for beauty and who go off on architecture astronautic expeditions with no provocation, so what's your point?
There are bad people on both sides, and that being good at one side does not necessarily mean you're good at the other. I thought that was clear in my previous replies, but maybe not clear enough.
> You pretty clearly have a bias that "academia" is this ivory tower place where nobody gets anything done, and that's a stereotype, not reality.
Please quote me where I've said anything like that.
Your first reply to me was taking something I said and repeating my point back at me, to be honest it's just getting a bit tiring. So here's a change, I'm going to ask you this: Do you think that there are some people who are good researchers but bad coders?
> used to taking a long time to think about and solve a coding problem.
that's a good thing. A good friend of mine is a phd and lead programmer on a radio telescope project. my solution to a problem he described was quick to code and hellish to run, his involved maths I did not understand took five lines and ran super fast. he chose an algorithm that fit the problem domain better - because of more extensive training than I got.
Algorithms count massively. Google is built on an algorithm that fits the problem domain better. finding and applying better algorithms to give "unfair" competitive advantage is a very good thing
so I want to see a slow code movement like a slow food movement
- Producing higher quality software delivers more business value in the medium term than lower quality
- Iterating to a solution is often slower than applying the best algorithm directly
- it takes time to understand and evolve a story
- rewrites are a good thing. most books and film scripts are rewritten a lot
- daily status meetings encourage trying to do something each day instead of taking a week to make something fantastic
But just surprisingly, the market looks like it is full of those crappy startups, which value speed over quality, and don't do rewrites. Wonder why that is so.
How do you think it is that google is able to return your search results in a fraction of a second? And how is a game able to represent and efficiently display a massive 3D world?
The comment I replied to implies that demand is specified by the companies, not by customers. I guess you get that kind of attitude in university jobs, where money comes from goverment grants etc, not from customers.
At this point, employers quite rationally want someone who can put together a combination of basic algorithms in 5 minutes over someone who takes a week to come up with an a elegant jewel of an algorithm that no one else at the company can understand. To put it another way, just like the slow food movement hasn't really made a dent in fast food culture, I don't expect a slow code movement to make much of a dent in the "ship yesterday" culture of modern software development.
I remember the bad old days, before Ruby on Rails, when we were writing Perl cgi-scripts to connect to the database, or when 100% of the time I'd inherit a PHP application the people either: (I) had magic_quotes_on which caused everything to be FUBARed or (II) didn't consistently quote HTML in forms or in SQL, so you could easily bust forms by typing in a " or '.
Today the ideas in rails have spread to every other language, and it's better, but people still screw up database schemas every day.
Steve McConnell's book "Rapid Development" is really about efficient development, which, in the grand scheme of things, takes about 10-20% more time than truly "rapid" development where you consciously spend more to compress the schedule over efficient development.
Efficient development is a matter of getting things mostly right the first time rather than expensive "iterations" on a series of bad database schema designs when a system has customer data in it.
Frankly, I've never seen a software project fail because the developers were too careful. I see them fail all the time when people set an unrealistic schedule and run around like chickens with their heads cut off and wind up taking much longer than they would have taken if they'd really practiced project management and design and all the other things that the usual (failing) company fails to practice.
I would add, though, that a lot of people in industry mistakenly believe that what the PhD provides is specialization. It is true that you end up with a specialized skill, but you also end up with the ability to ask important questions, cut through all the things you could do and decide what you should do, experience managing long-term projects, proposing work, managing people (undergrads, usually), and mentoring, just to name a few.
So to industry folks out there who get occupationally sucpicious when interviewing a PhD ("just to make sure, are you comfortable not continuing your current research?") try asking a few questions like, "besides your research topic, what do you think you've gotten out of the last 6 years that will help you at this job". Yeah that's a softball question for an interview, but it will help them show you all the other things besides some obscure academic topic that they've learned.
A BS in chemical engineering basically qualifies you to be a process engineer. Sure, you can get a R&D job, but it's going to be a lot harder than if you have Ph.D. And there's plenty of research jobs out there; it's basically the impetus for a company's success over its competition (again, the science-y industries). There's certain tasks I would not consider hiring anyone for unless they either had a Ph.D. or had somehow independently demonstrated the capability for novel, unique research and problem solving skills.
I know the article was about CS, but the title was "Ph.D. - Industry Gap" and there's many fields out there where I feel like a Ph.D. is almost essential for particular jobs in those fields.
Obviously I don't expect this to be a wildly popular sentiment...
The way I would put it, the things that CS PhD's typically end up working on in industry, are usually things that would require a CS PhD. E.g. almost all the cool computer vision stuff you see on HN is developed by people with PhD's or PhD students.
It's a grim, nearly bloody, literally life threatening fact of life, that doing such research is really tough and the main challenge of a Ph.D. The Ph.D. training is supposed to help people do such research, and it does. Still the research is tough. Without the Ph.D. training, such research is much more difficult.
Net, the idea that a Ph.D. is not needed for good research can be true only for really rare exceptions. Again, getting a Ph.D. is tough, yes, life threatening, and the main challenge is just the research.
Broadly there's another option: Now the richest area of the US is likely within 100 miles of the Washington Monument in DC. Why? Because the US Federal Government has a license to print money - literally! Simple.
My career started there, and that area just loves technical Ph.D. holders. I failed to understand just how much that area loves such Ph.D. holders and should not have left.
You can apply to US Civil Service and/or individual parts of the Federal Government or to the huge number of private companies that do work for the Federal Government.
The Washington Post used to be a fast way to get job ads, send a few resume copies, go on seven interviews, and get five offers (I did that once). Some routine Google searching should get you a lot of names and addresses, and sending 100 resumes should get you some good offers.
Doing the research necessary to write a dissertation really isn't that tough for somebody who is bright enough to tackle the problem. Its navigating the waters of academic politics and knowing the right people to be able to get into a research program that fits one's skills that is the toughest part. If this seems like it's familiar, it should be: academia has as much political and networking based "merit" as any place else. It just manifests differently there.
For your
> get into a research program that fits one's skills that is the toughest part.
I didn't bother with that and, instead, picked my own problem, did my own research, and submitted my results. My professors had very little idea what I was doing until I was done and submitted my work. Net, my "research program" was mine. Maybe in some fields, e.g., cell biology, can't do what I did and your description is correct.
But if pick a good school, then what I said is simple but correct. E.g., Johns Hopkins, where I got my Ph.D., then, and maybe still, has no official coursework requirement for a Ph.D. The official requirement for a dissertation was, and maybe still is, "an original contribution to knowledge worthy of publication". If there is any doubt, then the student can just submit the work for publication.
In my case, I took a problem I noticed in a course, did a pass by the library to confirm that there was no solution in the literature, worked up a rough solution, asked for a 'reading course' to attack the problem, got the course approved, and right away showed my rough solution. Two weeks later I had a nice, clean, solid solution. I walked my original work around campus to some world famous guys, and they saw my work as new. So my work was clearly publishable. So, in two weeks I'd met the official requirement for a dissertation. My work was not trivial: I found a new result comparable with the classic Whitney extension theorem and applied my result to solve the problem I'd started with but noticed that my work also solved a problem stated but not solved in a famous paper by Arrow, Hurwicz, and Uzawa. Poor Uzawa may still have not gotten his Nobel prize!
I didn't try to use that work as my dissertation, but the reading course credit filled out what I needed for a Master's. Also my research in that reading course put a halo on my head so that a Ph.D. was much more likely. Let me emphasize again, the key is in one word, research.
For my dissertation, I took a problem I'd brought to graduate school, done the research on it independently in my first summer, got the work approved as my dissertation, wrote and ran some illustrative software, wrote up my work, stood for an oral exam, and got my Ph.D. For the work of the reading course, I published that later in a good journal.
In most good research universities, a student who passes the qualifying exams and has some good, publishable work will have to do something really strange not to get a Ph.D.
Some good research can make the political mud wrestling disappear quickly.
The main reason for ABD is the challenge of doing some research that is publishable, "new, correct, and significant". If there is any doubt about the quality of the work, then just publish it.
The definition of a good algorithm is from J. Edmonds. He was a graduate student at U. Maryland, told the faculty to stuff it, went to NBS, did some research on graph theory, published it, and got a visit from U. Maryland saying that if he would take copies of his papers, put them in a stack, put a staple in a corner, and submit that as his dissertation, then he'd get his Ph.D. Again, the point is the research. The mud wrestling is nonsense easily wiped away by some good research.
edit: Incidentally, JHU is where I've gotten offers to continue. How did you like living around there?
When my wife and I were at Hopkins, there was some nice graduate student housing just east of campus, and I'd look into that.
For the neighborhood of the medical school campus, maybe it's okay, but I'd check to be sure.
0_o What Ph.D. program has no course requirements?
http://www.findaphd.com/search/phd.aspx?DID=8&Location=UK&TI...
I suspect there are more projects than this. The way it works is you like a particular lab, you contact their PI, agree on a topic, and work on it for 3-4 years. In Cambridge (where I work), the university enrols all research students in a "Research Skills" course/workshop, but that is it.
If they believe you are not up to snuff, you might be asked to complete an MPhil/MRes course first which does have a taught component. Apparently, if you fail this part, and in some circles, you will be seen as a PhD failure. Apparently.
When I got my Ph.D. at Johns Hopkins, the official statement was that there were no coursework requirements for a Ph.D.
At times the math department at Princeton has stated on their Web site that graduate students are expected to prepare for the qualifying exams on their own, that courses are introductions to research fields by experts in those fields, and no courses are offered for preparation for the qualifying exams. So, with no courses, pass the qualifying exams, do some research, and graduate.
How to do the research? If some of the courses help, fine. Still the main requirement is the research, just the research. It's all about the research. Can cover the main requirement in one word -- research. Did I mention that the main point is the research?
Sure, in high school and college, academic success is almost entirely about just courses, credits, and grades. For a Ph.D. at a good research university, the emphasis is elsewhere -- did I mention research?
One example that comes to mind is graphics: places like Pixar and ILM heavily hire PhDs for R&D work, developing rendering algorithms and infrastructure and the like. They do also hire non-PhDs, but without some equivalent research track record (e.g. a SIGGRAPH paper or two), usually not for R&D jobs.
I hadn't actually counted before, but if you go to http://graphics.pixar.com/research/people.html and look at the bios, 9 out of the 11 have a PhD, and there are more outside the formal research division.
There are many areas that are theoretically rigorous, of course. Machine-learning/AI, crypto/security, graphics, audio, video, etc, and unsurprisingly a PhD is quite valuable for programmers in those fields.
To use the example posed above: you cannot be designing chemical processes without a solid grasp of the theory. You cannot do your day to day job without applying the theoretical concepts of the field to daily problems. That's what makes a PhD valuable in that field.
In contrast, many software engineers have no idea about lambda calculus, set theory, type theory, etc. You can build a lot of software with just the crudest theoretical understanding that you can pick up from hacking on software. That's why I likened it to carpentry. A carpenter will have a crude theoretical understanding about wood, it's grain patterns and how it absorbs stains, etc, but he doesn't need a solid foundation in the science to effectively do his job.
If he were as good in academia as his rhetoric claims (building software that "revolutionized" a field) he should have no problems. He should not even need a job, as he ought to be able to just start something. He should have no shortage of strong ideas about what he could be doing.
Instead he is aimlessly searching for a job.
So, I have no choice but to disbelieve his rhetoric. He probably isn't particularly good at anything, and just stumbled through the PhD system. Well, surprise, that isn't worth much!
All his external awards point to someone who very probably can do any tech job the industry could throw at him, in contrast to the usual developer who just has a self-generated github profile and "Ruby ninja rockstar" on his resume.
One story I heard about a physics PhD candidate at UCSB (this isn't me, I wasn't in the physics department). A MechE prof he worked with stole his research and got a tenure track offer at another school, leaving him behind. He was living in his car for a while and was a paintball aficionado. One day he happened to pull up outside KITP just when some of the more senior professors were walking by. He, in a disheveled state, opened the car door and mounds of paintball ammo spilled out. Looks were exchanged, emails were sent and phone calls were made, and he was PNGed from the campus soon after.
In the corporate world, if adversity comes after you and you don't defeat it, you'll be out of work in a few days. You start to learn the signs. In the academic world, your career just stalls out one day leaving you with no idea why, and the causes were probably decisions you made years ago.
So it's quite possible, if not common, for very talented people to have no idea what the fuck to do for themselves out on the battlefield. That's how they end up aimless and clueless at ~27-30 despite strong technical talent.
You get out of academia what you put into it. If you go in with no plan of action or just wanting to be like your mentor/advisor, or solely dependent on his funding or the department's TAship still thinking this is like undergrad, but only you get paid for it, you will be in for a rude awakening.
The signs do take longer, maybe one or two years, but they are there. Your prospective advisor has no pull in the department, or can't twist elbows and get a RAship for you. Or he puts you on a project to reproduce someone else's experiment with unreplicable results. You start to notice his papers with foreign colleagues have serious mistakes that get through peer review. You wonder why he hasn't been properly funded in years.
I managed to get out alive, with a degree and a good publication on the way (with people at another institution), despite one adviser leaving the university nine months after I joined and everyone else being broke. How did I do this? I came in with a overarching research goal (learn how new techniques in quantum information could be used to solve chemical problems), one that encompassed more just my advisor's research area. I executed on the overall goal and when things got harder, I doubled down even more. I didn't let departmental barriers stop me from learning what I knew I needed (quantum field theory for condensed matter) to solve problems, even if those courses weren't in my department and even if the connection wasn't immediately apparent. I sought out conferences on my own, paid for them myself, networked my way into summer visiting positions and invited talks, because nobody was sitting there and spoon-feeding me little bite sized research topics. In fact, I lived in my office for three months when I couldn't afford rent anymore and no one gave a shit. I am reasonably sure people would probably have taken 3-4 months to wonder where I had gone to had I jumped off a bridge.
My central point here is that many people in grad school don't recognize that your adviser's interests and incentives are not necessarily your own (just like your boss's). Unlike your boss, your adviser will never be held responsible for bad management. If your _department's_ incentives and interests become unmoored from your own -- well, then in the words of Mr. Garrison it's a "big lowercase t for time to leave"
The sooner you recognize that and find a happy medium between your needs and theirs, the sooner you can be productive for all concerned. Every contingency measure I took in terms of classes, planning, networking in terms of the overall goal, all were leveraged and all were necessary to bring things to a reasonable close.
I know this was posted here a while back, but this really is the best description I've read (he finished, while I left with a Master's and may go back elsewhere but the hustle necessary was much the same.) http://www.pgbovine.net/PhD-memoir.htm
So you have a situation in which we're training a lot more PhDs, so mean research group size is growing dramatically, but we're offering fewer of the sort of academic jobs that can allow them to excel in their trained discipline. As these folks then leave academia, they're very often leaving for jobs outside their field. This is a really unfortunate loss of the incredible taxpayer investment in their education that we should all be supporting, not discouraging.
[1] http://www.aaas.org/spp/rd/fy2014/fedperf14p.jpg
[2] http://www.nsf.gov/statistics/infbrief/nsf10308/tab3.gif
[3] (embarassing wiki link, since labor statistics are a nightmare to sort though) http://en.wikipedia.org/wiki/Tenure_(academic)#From_1972_to_...
Also FWIW the part about "dropped out of PhD a plus" is intentionally taken out of context. The full context from the link
BS/MS in Computer Science or equivalent (PhD or dropped out of PhD a plus)
Yeah, but by definition of "great schools" and "doctorates", it's not all that likely.
I am not saying this guy is a slouch who got what he had coming to him, I don't know him. I am saying that no compelling argument was made that this article was anything more then conjecture.
It is impossible to convince a man of something when his salary rides on his not believing it. Your professors have certain incentives to hold particular beliefs about the value of a phD.
There is no polite way to say that, which must be why it was not included in the essay.
You need to have students to have grants. But the reason you gets grants is to pay your students. So, anticipating that this issue may come up, I don't think that's it, either.
Further, I encountered a fairly high proportion of people in grad school who had, for one reason or another, difficult personalities (heck, this probably applies to myself). How do we know he isn't bombing the interviews for such a reason? We don't.
That said, it sounds like his graduate work was performed in an area that's irrelevant to industry. Despite acknowledging this, it sounds like he believes that this work alone (and the academic reward it received) should qualify him to work. He needs to realize that many employers don't think this way. They don't make the leap "He did X, so certainly he can do Y for us". That's just the way it is, and the sooner he realizes that and refocuses his attention towards the employers needs, (rather than his prior solution to an unrelated need), the better off he'll be.
Part of the problem is simply that the market is "thin". There are less jobs, and less PhD's than there are people with Masters or Bachelors in CS. So the matching process takes longer. People won't want to hire you for a job you are overqualified for, and in the long run this would be bad for both parties.
As to the people who think that professors are just over-confident so they can keep churning out PhD's, I think this is wrong for two reasons. First, the situation is subtle, and hard to understand for someone who isn't actually going through it. Second, the market is not bad, it's just very peculiar: as you can see, the author did get a good job in the end, it just took a lot of time.
I don't agree with it, for what it's worth. I do understand the rough justification that academic work is very different than real-world coding, but even if you accept that reasoning 100% then at the very worst a PhD should be seen as a neutral to slightly positive indicator (at least they see things through) if the person is otherwise a fantastic fit, not a negative. However, in my experience this negative bias towards PhDs in the greater software development sphere is a real phenomenon.
(That was intended to be a joke, but it's also a surprisingly accurate description ;-).)
If you're a serious candidate, it's hard to imagine a phone screening where you don't ask a few questions to feel out that sort of thing.
If you're totally unsure for a given interview, a good rule of thumb is to ask about the expected dress up front and then dress a little bit better than what the answer is.
This is where "And (2) my Ph.D. background, while impressive, just didn’t fit the profile of a data scientist (whose background is usually in machine learning or statistics), a product manager (Ph.D.’s couldn’t even apply for Google’s Associate Product Manager Program until recently), or a programmer (my experience writing code at a university, even on a product with 47,000 unique downloads, didn’t count as coding “experience”)."
I don't know about the other profiles, but programmers with only academia experience rarely code better than complete juniors. I've recently helped a team of Stanford grads get to grips with coding in an industry environment and the biggest surprise seemed to be that just because it works it doesn't mean it's done. You have to make it supportable, make it workable-on by others etc. etc.
One time I got a C code from one of the leading lights in machine learning, compiled it, ran it, and it segfaulted right away.
I started up the debugger, set a breakpoint in main() and it didn't even get that far.
I looked at the code and saw that he was statically allocating a 4GB array that he never used. I guess this didn't cause a problem on the 64 bit machine he was using, but it caused the code to blow up on 32 bit machines. I removed the array and it worked OK.
I was lucky to be programming in Java while it was still in beta when I was working on my PhD. One thing I noticed was that even the more computer savvy physics professors didn't get object orientation or any of the architectural concepts around it at all, so I was often called a "bad programmer" and told I should be more like somebody who cut his teeth on FORTRAN.
In my experience as a PhD with (currently) almost 4 yrs postdoc experience, I rarely get an interview when I apply for an industry position. I think there is a good element of truth in what the original author is saying about a gap. As PhD students and postdocs we do a lot of things. We have to write code, we have to manage students, teach, publish, apply for grants, manage research groups or entire projects, handle scientific and financial reporting, etc. The problem is that our skills don't exactly match company requirements that normally have different people for most of these tasks. To make things worse, people tend to believe that we will be arrogant pricks who will demand really high salaries even though we are clueless about how things work outside academia. Just look at some of the comments here.
For an entry level programming position a recent CS BSc appears much more desirable and cheaper than any PhD. I doubt that we even get much of a chance to show how good (or bad) we are. For higher level positions a PhD rarely has the "right" experience. What we are left with is usually something that fits well with our research. This is what the author found and as far as I know he is not in the minority.
So I say, f*ck it, and keep doing what I know how to do best: teach, write and build tools for teaching and writing.
To all chemistry and biology PhDs out there, please get in touch with me if you want to write a short textbook to introduce UGRADs to your field. Together, we can partner and take over the textbook industry.
Companies hire people to make a specific problem go away. If you're not a "make a specific problem go away" guy, then you need to find another way in.
I'm a fan of, rather than responding to job calls, identifying what companies you want to work for, deciding what you want to do for them, then going and meeting the people that can put you in the position to do those things. If you can't meet them directly, find people that can introduce you. Make a plan and give it about one year (these things take time).
I'm going to start looking into some of the R&D labs out there.
And amazingly, they told me that many Ph.D. candidates who come through cannot write a single line of code. I don't understand how that's possible but I would be wary too, in their shoes.
Real world software production needs little "computer science" expertise. There's too many Ph.D.s that take a research angle on projects. The goal is to design/implement/test/ship/support/improve and make money.
Best programmer I ever met didn't graduate high school.
Supply and demand determine wages. Ph.D. does not entitle to more pay. The industry simply does not rely on credentials like, say , the medical industry which does require an M.D. for the best paying "line" jobs.
They tend to want to rewrite things that work but aren't very elegant, or they become despondent over the language we're using and the reluctance of management to allow them to use Haskell for their piece of our Java project.
That isn't specific to people with PhDs. Every programmer without job experience does that. Back when I worked on a Unix/C application for a company that hired new grads it was common to have code checked in without any error checking at all.
If the person is reasonably intelligent (and everyone who finishes a PhD is pretty intelligent) it takes all of about two weeks to break them of the homework coding style.
Let me guess... you don't have a high rate of acceptance when you extend offers.
This reeks of entitlement and, well, whining. When I first entered the SofEng job market after undergrad, I stopped counting rejections once the number passed 30. It took another 6 weeks after that, and the rejection count must have pushed 100.
Should I write a navel-gazing blog post too?
The PhDs that I've interviewed have applied for very senior positions, yet not brought all of the skills needed. Which is more than jusr abstract thought. You need to turn out code that is maintainable and readable by the team. You need to ruthlessly reduce the problem and algorithms to a level that allows you to launch before you run out of money. Far more than being a thinker or researcher, you need to be an engineer. You need to be very, very productive. And so on. I'm not sure what the answer is, but there is a pretty big gulf. You look at the projects and okay, it's somewhat interesting, but impractical. You ask them how they'd solve a problem that you are trying to hire them to solve, and you get "I'd conduct a study..." when we have 6 months to market and your study would take a year, and probably not provide definitive results. It's just a different mind set. I can't afford to bring you into a senior role and wait while you learn to be an engineer, while you learn to code at a higher level than 'University code' (a disparaging quote about the quality of the code Sergy and Larry brought to Google) and so on. We have a real, hard problem, and need an extremely focused, skilled, multidimensional person to perform it. I've yet to come across a PhD that made me think they could do it.
"I can learn to do that". Well, sure. Anybody can, it's just that so few do. Only a small handful of people actually produce at a high level in the ways I'm talking about. I don't want to bet on you being one of the few.
So I am actually a PhD student in physics, but I've been in love with tech for as long as I remember. I am actually doing a master's degree in CS along the way, both because I'm incredibly interested and because I thought that would really give me an edge when I want to finally go into the tech sector. (i'd be able to say more than "I have the technical chops and have done well throughout my academic career.")
This article troubles me though. Am I going to be faced with an employment brick wall in a few years? I have a lot of friends in tech, and they are very supportive of me joining. Will my PhD actually stifle my ability to get jobs I want? I haven't seen very convincing articles about it, but if there are Stanford CS PhD students having difficulties, what's going on?
I plan to do an industry internship next summer, and I hope that will at least help. What if I want to go into product management though? I have a google interview coming up in two weeks for a software engineering role, but I was shut down for even an interview for their APM roles. Maybe I needed more tech experience? Would this summer internship be enough? It's just unclear and troubling.
E.g. I interviewed a physicist for a data science position. While he had worked on a lot of problems which could be categorized as data science problems, 95% of the problems he was working had one specific tool that he used, Principal Component Analysis. It was kind of hard to get him to think of another ways of solving a problem.
In actual fact these days, there simply isn't a career model set up to support that anymore. Your PhD thesis is now supposed to qualify you for your first postdoc, your first postdoc or two qualify you for a permanent position, and you earn tenure or permanent contract in your permanent position, and then you can just focus on contributing to knowledge rather than on careerism.
Moreover, I don't think it's fair to say that people waste their time doing a PhD in physics. Sure, they could be learning a lot more job-specific skills, but the point is that the people who do a PhD in physics absolutely have the ability to learn those things. I'm sure if the physicist used PCA it wouldn't take him long to learn plenty of other ML techniques. I think the skills you learn during a PhD aren't as simple as just listing out the technical things you became familiar with. (Still, I'm certainly considering leaving grad school, but I don't think it's an obvious decision.)
Of course, I think there is a lot more to a job than the technical skills you bring to the table --- your ability to communicate, your understanding of the market & product, and your ability to work well with others. (Yes, I'm fully aware physics people don't often hold gold medals in these categories). I think that doing an internship could help in these respects, and it's why I'm interested in doing one.
I sympathize with your situation as someone who accidentally got into computer science. As much as I have tried to portray myself as a fast learner or someone who can learn on the job, a secret in Industry that I have learnt is that very few people care. Most managers are looking for someone who can fit a resource need yesterday. To illustrate: the other day I was talking to a recruiter from BigCo, the recruiter strongly suggested that I write code on the whiteboard in an actual programming language rather than pseudo-code. This seemed strange because BigCo is known for their algorithm intensive interviews. Then, I was talking to a friend who works for BigCo2 and it all made sense: if there are two candidates A and B, of whom you can hire only one and both A and B solve the algorithm but B appears to be able to write the code with very few mistakes on the board, you are probably going to hire B mainly because B appears to fit the hole in your organization. Now, as much as people like to protest they are not like that, companies are hierarchies of managers, managers who have concerns and reports to file. The physicist who used PCA would probably have been able to learn everything else in 6 months. Who is going to spend six months paying him a shit ton of money while he learns or maybe doesn't? I am not saying everyone who gets hired is hired with an expectation of being productive from day 1 but you have to be productive sooner than later.
Ultimately what am I saying? If you are thinking of going into industry; get real. Get all the skills necessary, your resume in order. There will be millions of blog posts bemoaning the disconnect between industry and PhD or how the interview process is broken. However, all that fluffy bullshit is not going to bring a paycheck down the line. What is going to get you interviews is having "good internships", code on Github. What is going to get you a job is the ability to convince a collection of random people that you are worthy of getting a job. Sure, the stochastic nature of the game means there will be some moron who thinks that him dropping out of high school and spending the last five years writing spaghetti code for industry makes him superior to you. Or you may have an attitude that you spending five years writing research papers makes you better. Ultimately, all that doesn't matter.
What I don't understand is why you are applying to Google if you're doing a Physics PhD. Why become a doctor of Physics and then throw it away trying to apply for jobs that have zero overlap? You're not trained for the job, and I strongly suspect this is the problem the OP was having. Why aren't you applying to NASA or SpaceX or something?
I'm not applying to NASA or SpaceX because I'm really interested in consumer tech, and always have been. Honestly, maybe I have the total wrong idea for what I want to do, and I understand that. That's again why I want to do an internship.
I actually have a Google interview in a couple of weeks. If you don't mind me asking, did you work at Google as a researcher or software engineer?
Matt Welsh has written a number of times about his job change from Harvard Prof to Google, and he's a software engineer by title AFAIK, but he's clearly doing research work.
As a someone who is 5 years out of his PhD (working at a research lab), I'm a bit perplexed at how I can make up for the lost years of industry experience. The best idea I have now is to start my own company.
I did do an industry internship toward the end of my PhD and I think it made a big difference. Not only did I have the experience to show potential employers that I was serious about going into industry, but I also learned more about software engineering in 3 months of my internship than 5 years of academic coding during my PhD. And I ended up with an offer from the company where I did the internship.
I actually think it's an even better idea to do a software engineering internship early in your PhD if you're doing some sort of computational science, because you'll pick up skills that nobody will teach you in grad school but that are also valuable for writing academic code.
As you know, industry internships are way outside the norm for physics, but they're pretty standard for computer science grad students. In my view, not doing an internship was this guy's first mistake, although I have no idea what to make of the fact that he had no offers after 30+ interviews, which does strike me as highly anomalous.
It also strikes me as odd that this person had no offers after that many interviews, but the truth is that there are a lot of other factors that go into these decisions. Maybe this person wasn't the best at communicating? Maybe this person sold himself as an all-around generalist, and the companies didn't really know what to do with him?
Otherwise though, I'm happy to hear about physicists going into tech!
Back in the 80's and 90's there were a handful of academics thinking about quantitative finance and no programs oriented at creating that sort of person, so of course you could walk out of your no-future physics PhD and get a quant job easily.
Since then there has been a shake-out ad you've actually got to be world class at something in particular if you want a quant job.
Many "quants" are now people who do algorithmic trading on their own accounts, the way people did day trading in the 90's. This may sound glamorous but it is definitely a business that comes and goes. You need a LOT of capital to do it safely, and most people blow up at least one account in the process of becoming good at trading.
It sounds very weird that someone coming from a Stanford CS PhD -- a top tier PhD program in the heart of the Silicon Valley would not be able to find a job quickly...
At any rate, whatever the circumstances, from anecdotal experience, such a case is almost unheard of for a PhD in CS coming from a good program.
> "Companies hesitated to hire a Ph.D. with no industry experience"
That, for me, would be the #1 red flag. How does one even go through getting a PhD without getting any industrial experience? Not a single internship, even as an undergraduate?
This tells me that either the PhD literally can't code (and hence nobody even wanted them as an intern) or despises industry so much that they never wanted an internship (and hence I'd be pretty worried about how much they'd be committed to the new job).
Even given unlimited research funding, to not once try working in industry (even during undergrad) betrays a worrying single-mindedness which I'd want to avoid in any technical hire. If all you do is proceed down one path, without ever trying anything else or being a little creative, I probably don't want you on my team.
I moved from academics (freshly graduated computer science PhD) into software development four years ago and when I did I had to answer a lot of questions about why I wanted to make the move.
My experience is of course not universal but I think that career trajectory is a big part of senior interviews once it's been established that you can do the job. It's a broader issue than just academics vs industry. No matter what field you're in, you have to sell you your career transitions as 'make sensing' to the jobs you're applying to. Job searching from the company side is more complicated than accumulating experience and rewards (the Porsche in the original article) to end up being granted a job.
I wrote up some longer thoughts on this: http://www.davehking.com/2013/09/22/jumping-from-academics-i...
It doesn't seem to be an efficient usage of time to pursue a PhD for the express purpose of earning a higher salary upon graduation.
"I eventually realized that, like many Ph.D.’s in many other fields, I had fallen into the Ph.D.-industry gap—i.e., the gap between highly specialized Ph.D. training and corporate-world expectations..."
There seems to be a dissonance in these two statements. If the former is true (based on description of his experiences) then companies would rather have specialized people to fit into slots (red/blue/black) than have a generalist (chameleon) who could grow to match nuanced needs. It seems like the problem was that he was not specialized enough in the very narrow skill they were looking for. If that's the case, then it goes against the commonly held opinion expressed in the 2nd excerpt.
So it seems like the real reason he wasn't hired was: 1. They might have to pay him more, for his potential (as illustrated by his degree) than they needed to pay some other fellow they could hire. 2. The HR management doesn't really know what to do with smart broadly skilled people. They have well-defined slots for cogs, but generalists are tougher to manage... you can't just shove them into a predefined slot.
To me, al it proves is that the industry is not ready/willing to hire a possible generalists (who also happened to have proven that they can learn something and be good at it). The cognitive dissonance making them think that he was "highly specialized" might also have been a possible contributor.
I can definitely imagine a situation (and being a recruiter, have seen situations) where a PhD will keep you from getting your foot in the door. There can be concerns about the ability to actually write code, the ability to be practical, to be OK with working with unclean data, the ability to get stuff out the door and have a hacker mentality, etc etc.
However, these concerns tend to arise BEFORE starting the interview process. Once your foot is in the door, unless there's a huge disconnect between the people doing the hiring and the people doing the filtering or unless the job description changes midstream, how you do in interviews is more important than your background.
These all seem to be concerns of someone who does not what doing a PhD entails, really.
If he really wants a programming job he should get his hands dirty and publish some apps, either alone or meeting up with tech meetup or study group. A PhD is not studying how to code, you just may code a little as a by product. Then he could talk about launching and features and getting code done day after day instead of talking incessantly about education, and he'd land the job.
Kind of strange he wasn't able to place due to the old boys network, though. I know tons of people running startups who simply check their alma mater first and foremost when hiring. Especially schools like his.
Speaking from the grad-school/research perspective: we never think of that. Why? Because our entire training is in thinking about research-hard problems. From our perspective, anything you can get together in a Hackathon and crank out an "app" for is trivial and beneath consideration for someone proposing to do serious work.
And yes, I do realize exactly how this sounds. The best I can say in our defense is that we still face all the normal design, development, testing/QA, and marketing issues after we've solved a research problem.
The difference is subtle, but its a trap that many highly qualified people fall into. I'm under the impression that the author spent a lot of time talking about his PHD and what he knows rather than the problems at hand.
* Are you sure you are not overvaluing yourself? Jobs are in short supply, PHD looks like risk.
And the example with the Ferrari was awful - if the luxurious car was on sale with a marginally lower price it would sell. Faster, the lower the price.
I would like to hear from academic departments on why they said no the academic superstar from Stanford. I would like to hear from IBM and Microsoft on why they couldn't use him. That's what's missing.
Maybe it's entitlement? Some of that creeps into the article. Maybe it's a personality that just isn't agreeable? Some of that is in the article too.
You are getting beat by people who want it more and who don't feel entitled by having the PhD.
- i think that as an algorithm expert one will find places that do value a Ph.D
- as developer/coder you will have to consider that your prospective manager will feel discomfort at managing you (regardless of the applicants coding skills); A manager needs somebody who gets the job done, he does not need somebody smarter/more educated than his own person.
Small weekend project: What type of PhD student are you? https://bohr.typeform.com/to/PPzzY8
If a supervisor has a subordinate that knows more, then the subordinate can challenge and compete with the supervisor on technical grounds, look more qualified than the supervisor, and threaten the supervisor's job. Academics has solutions to this problem: An assistant professor can win a Nobel prize without damage to the career of the department chair. But industry has no solution.
More generally, a Ph.D. must report to another Ph.D. or to the CEO of the company. Put this fact into the connected, directed, acyclic graph of the hierarchy of the organization chart and see the consequences.
In particular, industry just will not, Not, NOT evaluate qualifications in a reasonable way. Instead, they tend to have a list of 'skills' and want to check them off -- Java, JavaScript, PHP, Python, Django, JSON, C++, Mongo, SQL Server, etc. If you can code heap sort or an AVL tree on a white board, okay.
This whole situation is part of a larger one: Basically the US economy for 'careers' as hired 'employees' is nearly dead. So, GE, GM, AT&T, etc. are no longer offering 'career paths'. Actually, a career needs to last about 40 years which means that, really, only rarely since the start of the Industrial Revolution did large, US corporations offer 'careers'.
The flip side of this situation is good news: The role of a good Ph.D. with some interest in business should be a company founder, get some 'traction', then take a walk down Sand Hill Road and collect some Series A checks. Use the Ph.D. and/or the corresponding learning/talent for the crucial, core 'secret sauce' of the business. Then type in the software and get the 'traction'. No one on Sand Hill Road will question just what the heck your qualifications were for typing in the software.
Then f'get about a 'job'. That is, if you want a job, then create one for yourself. An advantage here is that, just as a solid consequence of just what you have seen, the big organizations will be essentially helpless at doing work that duplicates or equals yours.
One short term option is to be a contract employee. That is, some company needs some SQL stored procedures or some such written, so you write them.
One 'academic' solution may be to get a job in a B-school that wants to have a good course or course sequence in 'information technology'. There your qualifications are fine.
The 'gap' is all on the side of industry, and it can be taken advantage of.
You can tell the EE department at Stanford that a EE Ph.D. can easily want to swap their degree for an electrician's license.
Why? Because the Ph.D. needs a large organization which is in international competition, and an electrician with a license has a geographical barrier to entry, can work in nearly any town of the US, can't be fired, and can make money enough to buy a house. For the EE Ph.D. in a large corporation, he can be fired after a few years (because he didn't get promoted into management -- only a small fraction of new hires do) and find that the guy he knew in high school mowing grass now has eight trucks and 20 employees and is doing better than the Ph.D. is.
As you get better, the jobs available to you get better, but the job searches get a lot harder.
PhD programs exacerbate the problem. You get better fast (faster than people do on typical early-20s grunt work) and by 27, you're already at that level of skill that starts to complicate job searches. (Again, jobs available are better; but 3-5 month searches are common.) In addition, you haven't learned the judgment-of-character skills that most people pick up in entry-level hell; whereas most people, by the time they get good enough that it starts to actually cost them job prospects, have learned those skills.
1. I didn't want to be an academic, and 2. Having a PhD wasn't going to help me get a non-academic job, and 3. Grad school kind of sucks.
But, as a result, I have a lot of friends who finished their PhDs, and the idea that they are metaphorically "Porsches" compared to the many talented non-PhD developers and data scientists I've worked with is laughable.
I found his story valuable, and even if it is just anecdotal, it's a useful anecdote. He deserves credit for sharing it, because it certainly doesn't help your career to have an "I wasn't wanted" stigma attached to it.
Don't forget that the article is part of a blog about PhD's placement experiences--it's not just idle carping. Reading the authors responses to the comments, he sounds like a stand-up guy.
Well... apparently you do.