The $30/hr Data Scientist

Grad students tend to be intelligent capable people (from privileged backgrounds too!). Therefore we don't expect grad students to be desparate, because we infer that they ahve a lot of other options.

The only sense in which grad students are desperate, is that they really want the other advantages that being a grad student confers, most of all the education and experience, and the degree they get at the end. And this was precisely the point of the post you replied to: graduate students are getting non-monetary compensation.

If you disagree, I challenge you to find one testimonial from a grad student whose story is that they were so desperate for money, that they had no choice but to enroll in grad school to make ends meet.

https://en.wikipedia.org/wiki/Apprenticeship#Analogs_at_univ...

It's in no way the sole predictor, but the probability of someone have done hard things and being able to solve tougher problems [1] quicker is higher with a PhD than someone who spent the same time in industry. It's also tremendously competitive (in the same way, say, Goldman Sachs is in "business") so you can trust the PhD admissions committee to have done some filtering for you. Finally, it takes a lot of willpower to stick through it til the end (particularly in the US with its much longer programs), and drive is a strong predictor of adding value to a company later.

On the downside, few people in business realise that because they haven't hung around PhDs or thought about it much, and just see a "lack of industry experience", an "attitude" and "not the most commercial mindset" (usually with a mention of the "ivory tower").

A good corporate job will have you learn a lot (broader, less specialist, probably less hard); but most corporate jobs are not good, and pay well to compensate the drop in intellectual challenge and loss of skills that results vs your undergraduate self (although you'll pick up more useful soft skills, to an extent, vs learning to navigate bureaucracies which are the soft skills a PhD teaches you). I heard that across practically all academic fields.

[1] Unfortunately most of the problems people actually need solved in tech companies, like building a data warehouse, some reporting, a user friendly front end, are not "tough problems" in the way, say, building a machine vision engine tailored to your specific industry might be.

In my case, I was interested in machine learning, and after doing some internships as an undergrad concluded that there was no easy path to work on interesting ML problems in industry with only a bachelor's degree (both because the positions weren't available, and because I didn't know enough to do the work). So for me the first-order benefit is that I get to spend my time learning about and working on stuff that I'm interested in, and developing a skill set and a credential that should let me keep working on interesting things for the foreseeable future. It's different for other people, of course: some want to become professors, some want knowledge for knowledge's sake, some could pursue their chosen area equally well in industry but have a specific crazy idea they want the freedom to explore (I think this is more common in systems research), some want the lifestyle flexibility to go skiing in the middle of the week and make up by cramming work on weekends, etc. For most people it's really a mix of these factors and others.

The other main bit of grad school advice I have is that your relationship with your advisor is extremely important. The chance to be mentored one-on-one for six years by an extremely bright person you click with, share interests with, and who wants you to succeed is incredibly valuable. On the flip side, the possibility of being stuck for six years under a boss who doesn't care about you or know how to motivate you and develop your skills can be a massive liability (and one that's harder to escape than the equivalent situation in industry, since a lot of people internalize the value structure of academia and think of quitting a PhD as a failure on the part of the student rather than the advisor). So whatever you can find out about your likely advisors -- both by talking directly to them when you visit (are they looking for new students? do they seem enthusiastic about you specifically? how do they help their students find research topics? what is their philosophy of how to develop a student's skills as an independent researcher?), and by talking to their students: take them out to coffee and ask pointed questions like "is X a good advisor?", "what do you like/dislike about working with X?", "what sort of person is best suited to working with X?", "would you make the same choice again?" -- should have a huge bearing on your decision whether to attend, even if your program doesn't actually make you choose an advisor initially.

The thing is, there's no such thing as an "average" developer. Every company and its tech stack has a unique set of conditions that will match better with some people than others. If your codebase is bog standard [insert popular framework], someone who has done it 4 times before, but couldn't make a relational schema because he doesn't know what an SQL constraint is and thinks functional programming is "academic ivory tower stuff" is going to be a lot more value adding upfront than someone with a PhD in category theory and 5 years AI experience; but the latter is the person you want building your search algorithm or trading technology.

And then, as with academia, you have non-financial compensation; for example, we got a lot of people interested by offering Haskell jobs which are very rare today (at least in stable places); remote work was another plus, as many people have a comfortable situation and don't want to uproot their life for the sake of a new opportunity, and it allows people to arbitrage cost of living differences between cities. But Scala didn't work out for LinkedIn, which is rewriting all of it back to Java, and remote work didn't work out for Yahoo, so YMMV.