Hacker News Reader

Comment by sevensor | Hacker News Reader

As a former grad student, I strogly sympathize with the urge to get the hell out of there and make real money. But on the flip side, I'm pretty dubious about the idea of abandoning your program just because the grass is greener in industry. The Ph.D. improves your chances of getting to work on interesting problems down the road, even if you leave academia. Selling that out for a few years of big money isn't something that should be done lightly. Especially when we're in the middle of such an obvious bubble -- what's the plan for 2-5 years from now when it pops?

View on HN

So your argument is they should stay and work for pennies, hoping to get an "interesting" job in the future, instead of getting an interesting and highly paid job right now?

It's also not obvious at all it's a bubble. AI field is already much more complex than the regular CS field, which already suffers from talent shortages. AI is the future, and the demand is here to stay.

AI is currently less complex than many CS fields that pay far less.

This is basicly the boom times when most companies want a website but don't have one. Once the obvious applications are built these bubbles tend to burst as matence pays far less.

Plain vanilla convnets are not very complex. If you look at recent papers on Deep Reinforcement Learning with extra modules or Generative Adversarial Networks from DeepMind and the likes, the complexity is much higher than much of CS used in industry and it is increasing rapidly.

More importantly, the performance of many AI implementations could be improved with the increased sophistication, so the demand for such complexity is there for businesses in competitive sectors.

Any examples of topics in other fields of CS in practical use that are more complex than recent AI advances with potential practical applications?

I would argue a modern Graphics pipeline is significantly more complex. But, you run into issues of what the borders of a topic is. Lighting for example is a huge topic for research, but it does not mean much on it's own.

If universities believe that AI is here to stay, offer tenured positions now to your AI scientists. Universities cannot compete on pay (and I don't think this is bad), but they can compete perfectly well on job security. So play this trump card (no pun intended).

> It's also not obvious at all it's a bubble

It's breathtakingly obvious:

1. Large numbers of people getting poached from academic programs by industry.

2. "This time it's different" is what we've said during every bubble ever.

3. Lots of money chasing a few ideas.

The only thing that's not clear about this bubble is when it's going to pop. People who didn't see the potential of the internet in 1996 missed out. People who got into it in 2000 were left holding the bag.

I think you are missing the scale mentioned in the article:

The industry is paying five times what universities are paying. Not Fifty percent. Not double. Five times.

Professors/administrators whining about it are pure scum unless they themselves are making one fifth what they would be making in a different university that offered them a job purely because they love this specific college doing the specific research or teaching.

What is really pissing off these professors is that with their worker bees gone they themselves would need to do the work - something they are not quite used to.

> I think you are missing the scale mentioned in the article:

> The industry is paying five times what universities are paying. Not Fifty percent. Not double. Five times.

Nope, not missing that. You don't think this happened in 1999-2000?

I agree with you that grad students are ruthlessly exploited. I've seen that firsthand. I routinely discourage people from doing a Ph.D. for that reason. And if you're not too far along, taking a terminal master's and leaving for industry is a solid option, especially because this AI bubble won't last forever. But if, like the one student in TFA, you have just one year left, quitting grad school is not a great idea. Now you have, at best, an awkwardly long stretch of grad school on your CV with a master's degree on the end of it, and at worst, no degree at all to show for your pains. At that point, just suck it up and finish the degree.

> Nope, not missing that. You don't think this happened in 1999-2000?

I do. I'm yet to see people who jumped away from academic CS tracks at the time being upset about their decisions. I do know of dozens degreed people in their late thirties reporting to C students while making quarter of the salary of the same C students.

CS and AI are different subjects. As a CS dropout, your job is to write code. You can get quite competent at that without a higher education course.

You can't do that with AI. The amount of material you need to get on top of is really something else, especially if you want to be able to understand the foundations of what you're doing- which stretch back several decades before statistical machine learning, popular as it is today.

It is very naive to think that AI programs in most of universities is anything other than writing code and pretty lousy one.

Do you know that it was possible to get a PhD in genetic sequencing not a very long time ago from very serious schools? That's right, what today can be done by a random student or a random outsourcing shop in a third world country used to be a PhD worthy specialty.

In 5 years today's ML/AI PhDs would be in the same peculiar position as PhDs of genetic sequencing.

>> It is very naive to think that AI programs in most of universities is anything other than writing code and pretty lousy one.

Are you speaking from experience? Because my own experience from my degree and Master's AI courses and lectures is completely different to what you describe. Particularly during my Master's, code was only incidental; the meat and potatoes of each and every lecture was theory.

It depends of course on the quality of the teaching in a given institution, but, for instance, you can find here the lecture notes from Oxford's Deep NLP course:

https://github.com/oxford-cs-deepnlp-2017/lectures

As you will see, it's anything but just code, or lousy such. Of course, that's Oxford, their teaching is world-class- but most AI courses I've found on the internet are similar in scope.

Really, it's pointless trying to teach AI with "just code" - because in AI you can't really code much, unless you understand the theory.

>> In 5 years today's ML/AI PhDs would be in the same peculiar position as PhDs of genetic sequencing.

I think what you're saying is that in 5 years, people knowing Tensor Flow or Torch will be a dime a dozen.

Perhaps- but someone will still need to figure out some way to acquire new knowledge.

Also, it bears repeating that AI is not just statistical machine learning and statistical machine learning itself is not just deep learning. AI is a broad field with communities focused on many different subjects. Just as an example you can find information on, easily, try probabilistic programming.

Back a decade or so ago, the only way to really reliably get access to current research was to work at a University or a large company with journal subscriptions.

These days it's really easy to get all the information you need to teach yourself whatever you need to. If you have problems, ask the Internet for help.

And these days, if you want to work on the actual leading edge, it's likely at Google, not at a University.

>> These days it's really easy to get all the information you need to teach yourself whatever you need to. If you have problems, ask the Internet for help.

The information may be online but it's not organised in any way and if you're learning on your own, first you have to figure out what information is relevant to what you want to learn. University offers focus, support and guidance that you don't have when you're on your own.

I think a lot of developers treat machine learning as just another technology they can get good at if they apply themselves to it, like blockchain, non-relational databases, VR etc. In truth, AI is nothing like that. You really need to grok some background knowledge before you can do anything useful with it.

You can sure learn to use the free-access tools available today, like TensorFlow or Keras etc, but, for instance, you're not going to just figure out how to roll your own deep learning framework just by reading a few books you found online- like you could do for, say, a web framework. Nor does it mean that, just because you learned how to use a machine learnign framework, you can discover new knowledge and invent new machine learning algorithms, or optimise existing ones.

Which means- someone else always has to be there to do the hard work for you.

I'd say, that's the reason why it's AI PhD students who are being poached, and not just random devs who declare themselves "passionate about machine learning" on their facebook pages. Because the AI PhDs already have the background the big AI companies are looking for.

A PhD lasts five years in the us.

I would argue: what's the tangible difference between someone who has gone through all five years and someone who has gone through "just" four of them?

Plus, i don't think applied research is less important than base research.

>> I agree with you that grad students are ruthlessly exploited.

More so than junior developers? In my experience, "junior programmer" means you'll do ten times the work for one tenth the salary - and everyone will treat you like an idiot on top of that (because, hey, if you were smart you wouldn't be a junior, amirite?).

I do think the exploitation of grad students is a US phenomenon, btw. And the article is about universities in the UK, where I'm pretty sure the conditions are different.

> More so than junior developers? In my experience, "junior programmer" means you'll do ten times the work for one tenth the salary - and everyone will treat you like an idiot on top of that (because, hey, if you were smart you wouldn't be a junior, amirite?).

Still pays more than a grad student.

I thought the point was about exploitation, not absolute monetary values?

Because if we're going for absolute money amounts, we might as well all give up on any sort of career in technology and shift our focus to getting some managerial position in a big corporation, somewhere- that's where the money's at.

It is about being exploited for extremely small amount of money. This is a real world. In real world, people need money. Shit is expensive. Food is expensive. Shelter is expensive. Everything is expensive.

Now it is possible that you are independently wealthy and it is irrelevant to you, but I can assure you anyone who is not independently wealthy would always trade shit job (A) which pays $K for a shit job (A) that pays $K + several thousand dollars, which is the complain of this article

I'm not independently wealthy and I left a lucrative career in the industry to study what I am interested in, in academia, although neither my jobs in the industry were "shit jobs" nor is my current assignment.

So certainly not "anyone" would pick the best paying job, without any other consideration. Generally, people in academia are not in it for the money and their main motivator is not money- if it was, they would not be in academia.

I don't know where you get your ideas about academics from but I can tell they're not coming from experience with academics.

Also I'm not sure where you get your ideas about motivation. In my experience, the majority of people will choose work they find fullfiling over work that pays more but is not as interesting. Example: the typical developer prefers to be hands-on with code than become a manager, even if managers make more money than developers (in most companies).

It may appear obvious to you but you don’t really know that there is a current AI bubble. There’s an economics adage that seems apt:

Economists have predicted 15 of the last 10 recessions.

>3. Lots of money chasing a few ideas.

That's the real problem. There are only so many fundamental things you can do with deep convnets as function inducers.

There's a lot of money to be made by applying those to a thousand different things, but even that has to taper off at some point.

I suppose it's limited by the size of 'the set of things you can do with functions'. This seems like a large enough set that I wouldn't be worried about hitting its limits too soon...

More like "the set of things you can do with functions, where you've got a strong prior on the functions being things like spacial mappings with certain invariances." So basically more like, "functions for vision problems."

You are absolutely right that a lot of work to date (especially in applications and metrics-driven research) has focused on 'functions for vision problems'. But this is far from the only idea - that's the point I'm trying to make.

For instance what about learning value functions for reinforcement learning (e.g. AlphaGo)? Or natural language processing? These are definitely not vision problems, or if you believe that they are then 'functions for vision problems' is actually a pretty huge class!

The universal approximation theorem backs up my claim [0] - we can approximate arbitrary functions with neural networks. I think this theorem is overemphasised in practice: we don't generally want to approximate arbitrary functions, we _want_ to encode specific prior information into the function we approximate, as you rightly say. But that doesn't mean that we have to do so, or that we only have one idea about what functions to encode, or even how to encode them.

[0] https://en.wikipedia.org/wiki/Universal_approximation_theore...

>The universal approximation theorem backs up my claim [0] - we can approximate arbitrary functions with neural networks. I think this theorem is overemphasised in practice: we don't generally want to approximate arbitrary functions, we _want_ to encode specific prior information into the function we approximate, as you rightly say. But that doesn't mean that we have to do so, or that we only have one idea about what functions to encode, or even how to encode them.

I wouldn't say we have only one idea about functions, but I would say I haven't seen much of an active pipeline, outside maybe DeepMind, on coming up with new kinds of priors over functions that we can apply to larger-scale or more structured tasks. At some point, applied deep learning will run out of steam, and someone's going to have to go back to doing basic research.

That someone may find, as many have, that in terms of sample efficiency and transfer learning, deep neural nets are not always so great.

> Large numbers of people getting poached from academic programs by industry.

You say this like someone who thinks academia deserves those people and industry is "stealing" them.

The market for autonomous vehicles (land, air, water) alone could become larger than the entire IT industry in 2000.

When you include robots for industry (e.g. Sewbots, Warehouse robots, ...) and household (e.g. Aibo, Amazon Echo, ...) it is clear that the markets are potentially much larger than their current sizes and will grow with improvements in AI.

Recent papers in AI and Deep Learning have shown that it is possible to train robots to adapt to unpredictable environment and human interactions. This will realize many new potential applications in the physical space over the next decade.

Although the fundamentals are the same, the variations are legion and could become highly complex as the environment dictates. It takes a few years at least to become an expert in the field with experience in practical applications. Not everyone possesses a cognitive toolkit necessary to become one either. There will not be an oversupply of experts in AI and ML within the next five or even ten years.

Investors were left holding the bag.

The rest of us just kept working and earning.

We’ve had AI winters before, who knows if the next one isn’t coming.

Opportunity Cost is one factor that’s overlooked by nearly every student.

https://en.m.wikipedia.org/wiki/Opportunity_cost

It’s true that a PhD is valuable for your career. Also, grad school is a special, and hopefully enjoyable period of your life (and therefore valulable).

On the other hand, you are not only forfeighting the earnings that you could be making —- you are also sacrificing the difference in growth of those earnings. For example:

Option Grad School: $0, $0, $0, $0, $200, $210, $220, $230 = $860

Option Worker Bee: $100, $110, $120, $130, $140, $150, $160, $170 = $1080.

Tweak the numbers however you want. In particular, many good non-PhDs soon prove themselves and earn just as much as PhDs each year.

Don't forget about time value of money! That $230 you'll earn 8 years from now is worth less than $200 today if you account for 2% inflation. (And in reality, you tend to get much better returns)

And the value of time with money.

Not wasting your 20s being poor, eating microwave noodles and doing someone else's research is worth something.

I see your point, but I don’t think that’s as clear as the time value of money.

For many people, grad school is where you form some of your best life-long relationships, in an environment where you’re surrounded by like-minded people. Many people would tolerate ramen noodles and free pizza for that trade.

That's because they don't know that they have any alternative.

The situation that professors and schools would love to continue indefinitely.

Your claim is that grad students do not know that they could be working instead of going to school? That strikes me as a weak argument.

My claim is that the grad students do not know how much money they would be making and what kind of problems they would be solving if they were to say "See ya later, Suzi" and moved to companies working on those problems. Largely, it is because most of them are young and pretty street-dumb. That makes them naively believe that schools, professors and advisers have students best interests at heart, none of which barring a few exceptions can be further from the truth.

Schools and especially professors need the cheap labor and cheap ideas provided by the grad students. The age old adage "Those that can do, those that can't teach" applies. The moment grad students move from making peanuts, dealing with politics, kissing ass of their professors and doing work of their professors to making bank, dealing with politics, kissing ass of their managers and doing work for their managers academia as we know it would collapse.

If what you argue is true, then what do you think is the primary reason all of these grad students are blind to other opportunities? I have a hard time believing it’s because grad students are “young and pretty street-dumb.”

Also, do you have a proposed solution to open their eyes? Or do you think the situation is hopeless?

Without commenting on the verity of GP's claim, co-op is the solution.

> The Ph.D. improves your chances of getting to work on interesting problems down the road

That's how universities can compete -- let you work on the things that interest you. There doesn't have to be a commercial application.

Exactly. Most people without Ph.D.s have to do jobs made up by other people. The Ph.D. obligates you to cut your own path. It's not for everyone, but if it's the kind of thing you want to do, it's liberating.

People cry "bubbles" every time hot money flows into a new field, much like those doomsday prophecies. The whole CS field has been declared as a bubble year after year on Hacker News, yet these companies are doing better and better. Even if it eventually pops, remember, the market can stay irrational longer than you can stay solvent.

Besides, if a bubble exists and it will pop in two or five years, staying in school is worse than getting into industry now. When the bubble pops, no one will hire any new grads any more, but existing workers may survive the chopping block, and are far more likely to reenter the field when it recovers, simply because they have work experience.

> Selling that out for a few years of big money isn't something that should be done lightly.

What? I don't understand.

What stops you from going back to grad school when the bubble pops?

You can happily work for free then and still come out way ahead.

Same as for anyone else 2-5 years of experience in real company, doing real stuff, making real money. That train ended? Migrate to a slightly different field.

What if you're really interested in your subject?

To be fair, if you're not then there's little reason to start a PhD and it makes perfect sense to drop out of it and go do something else...

Btw, it's also perfectly possible to start a PhD after a few years of work in the industry. Not everyone goes on to have more studies right after graduating.

> What if you're really interested in your subject?

Pardon my french, but it is a stupid reason for someone who can barely has any money to support himself or herself. We have empirical evidence these grad students are not stupid because when Google or Apple or Facebook offers them $150k/year they move.

>> Pardon my french, but it is a stupid reason for someone who can barely has any money to support himself or herself.

I don't think there's any better reason to do research than to be interested in your research subject.

Actually, my guess is that this is the real difference between academia and the industry: if you're motivated by money, you'll optimise for making lots of money; if you're interested in generating new knowledge, you'll optimise for that.

I think the plan might be get your 'asters with thousands of dollars and 3 years experience in the field

>Especially when we're in the middle of such an obvious bubble -- what's the plan for 2-5 years from now when it pops?

I don't understand your argument. Wouldn't someone with 2-5 years of work experience, networking, and a much higher salary leave them better off than staying in academia? Either way you're still in the field you think is "an obvious bubble."

Or you know, maybe the world runs on computers these days?

> The Ph.D. improves your chances of getting to work on interesting problems down the road, even if you leave academia.

What's the starting point? 0.01%? Because 100x improvement in that case is just 1%.

> when it pops

Get a PHD.

Reply on news.ycombinator.com