Comment by AndrewKemendo | Hacker News Reader

Comment by AndrewKemendo | Hacker News Reader

Parent Full thread

This is largely because leveraging power on the Internet requires technical expertise, and most distributed power groups don’t have that expertise.

This is actually my biggest concern.

Our company is bleeding edge technology in AR and ML and it's really fucking hard to hire computer vision and machine learning people because Goog/Face/Apple can offer 300-400k for PhDs and give them exciting and engaging projects.

If giant companies constantly "disrupt themselves" and acquire anything that looks like competition then they just consolidate further and nobody can take a competitive position.

I know people will hand wave this away, but as RL becomes the architecture for software - especially led by these big companies - it's going to be increasingly hard to unseat big players.

At the AI meetup at Stanford I went to last week, Andrew Ng was asked by Jen-Hsun Huang, "How do small companies create the new technologies for the future better than the big ones" and Andrew's comment was basically: "They don't anymore."

I'm worried that the time for disruptive startups (at least in software) is coming to an end. Hopefully I'm wrong.

You raise a fair question, and I think the answer partially lies in the definition of "compensation."

Can a startup afford to pay market rates for PhDs and other qualified individuals? Probably not. That's always been the case though, and I don't really see that as being new for computer vision and machine learning talent.

So you need to look at what else you can offer them. A significant amount of equity seems like a big one. It may be more than you wanted to give, but that may be the reality of the market. Beyond that, since risk is significant and fewer and fewer liquidity events seem to be occurring, perhaps a straight profit-sharing model might be more appropriate and more immediate in terms of its ability to balance the risk equation of startups. If you start getting customers and someone knows they'll stand to make money as you grow vs. waiting for some increasingly unlikely IPO, that could be more favorable.

There's also other less tangible things like flexibility that comes in the form of remote work, flexible hours, etc. that can have very real value to people. Taken to the extreme, you may be able to land some awesome talent remotely in other countries where maybe what you can offer them is still a lot of money.

It would be interesting to know what you are trying to do to remain competitive in these other areas. Money solves a lot, but depending on what someone is looking for and where they are in their life, it may not be everything.

That's always been the case though

It's true, however the difference in talent is staggering between someone with experience in CV/ML and someone who has only studied it. So I suppose that just stating "PhD" is too vague.

Your point is correct though about giving other benefits, and that's what we (and others in the field) do - equity, flexibility etc...

Understand though that the key issue here is that most of the people with these skills are rabid about working on big projects with grand vision, and rightfully so. Part of the problem however is that there are only so many data sets for these problems and without those data sets the pace is slow.

The frontier today requires much more infrastructure than previous breakthrough projects. I was just watching the "rise of the nerds" and it was illuminating how "simple" even back then it was from a scale perspective to make great progress. The microprocessor really opened up this whole ecosystem because it was something that anyone could buy. That's just not possible with machine learning because you have to have so much data and that kind of data you can't buy, you have to build it.

I was talking with a major heavy hitter in this field (one you've heard of) about his work options this year. He was debating leaving a very well funded billion dollar company to go to one with some of the best people in ML in the world. He didn't leave because the company he was working for already had terabytes of vision data to work with and infrastructure that made his work possible. The newer company, which had amazing talent and some great funding, would have to start gathering data and it would take time to set up their GPU clusters etc... So it was because the bigger company was already at scale with data and could gather data faster, that he didn't leave. And these are the cream of the crop people with a lot of money we are talking about here.

So it's not so much compensation, it's that the biggest companies have the scale needed already to do the breakthroughs in tasks like reinforcement learning and machine vision - the tasks that are going to dominate the landscape going forward - that are impossible for startups to get to.

You are conflating technical and financial power. If your company has bleeding edge technology, and the only thing you want is to see it widely adapted, open source or freely license all your software, so a large number of small companies can build things using it. But (my guess is, I don't know what company this is) you won't do so because you want to leverage the financial power of that knowledge. In that case, why begrudge the Google researchers their 300k?

An example of a truly distributed purely technical power group would be something like Debian GNU/Linux. One of the reasons it's possible, and getting easier by the day to run desktop software that isn't locked down by one of the big corporations is the existence of distributed projects like the kernel and Debian.

I'm not conflating it, they are related. I had this argument at AGI 14 where someone was trying to argue that it was possible to make disruptive FOSS to the same scale as Google. I just don't believe it. Why? People have to eat. If you want to compete, at the impactful-change the power structure level, you have to pay people real competitive wages to work full time.

What you say is anathema to everything else written on this site about how to build good products with good people that have impact.

Otherwise you wait around till the next gauss/perelman shows up that's only working for passion.

Aren't there problems where large scale data sets are not available, even for the big players?

With small amounts of data and/or unconventional learning situations, there should be options for incorporating domain expertise in interesting ways. I might be wrong, but I think the fraction of problems that can be solved with deep learning/large data set analysis is a small fraction of the set of AI problems as a whole.

For example, at a statistics symposium at MIT last year Stanford Professor Robert Tibshirani mentioned a healthcare (cancer diagnosis) problem, done in collaboration with healthcare people: http://www-stat.stanford.edu/~tibs/ftp/canc.pdf (these slides are different from the ones used at MIT, the content is similar though). He mentioned how they talked to some deep learning experts at Google, who told him that the data set was too small for effective deep learning usage. Another reason is mentioned on pg 30 of the above, basically sparsity being more important than nonlinearity here.

In healthcare, people care a lot about "interpretable" models. The term is ill-defined, but in a sense it is something where the trained model is easily explainable, and easily written down. A linear model is a simple example of such a thing, though usually insufficient on its own. Deep neural nets arguably do not meet this, hence a reluctance (among other reasons) to shift in certain fields.

Your point may still be valid though - I do not know whether there are enough viable economic opportunities. Computer vision in particular is dominated by DNN/large scale analysis these days, and I sympathize with your difficulties here.

I think the counter point is Watson's application to healthcare:

http://www.ibm.com/watson/health/

Your point is valid though about there being silos of industries that don't have a lot of data, or the data not being accessible.

I think a great example of this is in Agricultural Tech. There is just incredible amounts of data that is totally siloed and fractured. But if you pay attention Google Ventures is right there, ready to fund or acquire whatever is happening in the space.

Out of curiosity, how about hiring masters or bachelors that have substantial projects, but might not qualify as research?

It's personal interest, I don't have a degree at all yet, but I understand convolutional neural networks fairly well. Stanford has a good number of open classes on such subjects.

Otherwise, you probably have to give them something they can't get at Google; autonomy, environment, time off, whatever.

Otherwise, you probably have to give them something they can't get at Google; autonomy, environment, time off, whatever.

Not to harp on what we do too much, but that's all part of what we offer. Remote, minimum vacation (meaning we set a mandatory 10 day minimum but no higher bound on vacation day), equity grants (not options) etc... to make up for not having 400k salaries.

As I mentioned elsewhere though, with most people money is what tips it over, but not they key thing. It's access to data that is important - it's why LeCun went to Facebook and Ng to Baidu. That's not something you can get as a startup without a product at massive scale already (though we have some staggeringly large proprietary vision data).

Since it's PhDs you're after, maybe you could offer something along the lines of a small lab that they can pursue their own projects in for 1-2 days a week and establish that you encourage them to publish research.

I guessed, but didn't mention it since it seemed somewhat unlikely that you would have more or better data than Facebook/Google/Etc.

Right, nobody does. That's why it's such a challenge.

Reply on news.ycombinator.com