Why Andrew Ng left Google and joined Baidu
venturebeat.com
venturebeat.com
> “Frankly, Kai just made decisions, and it just happened
> without a lot of committee meetings,” Ng said. “The
> ability of individuals in the company to make decisions
> like that and move infrastructure quickly is something
> I really appreciate about this company.”
> “He ordered 1,000 GPUs [graphics processing units] and got > them within 24 hours,” Adam Gibson, co-founder of > deep-learning startup Skymind, told VentureBeat. > “At Google, it would have taken him weeks or months to get that.”
According to Wikipedia [1], Ng built a 16,000-CPU-core system at Google. (It's unclear what the CPU<->GPU conversion factor is)
The hiring of Geoffrey Hinton changed all that with Deep Neural Networks which are easily 50x more efficient on GPUs. And in the meantime, Baidu demonstrated what one could achieve by integrating them top to bottom in their search engine while Google happily stuck its technological fingers in its ears singing la-la-la-la.
Given that Andrew Ng's lab's other achievement was to demonstrate that 3 quad GTX 680 workstations were equivalent to that 16,000 CPU core system at Google, Baidu seems like a really great place for him to go.
So Baidu is 2/3 of Google - it's not just the size.
As a company becomes older, the need to run all kinds of crazy experiments suddenly doesn't matter anymore. They care more about stability and profit. Yes, there is Google X and I know Google still run all sorts of crazy data analysis. But the point is, they want to become more structural because of the number of projects needed to approve. If twenty researchers each request to run a project each takes 1M out of Google, those research better have some formal proposal or so so they can evaluate which gets priority. Also seniority? Two top talents can compete for the same resource but if one team makes more "promising" result than the other team does, the winner is obvious.
Now coming to BD. BD isn't exactly unlike Google. It's exactly China's Google. It has reached its own maturity, but when they want to expand westward and when they want to steal talents from Google, they will have to strike better deals, be it compensation, work freedom and resource support.
I will not be surprised a few years down the road Ng will find similar hurtles within BD once he's all "used up" (sorry for the lack of better word). But of course, what matters is what he can do now.
Quotas suffer from the standard budgeting problem: everybody inflates their needs and hoards resources so that if they need extra in the future, they don't have to go through the budgeting process again. As a result, when a new project comes along that's not just an off-shoot of an existing high-priority project, they find that they're starved out of quota. They don't have an existing quota because the project previously didn't exist, and then when they go to get some, they find that all machines are currently allocated because everybody else overbid to secure room for expansion.
Google was rolling out a fix for this process when I left them, but I'm not sure if I can share the details on it.
But if you don't have enough internal customers, you might have budget shortfalls, so guess what happens for vendors.
> Google was rolling out a fix for this process
I work at a place small enough where the budgeting process is for IT as a whole and we each have our corner to play in of fixed size.
A little off topic, but: Corporations no longer are loyal to employees (unless they are at the top of their field). Some countries now seem much more interested in the interests of corporations than people. Therefore, I think that it is very smart for individuals to more consider themselves as 'citizens of the world' and be very open to working for whoever pays them the most, provides the best infrastructure, etc.
I don't find this to be universally true. I was at Google for five years. When my first daughter was born, they gave me 2 weeks of paid paternity leave. When they upped the policy to 7 weeks months later, they made it retroactive and gave me another 5 weeks to spend with my new family. When I was dealing with health problems, my managers were extremely understanding and put a number of different options on the table for how they could help me cope. When my wife and I decided we needed to move somewhere without a real Google presence, they bent over backwards to find a way to make it work without my leaving the company.
It's difficult to separate "company X is being nice to employees because it is in their best interests right now" from "company X is showing loyalty to its employees," and in general morally upstanding behavior coincides closely with long-term self-interest. But to my mind the way I was treated constituted loyalty. The clincher to me is that Google has a newer extraordinary suite of benefits for the families of Google employees that die. That is a good, loyal, and _expensive_ thing to do, and I have a hard time seeing how it makes sense as part of some cynical calculation. I don't think a lot of employees are going to be attracted to or retained by Google because of its stellar death benefits.
To be sure, I know this doesn't mean Google would never ever lay devs like me off in a downturn, nor does it mean that I would never quit Google. (In point of fact, I did.) But I definitely don't advocate, as you suggest, that developers simply go to whatever employer is paying the most right now. Go to whatever employer has built a reputation for treating its employees the best.
It has been that way since at least the mid 90's. I am amazed at the number of people who still think they need to show where they work any kind of loyalty.
Also if corporations can't show loyalty, a human quality, then how can they be people?
For that I despise him and whoever actively takes part in Baidu's business. And the investors. This is the worst online fraud in the world because it is literally killing people.
[Ng said:] “The ability of individuals in the company to make decisions like that and move infrastructure quickly is something I really appreciate about this company.”
That might sound like a kind deference to Ng’s new employer, but he was alluding to a clear advantage Baidu has over Google.
“He ordered 1,000 GPUs [graphics processing units] and got them within 24 hours,” Adam Gibson, co-founder of deep-learning startup Skymind, told VentureBeat. “At Google, it would have taken him weeks or months to get that.”
Weeks or months? I wouldn't expect Google to get them in 24 hours, but has it really gotten that bureaucratic there?
I'm not necessarily disagreeing with any of the points here- there are just many different ways of looking at these data points.
[1] http://www.wired.com/2013/05/neuro-artificial-intelligence/a...
Imagine 16 different groups all want 1000 GPU's, each decided they want a different type. Ignoring even the GPU's for a second, somebody has to support programming them (now you get a whole bunch of incompatible GPU toolchains), etc.
Especially for something expected to be long term, it would be stupid to let this happen en masse without at least some different folks sitting down and talking about it, which is red tape.
Granted, in Google terms that's pocket change, but still, I'd imagine there is a rigorous requirement for a solid business/project plan before management sign the dotted line.
Back in the day I looked at using a neural network for an experiment to measure the efficiency of toilets that would have been £250,000 in 1982 just for the hardware - that was a bit to much.
Though back then for a single piece of HP test gear we spent around a 100 grand.
And even amongst my parents' generation, my uncles who have gotten PhD and tenure at US universities and left for universities in China. The reason being that they can get paid for just as much as in US, for lower living cost, a more compatible culture and in Beijing or Shanghai for better standards of living (yes before the critics jump in with air pollution, human rights issue etc., a good US wage in China could afford you a full-time personal assistant and/or nanny to do your laundry, take care of your kids, clean and cook good Chinese food and you can live in a nicer part of the city with better infrastructure than most of US lol; and you could still come out with lower cost of living).
Ng is Singaporean (born in the UK), not Chinese. He has no connection to the PRC apart from his ethnicity. He may not even speak Mandarin.
> Beijing or Shanghai...lower cost of living
You've gotta be kidding me. I know people who specifically won't return because it's too expensive to live in Beijing or Shanghai.
He was born in the UK, educated in Singapore, then at CMU, and then at MIT and Berkeley. There's more, but the point is: you're projecting.
If we put health issue aside, China's living standard isn't exactly that low, housing price is crazy... and you are a foreginer, you won't go to public school. You will go to international schools and they aren't cheap. But if they earn the same wage as they do back in US, like at least 100k a year? sure, maybe.
Otherwise, I think one of the tedious part of living in China is applying licenses and IDs. According to a research [1], a Chinese citizen may need to apply between 74 to 103 different kinds of ID, license and certificate in one life time.
It's definitely hard to do as a foreigner without a package, international schools are super expensive.