Google Sprints Ahead in AI Building Blocks, Leaving Rivals Wary | Hacker News Reader

Google Sprints Ahead in AI Building Blocks, Leaving Rivals Wary

bloomberg.com

131 pointsucaetano72 comments

The way I see it, TensorFlow has already won, even if competing frameworks don't yet see it that way. New ideas and research breakthroughs will spread faster to the framework that has the most users and contributors, thereby attracting more users and contributors, in a feedback loop.

Some of the other frameworks will continue to be used (for example, FaceBook's AI team is unlikely to back away from its commitment to Torch), but the TensorFlow ecosystem and its network effects appear to have reached 'escape velocity' in a winner-take-all space.

New higher-level services and products will be built atop the framework most widely used by the most software developers, almost by definition.

Expect a shakeout: https://medium.com/@mjhirn/tensorflow-wins-89b78b29aafb#.bzl...

EDITS: Added the "new higher-level services and products" sentence.

TensorFlow has kind of won the Python deep-learning community, although frameworks like Keras[0] make it easier to use. While Python programmers make up the majority of deep-learning practitioners, they don't have much penetration in enterprise, which is chiefly JVM and lower-level languages.

Deeplearning4j [1] has won deep learning on the JVM. Nothing else comes close. (Disclosure: I'm one of its creators.) DL4J has the most sophisticated Spark integration, plugs into Kafka and works as a Hadoop job. It does all that while running on distributed GPUs, multiple CPUs and heterogenous hardware. It wraps cuDNN, which is faster than Tensorflow.

Finally, part of our stack, JavaCPP [2], is used by the TF communities as well as other big company projects to bridge the gap between Java and C++.

Dev mindshare in the form of Github stars doesn't equal revenue.

Finally, there are a number of strategic problems companies face when adopting a Google project. First, they are leery of depending on Google when they also compete with it. This is particularly true because Google doesn't open source a lot of its best work, like code from DeepMind. So Tensorflow adopters are getting the leftovers in a way, and they're kneecapped in the AI race.

Secondly, Tensorflow is not platform neutral. It's optimized for TPUs, which are not publicly available. It's a play to attract people to the Google Cloud.

[0] http://keras.io/ [1] http://deeplearning4j.org/ [2] https://github.com/bytedeco/javacpp

Upvoting, because I agree that JVM and .NET shops, like commercial banks and industrial manufacturers, are more likely to use Deeplearning4j (DL4J) and CNTK, respectively, than TensorFlow (or anything else, really).

However, those corporate shops are typically many years behind the state-of-the-art in AI, and in many, if not most, cases, they will likely end up paying for higher-level "AI services" provided by a third party than building new kinds of applications in-house with lower-level frameworks.

My view is that those new AI services will be created by startups we haven't heard about yet, led by developers outside of Corporate America who by and large are betting on TensorFlow and non-Oracle, non-Microsoft stacks.

Google's competitors will of course want to use their own in-house frameworks regardless.

PS. By the way, I think DL4J is awesome. Great job!

(Disclosure: Vonnik's cofounder creator of dl4j)

That's more than likely the valley echo chamber. These kinds of startups are going to continue to appeal to people like themselves getting acquired by google eventually. The same scientists who create these services have zero clue about the sales process in enterprise and can't often figure out how to sell in to these corporations and actually create some kind of long term value. Our differentiation in this space is being able to do EXACTLY that while also adding features that are actually useful. By the time they have, they're usually bored and realize they just want to work on cool problems and therefore: sell.

Most research out there generated quickly is actually useless for products.

I'd rather implement every 5th iteration of a technique that has significantly better results than jump on the feature treadmill.

As much as I want you guys to succeed, I must disagree with this comment, because there are an awful LOT of people in the valley who know how to build scalable enterprise-sales organizations, and "AI service" startups with traction will have little trouble finding and recruiting those people.

I would not bet against the valley's "how to build a giant business" expertise.

We're far from that. Much of what we actually sell are solutions focused on anomaly detection. We actually do most of our business in asia where the on prem market is much larger and people move faster.

What I'm trying to point out here is that the people who run these startups are typically Phds with little interest in building an actual business.

Many of these deep learning startups are built to be quick flips while having fun for a few years building something cool. That in and of itself is fine.

They build something differentiated and valuable, get paid a handsome chunk of money. No one really loses when that happens, it's just different when you want to go to market.

That being said, you have every right to bet against us. That's how many large companies start is with detractors.

90% of startups also fail. We have a different opinion on how it's done. Only time will tell whether we succeed or not.

That being said: Where's your deep learning startup? ;) At least we're trying something different.

There will be room for a good number of niche frameworks, some of which could be highly profitable for their creators/maintainers. I don't disagree with that :-)

My point was -- and still is -- that the competition to become the dominant framework has already been won, by TensorFlow, because it has captured the hearts and minds of the most developers. Most new AI services are being built atop TensorFlow -- not DL4J, nor Torch, nor Theano, nor Caffe, nor CNTK, nor anything else.

Let me finish by sharing a video of Steve Ballmer, former CEO of Microsoft, in which he emphasizes the importance of attracting developers to a stadium packed with Microsoft employees: https://www.youtube.com/watch?v=Vhh_GeBPOhs

There's 2 things I'm not so sure about: python as the dominant ecosystem long term and 2: most people doing machine learning yet.

I'm still convinced most people just star tensorflow "cuz google". I'd like to see a statistical analysis of people actually using tensorflow for their job. I get the developers bit...but not much of it equates to revenue.

Developers don't pay. Period. That devs bit might work for lock in to cloud services (google compute engine). That play doesn't work for on prem though.

What's happening here is you're equating startups/cloud to "the whole world" while ignoring where most of the money is being made and how many people are actually using machine learning. I'd love to see this play out more before declaring a "winner". Still seems too early yet.

Anyways - as I said we'll see who wins and who doesn't. I won't say whether my guess is absolutely right or wrong. I just don't believe in silicon valley's ability to build "hard stuff". I think we're still very much in the hype phase yet. It might return some day though.

On #1, although TensorFlow has well-supported APIs only for Python and C++ as of now (see https://www.tensorflow.org/versions/r0.9/api_docs/index.html ), support for other languages is on the way (search Google for specific examples); moreover, TensorFlow can be readily deployed as a service (see https://tensorflow.github.io/serving/ ) in mixed-language environments.

On #2, I agree: most people will use higher-level AI services developed and provided by third parties.

How about we revisit this in, say, three years, and see who turns out to be right?

"It wraps cuDNN, which is faster than Tensorflow."

TF uses CuDNN, so this is a nonsensical statement.

No, it's not. Why does TensorFlow have separate benchmarks for Soumith's convnet comparison that show it to be significantly slower?

https://github.com/soumith/convnet-benchmarks

It says right there: "Some of the frameworks like TensorFlow and Chainer are benchmarked with CuDNN, but it is not explicitly mentioned"

So yeah, I'm also not sure what you meant by your statement.

But cuDNN itself is faster than those frameworks, so they are doing something to slow themselves down, and my statement stands.

Again, this does not make sense. CuDNN is the highly optimized code to perform a specific numerical calculation (e.g. convolution) on Nvidia GPUs. Frameworks such as TensorFlow or Deeplearning4j can use CuDNN to speed up its convnet calculations, but they don't have to.

In summary, your statement is like saying: "Numpy is faster than Python".

It feels strange having to explain these things to a creator of a NN framework.

That's a false analogy. What I'm saying is not like "Numpy is faster than Python"; it's like "Numpy is faster than some scientific computing that the creators of a Python framework hand-rolled separately."

Some frameworks write their own CUDA kernels for operations beyond what cuDNN offers, and also wrap cuDNN. So certain operations may run faster or slower depending on how you choose to execute them with the framework.

Ok, now we can agree.

Are deep learning and other frameworks really that much more subject to vendor lock in than frameworks in other areas of tech?

People make it seem like whichever framework wins now will own AI for the future, but transfer that logic to web development and you would expect perl and php to still be on top of the world instead of whatever the js framework of the day is.

GPU acceleration seems to be a pretty big barrier to entry for upstarts. But I think that has more to do with the crappy state of tooling than intrinsic difficulty of GPGPU. Cuda showed that GPGPU could be much easier and I think there is still plenty of room for improvement. Compare the difficulty of setting up Apache a decade ago vs Nodejs today for how much simpler servers have become.

Anyway Tensorflow is really hyped right now, but there is no way to know if that will last.

Whoa, I did not write (or even intend to mean) that TensorFlow "will own AI for the future," whatever that means. No one is saying that!!!

I claim only that TensorFlow has won the "framework for numerical computation using software-specified data-flow graphs" space, at the expense of many other competing frameworks (Caffe, Theano, Torch, CNTK, etc.). In other words, if you want to build a large-scale machine/deep learning model (e.g., with tens or hundreds of millions of parameters) and you need to train this model with, say, tens of billions of samples in a cluster of multi-GPU machines (or TPUs), TensorFlow has become the go-to software infrastructure for doing that.

In all fairness, PHP still does dominate the field - it routinely picks up new web developers, is the backing language of some of the largest well-known, simple, and open source CMS (WordPress, Magento), and inertia alone places it at ~80% of websites.

Is it new/shiny? Nah. But while other languages/frameworks may get the lion's share of attention for reasons that circulate performance, concurrency, and/or scale and large companies do adopt them from time to time... PHP enjoys ridiculous levels of usage.

And this is from someone that dislikes and never uses PHP.

This is silly. There have been so many "wins" in AI.

Get away from the hype. Go to an AI conference like IJCAI or AAAI. See what is used there.

I think GP was specifically referring to GPU accelerated linear algebra a.k.a. deep learning frameworks.

It's been less than a year, eh. A bit early to declare it has "already won"? There's decades of research ahead of us.

How many of those decades will be human driven?

Barring some Einstein/Shannon-level of momentary insight that catapults understanding of the nature of intelligence forward by orders of magnitude from where it is now, likely all of them. And entirely likely that it will be more than decades.

Since no one even in concept has figured out how to give an AI motivations or goals other than ones specified by humans: all of them.

Surely someone would have tried if "mind building" was in anyone's monetary interest.

People have tried. No one's succeeded. Not every endeavor is a success.

Can you supply a list of monetary interest groups for building sentient minds?

It's a little early if you ask me, IMO you win once you make the first killer product that changes the market Eg: search, iPhone, self-driving cars etc

Google wasn't the first search engine.

I said first killer (market defining and expanding product) product, iPhone wasn't the first smartphone either, but they pushed them from cool towards ubiquitous

I remember trying out Google in the early days and deciding that it is about the same quality as other search engines. I might be wrong, but I would say that it's continuous improvement that gave them their current position. Later that history was kind of rewritten that it was the genius invention of PageRank.

> I remember trying out Google in the early days ... it [was] about the same quality as other search engines... continuous improvement that gave them their current position... history was kind of rewritten that it was the genius invention of PageRank.

I had the same thought too. If memory serves me right google was not magnitudes of scale better than other engines just better marketed, more accessible, and easier to use(interface).

Even now, despite Googles distinct capabilities, I almost exclusively use DuckDuckGo as I find that it is good enough for me, and doesn't leave me spooked after a search with its insidious snooping. So much for 'Don't be Evil'

Not having ads and bs all over the page helped a TON too.

I remember when this was the big debate as to which site was better: http://www.madisonavenuejournal.com/images/crystalball-7.jpg

Alfa Vista started as a clean interface too. But their failed monetisation efforts (ads etc) on the search home page I seem to rember triggered me to move to Google.

That is fascinating to me. I feel like there have been numerous instances of design having a huge impact on success, but many of these instances have been overlooked because people attribute success to other factors. I always assumed Google won because it was b was better technically, but to think that it was due to other reasons is very interesting.

I think it was due to having a sales model that worked, and a decent design. In ye olden days, Altavista and Yahoo were alternatives. Yahoo faded fast, as it was more of an aol type portal, and the business model (which required a large sales team and heavy analytics to price the ads) wasn't as friendly as Google. Altavista died with DEC. Google continued to print money without hiring a huge sales team.

Just to clarify, I am not saying Google wasn't better at that time. It's just that subjectively they weren't better for the queries I tried. They might have been a little bit better, but kept improving until a point where other search engines can't keep up anymore.

> winner-take-all space

What's the reasoning behind the claim for AI frameworks to be a "winner-talk-all space"?

I think the TensorFlow the API has indeed won. TensorFlow the Machine Learning Engine not so much. It's not derisively referred to as TensorSlow(tm) without cause.

Number of stars on Github is not a great metric of the health of a project. TensorFlow, in particular, has received a lot of attention, so a lot of people have starred it, but this does not measure the number of non-Google contributors or what would happen to the project if Google abandoned it.

Note that I'm not saying TensorFlow is going to fail in the future; I'm saying that the measures used in this article are dubious at best.

Does nobody remember Elefant, Alex Smola's machine learning library that had a lot of hype then suddenly died after he left NICTA? The machine learning world is littered with tons of dead projects, and many of these died without a lot of warning. I'm concerned that this effect is only going to get worse now that machine learning libraries are often company-controlled instead of academic.

Hi there, I wrote the article. We also looked at data on things like number of forks and contributors on GitHub as well as massive rise in questions on StackOverflow (thanks to Delip Rao's great post here http://deliprao.com/archives/168 and invaluable ecosystem analysis from Francois Chollet). These other things were cut for purposes of length and because we felt a mainstream reader (which is more Bloomberg's audience) would be able to understand the 'stars' figure most easily of all metrics.

Obviously I agree with you that this measure isn't definitive, but we felt it was relatively easy to understand. And the rest of the reporting we did for this story bore out the idea that TF has gathered an unusual level of both enthusiasm and commitment in a short amount of time.

Thanks for reading!

good points, though I would argue that tensorflow fundamentally is a way to define transformations. Think of it like assembly language. There are higher languages that expose higher level constructs (see keras)

Agreed it's not a great metric, but it is a good proxy especially if you suplement it with another source, such as the number of HN front page articles discussing TensorFlow. Given those two signals, I imagine Tensorflow will continue to gain momentum, at least in the short term.

However, like all SDKs (is that still a thing?), it can be easily be supplanted when something that is easier to use, fixes the flaws of, and is better documented comes out. This is doubly true given that the AI space is still (imo) in its infancy, wrt to developer tooling.

Politicians quote the number of "Likes" they get on social media, too. Stars are no different. It's a measure of interest, where the error bars are quite significant.

The article makes the argument that, by opening up TensorFlow, other companies are at risk of having their TensorFlow-fluent employees poached by Google.

Trouble is, if other companies are using the same tooling as Google, it's equally easy for the outside company to poach a Google employee.

Opening up the code to the world is more likely to be a net-good act, propelling everyone forward.

https://github.com/tensorflow/tensorflow/blob/master/LICENSE

The hard parts of these frameworks isn't the framework itself. The hard part is understanding what's going on; the same properties common to all frameworks.

CNNs are hard to understand. Understanding why gradient descent improves the model is very hard to understand. And understanding the right architecture to solve a problem takes a significant chunk of your research career unless you're already an expert.

But languages and frameworks are simple. If you know Caffe, Torch will be easy to understand. If you know Torch, Tensorflow will be easy to understand. If you know Tensorflow, the others will be easy to understand. It's quite possible to get up to speed with a different framework in a few weeks.

In the corporate world even a few weeks of ramp-up time can create lock-in and network effects. Most programming languages can be learned in a few weeks by an experienced programmer, but there are still huge network effects. Part of it is that a lot of times you just don't have a few weeks to learn. But the other part of it is that the more popular it gets the library and tooling ecosystem also starts growing exponentially, so then you have to add the learning time of the tooling and library to the switching cost.

I think the network effects have more to do with clueless recruiters trying to tick off a list of keywords for a position, than the actual cost to a company of an engineer ramping up on a new framework for a few weeks.

> it's equally easy for the outside company to poach a Google employee.

The effect doesn't become symmetric until we reach an equilibrium where TensorFlow is ubiquitous.

> Opening up the code to the world is more likely to be a net-good act, propelling everyone forward.

These things aren't contradictory. It can be a net good for the world, and also a net good for Google, and also a net bad for Facebook.

It's interesting that none of the quotes from Torch and Theano folks indicate any shortcomings of Tensorflow, their sole counterargument is that using tensorflow might give Google more control.

Hiya, I wrote the article. What might not be captured here is that this is above all a community of people all working on AI so it's not like anyone is really emotional and/or aggressive about this issue. It's more that it's always better for an ecosystem to have multiple dominant players who are all relatively equal. Torch & Theano people were saying it's non-optimal if TF becomes the majority framework, which is a fair point. Everyone I spoke to for this article said TF was engineered well, albeit a bit of a black box in certain aspects.

Some people are noting that TensorFlow evens the playing field, allowing other companies to poach Google's employees. Whilst that is true to some light degree, there are amazingly large caveats - Google still have the major advantage when it comes to poaching. The other important note is that open sourcing TensorFlow is likely more important for attracting recent graduates in machine learning than it is for attracting employees of other companies.

Most machine learning engineers and researchers get excited by two things: underlying hardware and underlying tooling.

Google have enough GPUs to throw 50 K80s for a few weeks into the hands of a single researcher for a single paper[1]. That's impressive. It gets even more impressive considering Google are making custom made ASICs that are more efficient - both power and computation - than GPUs too. None of that hardware is open source and even if it was, I don't see anyone getting them into general production in the near future. Nervana[2] are the closest to custom hardware I know about yet still have many steps until production.

TensorFlow is also just the beginnings of the tooling Google has internally. The original TensorFlow whitepaper talks about their internal distributed training stack. A tiny recreation of this has reached the wild but it misses many of the interesting optimizations. (1) Automated efficient allocation of tensor operations to components (does this op go on GPU? CPU? Across the network to another GPU machine?). (2) Minor optimizations, such as truncating 32 bit floats to 16, then re-expanding - they noted it in the paper but I've not seen external verification. (3) Google DeepMind moved to TensorFlow but given they do a lot of reinforcement learning, they likely have a modified TF internally that's better tailored for RL. (4) etc etc etc

Given the majority of the contributors to TensorFlow are also from Google, that gives them the strongest say in the future of the framework. This is especially important given that Google would likely be trying to ensure their internal modifications and variations of TensorFlow remain in line with the open version. For that reason, I don't expect to see TensorFlow handed off to a third party caretaker any time soon.

Releasing TensorFlow is a net win for Google, both in "making the world a better place" and "making Google a better place (for attracting talent)".

[1]: https://arxiv.org/abs/1607.05691

[2]: https://www.nervanasys.com/

I'd really like to see what Apple is doing, they usually come late to the table but they get a lot of things right, even though they may not be the most technologically advanced. Granted this may not be applicable since this is more about AI infrastructure.

This is actually where I'm worried Apple is falling behind, _because_ they are a bit private & come late to to the table. Microsoft, Google, and others very openly share and learn from the AI community, but Apple has been known to be only a "lurker" at best (for lack of a better term.)

They have almost constantly innovated by acquisition. That explains their lurker status, I think.

They're also very secretive. They only say they do something new publicly when they have a product to show it off.

Brett Victor mentioned Apple experimenting with many futuristic interfaces, but not developing them further after realising they didn't work from a UI pov.

I suspect they may have similar feelings about AI. Many 'clever' systems which aim to predict what the user wants can be frustrating in practice. People prefer tool-like UIs where they can learn the mapping between their actions and the software's behaviour.

Personally, I think there should be more focus on AI augmenting human capabilities, and maybe Apple is working on something like that.

Well, I doubt they are going to open source a machine learning framework to rival TensorFlow anytime soon, but they recently added CNN API's to iOS and macOS, so you can do optimised inference on Apple devices.

https://developer.apple.com/videos/play/wwdc2016/715/

I'm kind of curious too. They've got the revenue to fund it, but I wouldn't say "theoretically advanced software" has historically been Apple's notable quality (subject to a few periods and exceptions).

Incredible design, hardware engineering, and product fit, yes. But the software typically seemed a more polished version of mainstream ideas.

Agreed. Took Apple a long time for their cloud based services to work out their kinks.

Apple "worked out the kinks" of iCloud by outsourcing it to actual cloud providers. They may become a cloud services powerhouse but they are way behind today.

Maps too, they're adding some really cool features that google doesn't have

I definitely agree with your point when it comes to consumer electronics and software, but developer tools/services? I'm not sure. Their investment in AI technology so far looks very small as well.

Apple is not really a leader in software the same way they are in hardware.

"If the company changes the software too much, then other companies that have adopted it will need to make a copy of the software and rewrite it to suit their own needs -- an expensive and time-consuming process known as forking."

Seems like it's the opposite? Forking a project is technically easy. Writing something equivalent from scratch is much harder.

(Of course there are marketing considerations - it's "just a fork" so it might not get much attention.)

While Google may be the leader in Machine Learning tools, they are way behind when it comes to Machine Learning services.

They've only just now released a Natural Language Understanding service in beta, and it is more limited compared to other NLU services from Microsoft/others.

While tooling is important, the market for low level tools is much smaller than for the services built using those tools. The vast majority of businesses who could benefit from Machine learning don't have the expertise to run RNNs using Tensor Flow, but do have engineers who can integrate API's that leverage trained classifiers.

> they are way behind when it comes to Machine Learning services.

Umm, there's five separate managed services within the Google ML family: Vision (GA), Translate (GA), Natural Language (Beta), Speech (Beta), and Cloud ML (Alpha)

https://cloud.google.com/vision/ https://cloud.google.com/translate/ https://cloud.google.com/natural-language/ https://cloud.google.com/speech/ https://cloud.google.com/ml/

The market for services is several orders of magnitude less than the market for products that make use of ML. It's probably ok if they don't focus to heavily on the services side

Google makes heavy use of these tools internally to build a wide range of products (and enhance existing ones)

How do you see those two as being different?

In my mind, products that make use of ML are using ML services as the back end. An example would be the recent wave of bot companies: many are not rolling their own NLU system but rather leveraging services like wit.ai or Microsoft's LUIS.

Google has done a far better job of abstracting the underlying math of machine learning into vertical-oriented services like the CloudVision API and the Cloud Natural Language API.

In contrast, the AWS Machine Learning service provides a 100% vendor-locked interface to logistic regression and that's it. You can't even import or export models. They just hired Alex Smola to do something about that. We'll see what comes of that.

Reply on news.ycombinator.com